Data Engineering Manager
Posted May 14, 2026
Data Engineering Manager
Role Overview
We are seeking an experienced Data Engineering Manager to lead the technical evolution of our Databricks Lakehouse. You will be responsible for the architecture, scaling, and reliability of our data products, ensuring that the platform provides high-quality, actionable data for retail and supply chain analytics. This is a leadership role that requires a balance of advanced systems design and hands-on operational rigor. ### Key Responsibilities
🔹 Architecture & Platform Ownership
-
Lakehouse Strategy: Own the end-to-end design and optimization of the Databricks Lakehousearchitecture, leveraging Unity Catalog and Delta Lake for multi-tenant isolation. - Scalable Ingestion: Drive multi-tenant ingestion strategies from Azure Table Storage, Cosmos DB, and APIs, focusing on watermark logic and idempotent pipelines. - Framework Development: Lead the evolution of in-house PySpark-based ETL frameworks to support complex transformations and downstream consumption. #### 🔹 Operational Excellence & Data Observability
-
Data Quality Validation: Implement comprehensive data validation frameworks and automated checks to ensure accuracy, completeness, and consistency across the Lakehouse. - Observability & Monitoring: Drive the strategy for end-to-end data observability, using monitoring and alerting to proactively identify pipeline drifts or failures before they impact stakeholders. - Improve cost transparency through DBU tracking and compute profiling (Serverless vs. Classic); drive performance tuning via clustering and partitioning strategies. #### 🔹 DevOps & Technical Governance