As a Senior Azure Data Engineer (Databricks), you will be responsible for independently designing, developing, and optimizing cloud-based data solutions. Your expertise in Azure Databricks, Data Lakehouse architecture, ETL/ELT development, and data warehouse modeling will be crucial for this role. You will be actively involved in building and enhancing pipelines, migrating enterprise data (SAP/SQL Server) to the cloud, and implementing scalable solutions using Azure services. Initially, this role will be development-heavy (100% development), transitioning to around 60% development later on. The position is based in Gurugram with a shift timing of 2-11pm. **Key Responsibilities:**
Work on Azure Databricks pipelines: enhance, modify, and optimize existing pipelines and develop new ones when required. - Handle data migration from SAP/SQL Server and enterprise databases into Azure cloud environments (Lakehouse, Delta, Synapse). - Configure, manage, and optimize Databricks clusters, compute, workspaces, and resources for performance and cost efficiency. - Build scalable ETL/ELT processes using PySpark, SQL, Delta Lake, and medallion architecture across BronzeSilverGold layers. - Work extensively on Azure Data Factory: orchestration pipelines, triggers, monitoring, and integration with Databricks. - Design and maintain data warehouse & dimensional models (Facts, Dimensions, SCD, Star/Snowflake schema). - Ensure strong data quality, validation, governance, and security including Unity Catalog, RBAC, and Azure platform controls. - Develop solutions independently, involving end-to-end development, not just support (initially 100% development-focused). **Qualifications Required:**
Azure Databricks (Notebooks, Clusters, Delta Lake, Optimization, Resource Configuration)
Azure Data Lake / Lakehouse Architecture
Azure Data Factory (Pipelines, Orchestration, Integration, Monitoring)
Azure SQL / SQL Server
PySpark, SQL for transformation and performance tuning
ETL/ELT development (end-to-end implementation) As a Senior Azure Data Engineer (Databricks), you will be responsible for independently designing, developing, and optimizing cloud-based data solutions. Your expertise in Azure Databricks, Data Lakehouse architecture, ETL/ELT development, and data warehouse modeling will be crucial for this role. You will be actively involved in building and enhancing pipelines, migrating enterprise data (SAP/SQL Server) to the cloud, and implementing scalable solutions using Azure services. Initially, this role will be development-heavy (100% development), transitioning to around 60% development later on. The position is based in Gurugram with a shift timing of 2-11pm. **Key Responsibilities:**
Work on Azure Databricks pipelines: enhance, modify, and optimize existing pipelines and develop new ones when required. - Handle data migration from SAP/SQL Server and enterprise databases into Azure cloud environments (Lakehouse, Delta, Synapse). - Configure, manage, and optimize Databricks clusters, compute, workspaces, and resources for performance and cost efficiency. - Build scalable ETL/ELT processes using PySpark, SQL, Delta Lake, and medallion architecture across BronzeSilverGold layers. - Work extensively on Azure Data Factory: orchestration pipelines, triggers, monitoring, and integration with Databricks. - Design and maintain data warehouse & dimensional models (Facts, Dimensions, SCD, Star/Snowflake schema). - Ensure strong data quality, validation, governance, and security including Unity Catalog, RBAC, and Azure platform controls. - Develop solutions independently, involving end-to-end development, not just support (initially 100% development-focused). **Qualifications Required:**
Azure Databricks (Notebooks, Clusters, Delta Lake, Optimization, Resource Configuration)
Azure Data Lake / Lakehouse Architecture
Azure Data Factory (Pipelines, Orchestration, Integration, Monitoring)
Azure SQL / SQL Server
PySpark, SQL for transformation and performance tuning