Senior Lead Data Engineer

10+ years of hands-on experience in Data Engineering, focusing on delivering enterprise-scale data platforms. - Proficiency in Databricks (PySpark, Spark SQL, Delta Lake, workflows, cluster tuning) and Azure Data Factory (ADF). - Advanced skills in Python scripting for data processing, automation, and tooling. - Track record of leading Data Engineering teams of 810 engineers in a senior or lead capacity. - Strong SQL skills and in-depth understanding of data modeling (dimensional, Data Vault, Lakehouse/medallion). - Experience with CI/CD for data (Azure DevOps, GitHub Actions) and Infrastructure as-Code (Terraform, ARM/Bicep). - Familiarity with the broader Azure data stack (ADLS Gen2, Synapse, Key Vault, Event Hub, Azure Functions). - Excellent stakeholder engagement and communication abilities, enabling you to convey technical concepts to non-technical audiences effectively. Desirable Experience:

Databricks certifications (Data Engineer Associate/Professional) or Azure Data Engineer certification (DP-203 / DP-700). - Exposure to Unity Catalog, data governance, and lineage tooling (Purview, Collibra). - Familiarity with streaming data technologies (Kafka, Event Hubs, Structured Streaming). - Previous experience in regulated industries such as financial services, healthcare, or the public sector. - Background in consulting or professional services would be advantageous. As a Senior Lead Data Engineer in this contracting role based in India (remote) for a duration of 6 months, you will have the opportunity to join a growing data practice. With over 10 years of hands-on Data Engineering experience, your deep expertise in Databricks, Azure Data Factory (ADF), and Python scripting will be essential. This position requires both technical proficiency and leadership skills to design and construct enterprise-grade data platforms while guiding a team of 810 Data Engineers through complex delivery programs. Key Responsibilities:

Lead the design, development, and implementation of scalable data pipelines and platforms using Databricks and Azure. - Take ownership of the technical aspects related to ingestion, transformation, and curation workloads utilizing ADF, Databricks (PySpark/Spark SQL), and Python. - Manage and lead a team of 810 Data Engineers, including assigning tasks, conducting code reviews, offering mentorship, and providing performance feedback. - Establish and uphold engineering best practices encompassing coding standards, CI/CD, testing, observability, and data quality. - Collaborate with architects, analysts, and business stakeholders to transform requirements into robust, production-ready data solutions. - Optimize the performance, cost, and reliability of existing data pipelines and Databricks workloads. - Promote the adoption of Lakehouse, Delta Lake, and Unity Catalog patterns across the organization. - Contribute to roadmap planning, estimation, and resource allocation for multiple concurrent workstreams. - Serve as the senior technical escalation point for complex engineering challenges. Qualification Required:

10+ years of hands-on experience in Data Engineering, focusing on delivering enterprise-scale data platforms. - Proficiency in Databricks (PySpark, Spark SQL, Delta Lake, workflows, cluster tuning) and Azure Data Fa