Senior Associate_ Data Engineer

Key Responsibilities:

• Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL)
• Implement batch and incremental data processing using Delta Lake and multi-hop architecture
• Develop and optimize Databricks notebooks, jobs, and workflows
• Ingest, transform, and curate large-scale structured and semi-structured datasets
• Support analytics, reporting, and downstream data consumption use cases
• Ensure data quality, reliability, lineage, and governance
• Collaborate with data scientists, analysts, and architects on AI/ML workloads
• Optimize Spark jobs for performance and cost efficiency
• Adhere to enterprise security, access control, and compliance standards
• Provide production support and troubleshoot data pipeline issues
• Document technical designs, data flows, and operational runbooks
• Mentor junior engineers and contribute to best practices

Qualifications Required:

• 3+ years of experience as a Data Engineer with strong Databricks expertise
• Hands-on experience with Apache Spark, PySpark, and Spark SQL
• Strong knowledge of Delta Lake and Lakehouse architecture
• Advanced SQL skills
• Experience with ETL/ELT patterns and data warehousing concepts
• Exposure to at least one cloud platform (Azure / AWS / GCP)
• Understanding of distributed computing concepts
• Experience working in Agile teams
• Bachelors or Masters degree in Computer Science, Engineering, or related field (60% above)