As a Lead Data Software Engineer at Dodge Construction Network, your role will involve driving the modern data infrastructure by focusing on enabling data science teams, integrating ML and LLM solutions, and building scalable, event-driven data platforms using cutting-edge AWS services. You will report directly to the VP, Data Innovation & AI. **Key Responsibilities:**
Design and implement data lake, Lakehouse, and data warehouse architectures leveraging AWS Data Lake Formation, Redshift, and Delta Lakes
Build and maintain scalable ETL pipelines using AWS Glue, Apache Spark, Databricks, and EMR
Develop data ingestion, transformation, and enrichment workflows using Python, Spark, and SQL
Optimize data storage and partitioning strategies (Parquet, Delta, Iceberg) for performance and cost efficiency
Implement real-time and batch data processing frameworks to support analytics and AI-driven use cases
Leverage serverless computing (AWS Lambda, Fargate) and containerized compute (ECS, EKS, Kubernetes) to scale data workloads
Integrate machine learning (ML) and large language model (LLM) solutions into production data pipelines
Utilize AWS SageMaker, Databricks ML, Bedrock, and Redshift ML to support AI/ML workloads
Apply MLOps frameworks to manage model deployment, monitoring, and retraining at scale
Incorporate AI coding tools into daily development workflows to accelerate delivery
Effectively prompt, review, and validate AI-generated code across Python, SQL, and Spark workloads
Integrate AI tools into CI/CD pipelines to improve code quality, reduce cycle time, and increase sprint velocity
Track and report on AI tool ROI using metrics such as story point reduction, PR throughput, and cycle time improvement
Model responsible AI-assisted development practices
Uphold and advance DevOps best practices across application development and deployment workflows
Containerize and orchestrate data workloads using Docker, Kubernetes, AWS ECS, and EKS
Drive automated testing integration into DevOps pipelines
Monitor system health and data platform observability using AWS CloudWatch, Datadog, and OpenTelemetry
**Qualifications Required:**
Bachelor's degree in a related field or equivalent education and work experience
8+ years of experience in data engineering, cloud architectures, and ML/AI integrations
Hands-on experience with Databricks, Delta Lake, AWS Redshift, and modern data Lakehouse solutions
Demonstrated use of AI development tools to improve personal and team productivity
AWS certifications (Solutions Architect, or equivalent)
You will be based in or near Kochi and work from the Kochi office as part of a hybrid schedule.
