Senior Data Engineer AWS Data Platform

Strong expertise in Python and SQL

Solid understanding of ETL/ELT concepts, data partitioning, and distributed processing

Experience with version control systems such as GitLab or Bitbucket (Note: Soft Skills, Good to Have Skills, and Any additional details of the company have been omitted as they were not included in the provided Job Description) Role Overview: You will be a part of the data platform team as a Senior Data Engineer, where your primary responsibility will be to design scalable data pipelines, workflow orchestration frameworks, and large-scale data migration solutions using AWS data services. Your role will involve building cloud-native data engineering solutions on AWS, migrating datasets from legacy systems and data warehouses, and ensuring secure and efficient data processing pipelines across distributed environments. Key Responsibilities:

Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3

Build and maintain high-performance data ingestion frameworks for processing large-scale datasets

Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift

Develop and maintain data workflow orchestration frameworks using Apache Airflow or AWS Step Functions

Automate complex workflows including data ingestion, transformation, validation, and loading processes

Lead data migrations from legacy data warehouse technologies to AWS platforms

Perform data migration from RDBMS systems (MySQL, SQL Server, Oracle) to AWS S3 or AWS Redshift

Implement secure data pipelines using AWS security best practices

Monitor data pipelines and troubleshoot performance issues

Optimize ETL workflows for scalability, reliability, and cost efficiency