Python Data Engineer

Greetings for the Day!!! We are looking for Python Data Engineer for one of our esteemed clients in Bangalore and Hyderabad. Key Responsibilities:

Design, develop, and maintain scalable data pipelines usingPythonandApache Sparkfor large-scale data processing. Build and optimize ETL/ELT workflows to ingest, transform, and load data from multiple structured and unstructured sources. Work with distributed data processing frameworks likeSpark (PySpark/Scala)to handle big data efficiently. Develop data solutions on cloud platforms such asAWS / Azure / GCPusing services like S3, Glue, Databricks, or BigQuery. Implement data modeling techniques and optimize data storage usingdata lakes and data warehouses. Ensure data quality, integrity, and governance through validation, monitoring, and automated checks. Collaborate with data scientists, analysts, and business teams to deliver reliable datasets and insights. Optimize performance of Spark jobs through partitioning, caching, and tuning techniques. Use orchestration tools likeAirflowor similar for workflow scheduling and monitoring. Work with SQL/NoSQL databases such asPostgreSQL, MySQL, MongoDB, or Hive. Implement CI/CD pipelines and follow best practices for version control usingGit. Troubleshoot production issues and ensure high availability and performance of data systems. Key Skills

Primary:Python, Apache Spark (PySpark) Big Data:Hadoop, Hive, Kafka (optional) Cloud:AWS / Azure / GCP Tools:Airflow, Databricks, Docker Databases:SQL & NoSQL Concepts:ETL/ELT, Data Warehousing, Data Lakes, Distributed Computing

We are looking for candidates who can join us within 30 days

If you find this interesting and in line with your career aspiration, please share your CV on shamili@rightmoveconsultants.com or call on 7066010314

Thanks and Regards,