As a Senior Data Engineer (Scala) at a technology services & software exports company specializing in Enterprise Data Engineering and Big Data Analytics, your primary responsibility will be to design, build, and operate high-throughput batch and streaming data pipelines. Your hands-on expertise in Scala will be crucial in implementing production-quality Spark jobs, integrating Kafka-based event streams, and ensuring data reliability throughout the analytics lifecycle. This on-site role in India will require you to take ownership of end-to-end data solutions for mission-critical workloads. **Role Overview:**
Design, develop, and maintain scalable data pipelines using Scala and Apache Spark for batch and streaming workloads. - Implement Kafka-based ingestion, process streams with structured streaming, and ensure exactly-once or at-least-once guarantees as required. - Author efficient Spark SQL transformations, optimize job performance, and resource utilization on cluster platforms. - Build robust ETL workflows, handle schema evolution, and implement data quality checks and monitoring. - Collaborate with data consumers, data scientists, and platform teams to translate requirements into reliable data products and APIs. - Participate in CI/CD for data pipelines, troubleshoot production incidents, and create runbooks to improve operational maturity. **Key Responsibilities:**
Showcase your Scala Spark project experience. - Provide examples of Kafka or streaming implementations. - Highlight any performance optimization or production debugging you have led. In addition to the technical aspects of the role, you will be part of a fast-paced engineering culture with ownership of end-to-end data products. You will have opportunities to work on large-scale streaming and analytics projects for enterprise clients, with structured career growth and technical mentorship within the data engineering practice. If you are passionate about Scala, Apache Spark, Apache Kafka, ETL, and data engineering, and are looking to contribute to cutting-edge data solutions, we encourage you to apply and showcase your expertise in the specified areas. As a Senior Data Engineer (Scala) at a technology services & software exports company specializing in Enterprise Data Engineering and Big Data Analytics, your primary responsibility will be to design, build, and operate high-throughput batch and streaming data pipelines. Your hands-on expertise in Scala will be crucial in implementing production-quality Spark jobs, integrating Kafka-based event streams, and ensuring data reliability throughout the analytics lifecycle. This on-site role in India will require you to take ownership of end-to-end data solutions for mission-critical workloads. **Role Overview:**
Design, develop, and maintain scalable data pipelines using Scala and Apache Spark for batch and streaming workloads. - Implement Kafka-based ingestion, process streams with structured streaming, and ensure exactly-once or at-least-once guarantees as required. - Author efficient Spark SQL transformations, optimize job performance, and resource utilization on cluster platforms. - Build robust ETL workflows, handle schema evolution, and implement data quality checks and monitoring. - Collaborate with data consumers, data scientists, and platform teams to translate requirements into reliable data products and APIs. - Participate in CI/CD for data pipelines, troubleshoot production incidents, and create runbooks to improve operational maturity. **Key Responsibilities:**
Showcase your Scala Spark project experience. - Provide examples of Kafka or streaming implementations. - Highlight any performance optimization or production debugging you have led. In addition to the technical aspects of the role, you will be part of a fast-paced engineering culture with ownership of end-to-end data products. You will have opportunities to work on large-scale streaming and analytics projects for enterprise clients, with structured career growth and technical mentorship within the data engineering practice. If you are passionate about Scala, Apache Spark, Apache Kafka, ETL, and data engineering, and are looking to contribute to cutting-edge data solutions, we encourage you to apply and showcase your expertise in the specified areas.