As a Software Development Engineer (Data) at Apple, you will be part of the Advertising Platforms group that enables users worldwide to access informative content while assisting publishers and developers to promote and monetize their work. Your role will involve building highly scalable data platforms and services to support effective advertising while prioritizing user privacy. **Key Responsibilities:**
**Airflow Platform Development:**
Design, architect, and maintain scalable Apache Airflow platform infrastructure. - Customize and extend core functionality of Airflow internals. - Develop custom Airflow operators, sensors, hooks, and plugins. - Create standardized patterns and reusable components for data teams. - **Infrastructure & Scalability:**
Deploy and manage Airflow on Kubernetes at scale. - Optimize Airflow performance for thousands of concurrent DAGs and tasks. - Implement multi-tenancy and isolation strategies. - Architect high-availability and disaster recovery solutions. - **Platform Reliability & Operations:**
Implement monitoring, alerting, and observability for the platform. - Troubleshoot Airflow internals and system issues. - Establish SLAs and ensure platform reliability. - Plan and execute zero-downtime upgrades and migrations. - **Integration & Ecosystem:**
Build integrations with Spark, EMR, Kubernetes, and Hadoop ecosystem. - Develop authentication and authorization frameworks. - Integrate with CI/CD systems for DAG deployment. - Connect Airflow with observability stack and build APIs for automation. - **Developer Experience:**
Create documentation, best practices, and guidelines. - Build developer tooling and provide technical consultation to teams. - Conduct code reviews and evangelize platform capabilities. **Qualification Required:**
**Deep Airflow Expertise:**
5+ years in Platform/Infrastructure Engineering or Data Platform Engineering. - 3+ years of hands-on experience with Apache Airflow internals. - Expert-level Python and Java proficiency. - Production experience with Kubernetes and AWS EMR. - Knowledge of Hadoop ecosystem and Apache Spark architecture. - **Platform Engineering Skills:**
Distributed systems concepts, database performance tuning, and infrastructure as code. - CI/CD systems, monitoring, and observability tools. - Strong software design principles and test-driven development. **Additional Company Details:**
Apple's Advertising Platforms group focuses on creating products that enrich lives and drive innovation. By joining Apple, you become part of a diverse community that fosters groundbreaking technology and environmental efforts. Embrace the opportunity to contribute to a company that aims to leave the world better than before. As a Software Development Engineer (Data) at Apple, you will be part of the Advertising Platforms group that enables users worldwide to access informative content while assisting publishers and developers to promote and monetize their work. Your role will involve building highly scalable data platforms and services to support effective advertising while prioritizing user privacy. **Key Responsibilities:**
**Airflow Platform Development:**
Design, architect, and maintain scalable Apache Airflow platform infrastructure. - Customize and extend core functionality of Airflow internals. - Develop custom Airflow operators, sensors, hooks, and plugins. - Create standardized patterns and reusable components for data teams. - **Infrastructure & Scalability:**
Deploy and manage Airflow on Kubernetes at scale. - Optimize Airflow performance for thousands of concurrent DAGs and tasks. - Implement multi-tenancy and isolation strategies. - Architect high-availability and disaster recovery solutions. - **Platform Reliability & Operations:**
Implement monitoring, alerting, and observability for the platform. - Troubleshoot Airflow internals and system issues. - Establish SLAs and ensure platform reliability. - Plan and execute zero-downtime upgrades and migrations. - **Integration & Ecosystem:**
Build integrations with Spark, EMR, Kubernetes, and Hadoop ecosystem. - Develop authentication and authorization frameworks. - Integrate with CI/CD systems for DAG deployment. - Connect Airflow with observability stack and build APIs for automation. - **Developer Experience:**
Create documentation, best practices, and guidelines. - Build developer tooling and provide technical consultation to teams. - Conduct code reviews and evangelize platform capabilities. **Qualification Required:**
**Deep Airflow Expertise:**
5+ years in Platform/Infrastructure Engineering or Data Platform Engineering. - 3+ years of hands-on experience with Apache Airflow internals. - Expert-level Python and Java proficiency. - Production experience with Kubernetes and AWS EMR. - Knowledge of Hadoop ecosystem and Apache Spark architecture.
