Software Development Engineer in Test, ML

• Design, develop, and maintain automated test frameworks, test scripts for ML inference services and related ML platform components using python/Java
• Build and automate realistic, representative test account creations and datasets to evaluate model behavior across different scenarios
• Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models prior to release
• Perform different types of testing, including functional, integration, regression and API testing working with RESTful APIs, microservices and databases, and non-functional performance and load testing to verify inference service scalability, latency requirements and reliability under production like conditions
• Work closely with ML engineers and ops teams to ensure that all features and bug fixes come with automated test coverage, ensuring continuous integration and deployment
• Debug and analyze test failures, model anomalies and report defects
• Drive high standards of quality for both engineering decisions and customer-facing features with optimal test and automation strategy
• Develop and implement best practices for test automation that is highly scalable and maintainable
• Monitor test execution and optimize performance in test environments
• Come up with innovative solutions to organizational problems

Required Qualifications:

• Bachelor's degree or equivalent

• 3+ years of experience as an SDET, with strong experience testing backend systems. - Proficiency in Python/Java/Scala/go for test automation. - Hands-on experience with API testing tools (e.g., Postman or similar) for distributed systems

• Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments such as jenkins, GitHub Actions, ArgoCD, etc

• Hands-on experience building and maintaining automated test frameworks (e.g., pytest, JUnit, TestNG, etc.)Experience in working with databases  for backend validation. - Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker, etc.)

• Familiarity with monitoring and observability tools (e.g., Prometheus, Grafana, Datadog, etc.) for validating service health

• Strong understanding of software development lifecycle (SDLC) and agile methodologies. - Excellent problem-solving skills and excellent communication skills. ## Preferred Qualifications:

• Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML application systems. - Experience creating synthetic or seeded test data to simulate realistic customer/account behaviors. - Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation, rather than deterministic assertions. - Knowledge of data pipelines, feature stores, inference systems or model-serving infrastructure (Seldon, KServe, Ray Serve, etc.) is a plus. - Experience testing online services with high RPS and low latency requirements

• Experience with performance and load testing tools (Locust, k6, Gatling, JMeter, etc.)

• Experience with Databricks or similar ML platform tooling