Senior AI QA Engineer

Title: Senior AI QA Engineer Location: Bengaluru (Bangalore)

Key Responsibilities:

• Architect Multi-Layered Validation Frameworks: Design and implement structured testing strategies that combine deterministic checks, semantic similarity metrics, and model-based evaluations. - Automated Model Grading: Develop systems to evaluate clinical pipeline outputs for faithfulness, safety, and hallucination detection using various automated scoring techniques (e.g., BERTScore, ROUGE, or custom heuristics). - Vibe-Driven Development: Leverage agentic AI tools to rapidly prototype complex test harnesses, "red-team" clinical logic, and build internal validation utilities at high velocity. - Data Pipeline Integrity: Execute integration and regression tests for data-heavy backend processes, ensuring medical data remains consistent from ingestion to insight generation. - Collaborative Strategy: Work closely with Data Scientists and Product Managers to define "Ground Truth" datasets and clinical evaluation rubrics. - Root Cause Analysis: Deep-dive into complex system failures to identify whether issues stem from code logic, data drift, or model behavior. ### Requirements:

• 6+ years of technical experience in Quality Assurance, with a strong focus on system architecture and backend data validation. - Expert-level skills in Python for building custom test scripts and working within AI/ML ecosystems. - Proven experience testing model outputs using diverse metrics (e.g., ). Familiarity with tools like . - Proficiency with AI-native development tools such as . - Proficiency with frameworks like , and experience with traditional tools (Cypress, Playwright, or Selenium) for end-to-end flows. - Solid understanding of API testing (Postman/RestAssured) and database validation (SQL/NoSQL). - Experience serving as a Subject Matter Expert (SME) on test process development for large-scale systems.