Member of Technical Staff (Model Behavior Architect)

Key Responsibilities

• Context Engineering: Design, test, and optimize context strategies and system prompts that shape answer engine behavior across products, features, and use cases. - Evaluation Systems: Build automated and semi-automated evaluation pipelines that measure model quality, catch regressions, and scale across product surfaces. - Model Launch Support: Partner with research and engineering to validate model behavior before and during rollouts, ensuring smooth transitions with no degradation. - Research & Analysis: Identify inconsistencies and failure modes in model outputs through well-designed research projects — for both internal and production-facing systems. - Cross-functional Collaboration: Work closely with design, product, and research teams to translate product goals into concrete model behavior requirements. - Knowledge Sharing: Help engineers across teams build intuition for prompt design, context engineering, and evaluation best practices. - Staying Current: Track the latest alignment, evaluation, and prompting techniques from industry and academia, and bring the best ideas back to the team. ## What We're Looking For

Required

• Experience designing evaluations, benchmarks, or metrics for AI systems. - Strong written and verbal communication skills, particularly in explaining complex concepts to diverse stakeholders. - Ability to manage multiple concurrent projects in a fast-moving environment. - Strong experience with Perplexity or other frontier AI models in production settings. - Demonstrated experience with Python — you'll prototype, debug, automate, and build systems at scale. - 3+ years of experience working with LLMs in a product or research setting. ### Preferred