Staff AI Test and Evaluation Engineer

Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology.

The Discovery team at Anduril is looking for a Staff AI Test and Evaluation Engineer to join their team.

As a Discovery engineer, you will pick up a concept nobody has proven and build the analysis or prototype that tests it to get to an answer.

The role involves developing test scenarios, simulation environments, and evaluation pipelines to assess agentic AI performance in classified simulations.

Responsibilities:

• Develop comprehensive test scenarios and simulation environments to assess agentic AI performance in classified simulations.
• Lead the integration and validation of agentic AI systems onto classified platforms and edge hardware.
• Build reusable evaluation pipelines, automated test harnesses, and monitoring dashboards for continuous validation.
• Define actionable metrics for AI models and traditional models.
• Partner with cross-functional teams to define requirements, document test results, and contribute to AI model cards and deployment readiness reviews.
• Architect, build, and maintain the evaluation and validation infrastructure for agentic AI and ML models.
• Analyze and resolve issues uncovered in evaluation and in deployment.

Requirements:

• Bachelor's or Master's degree in Computer Science, Machine Learning, Software Engineering, Computer Engineering, Electrical Engineering, or a related technical field.
• At least 12+ years of hands-on experience writing production-grade code, with strong Python skills.
• Demonstrated experience owning evaluation or automated testing for production ML or software systems.
• Hands-on experience designing evaluation methodologies for AI or machine learning models.
• Experience designing or working extensively with simulation environments.
• Working knowledge of how to define, capture, and reason about performance metrics for AI models and traditional models.
• Ability to navigate and contribute to complex systems and established codebases.
• Comfort operating between technical program management and software engineering.
• Passion for building the evaluation infrastructure that proves AI works.
• Must be eligible for a US security clearance.

Preferred Qualifications:

• Experience as an ML Test & Evaluation Engineer, SDET for ML systems, ML/Evaluation Engineer, Simulation Engineer, or Technical Program Manager for AI/ML.
• Experience designing test methodologies for agentic AI systems.
• Familiarity with validating and deploying AI models onto classified platforms, edge hardware, or resource-constrained environments.
• Experience contributing to AI model cards, deployment readiness reviews, or similar model-governance and release-gating practices.
• Familiarity with test and evaluation practices in aerospace or defense.
• Experience with MLOps tooling, monitoring dashboards, and continuous validation pipelines for ML models in production.
• Additional experience with Go, C++, or scripting for test automation and tooling.

Benefits:

• Highly competitive equity grants
• Top-tier benefits for full-time employees, including comprehensive, competitive benefits package