# Principal Engineer - Autonomous Agents Evaluation

**Company**: NVIDIA
**Experience**: senior
**Job type**: full-time
**Salary**: 224,000 USD - 356,500 USD for Level 5, and 272,000 USD - 431,250 USD for Level 6
**Category**: Engineering
**Industry**: Technology

**Apply**: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Agent-Evaluation-and-Self-Improvement-Engineer_JR2028060-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_b9a09443-7ca

## Description

We are seeking a Principal Engineer to set the evaluation strategy for autonomous agents across NVIDIA. The successful candidate will design and implement evaluation systems to ensure agents improve themselves reliably.

**Job Description:** As a Principal Engineer, you will be responsible for building evaluation systems that work across various layers, including individual skills, MCP servers, whole agents, and products. You will also design how agents use evaluation results to improve themselves.

**Key Responsibilities:**

- Develop evaluation systems that work across different harnesses with automated checks to catch regressions before users do.

- Analyze production traces to identify failures, new usage patterns, and areas for improvement.

- Evaluate evaluation datasets for coverage, duplicates, and relevance to production environments.

- Build self-improvement loops for evolving harnesses, allowing agents to propose changes and test them in sandboxed environments.

- Create realistic enterprise environments for agents to run long, end-to-end tasks.

- Measure multi-agent swarms and record task completion times and token usage.

- Ensure evaluations are honest and unbiased by calibrating model and agent judges against human labels.

**Requirements:**

- BS or MS in Computer Science, Engineering, or related field (or equivalent experience).

- 12+ years of experience building large-scale systems, including evaluation systems for ML or agents.

- Hands-on experience building and evaluating agents on an open or closed agent harness.

- Solid grounding in experiment design, statistical significance, and A/B testing.

- Experience setting technical direction as a senior individual contributor.

- Proficiency in Python, Go, Rust, or similar programming languages.

**Nice to Have:**

- Experience building self-improving or self-evolving agent systems.

- Shipped an agent benchmark, eval framework, or simulated test environment adopted by other teams.

- Detected reward hacking or eval gaming, or built oversight for systems that modify themselves.

- Built domain evaluations, such as for chip design or hardware verification.

**Salary Range:** The base salary range is 224,000 USD - 356,500 USD for Level 5, and 272,000 USD - 431,250 USD for Level 6.

**Benefits:** You will also be eligible for equity and benefits.

## Skills

### Required
- Python
- Go
- Rust
- Computer Science
- Machine Learning
- Experiment Design
- Statistical Significance
- A/B Testing

### Nice to have
- Self-improving agent systems
- Agent benchmark
- Eval framework
- Simulated test environment
- Reward hacking detection
- Domain evaluations

---

Source: [Apply at nvidia.wd5.myworkdayjobs.com](https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Agent-Evaluation-and-Self-Improvement-Engineer_JR2028060-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)