Description
Join NVIDIA, a leader in technology, and work on AI safety at the forefront of innovation. You'll lead cross-functional planning and execution for evaluation identification, selection, and execution, focusing on agentic-safety evaluations.
Responsibilities:
- Lead cross-functional planning and execution for evaluation identification, selection, and execution, with a focus on agentic-safety evaluations, including multi-turn tool calls, unsafe tool use, unintended actions, excessive autonomy, and multi-step failures.
- Track execution, results for all evals, analysis, and the mitigation and research plans for improvement that will stem from results analysis.
- Translate evaluation findings into prioritized mitigation plans with accountable owners, committed dates, and measurable closure criteria.
- Lead end-to-end safety planning and execution for Nemotron models, including program scope, dependencies, risks, resources, and release-readiness criteria.
- Translate technical safety priorities into executable programs covering LLM security, frontier risks, agentic safety, hallucinations.
- Establish and track multi-turn, multi-modal, multi-lingual, long-context, and reasoning model evaluations across relevant use cases, domains, languages, modalities, and model releases.
- Establish governance for reviewing safety findings, assigning severity, determining release impact, and escalating unresolved risks.
- Build and maintain dashboards and executive reporting for evaluation coverage, critical findings, mitigation progress, residual risk, and release readiness.
Requirements:
- Bachelor's degree or equivalent experience in computer science, engineering, data science, or a related technical field.
- 10+ years of experience in technical program management, engineering, product development, technical operations, or a similar area.
- Strong understanding of LLM architecture, frameworks, and model evaluation.
- Familiarity with LLM development, post-training, inference, tool calls, evaluation datasets, and model-release lifecycles.
- Experience directing programs related to AI/ML development, model evaluation, agentic safety, security, content safety, hallucinations, and/or production releases.
- Familiarity with AI safety risks such as hallucinations, timely injection attacks, unsafe tool use, data poisoning, model manipulation, and unintended agent behavior.
- Experience supporting model-safety evaluations, Red Teaming, adversarial testing, security assessments, or responsible-AI programs.
- Ability to interpret technical evaluation findings and communicate their product, schedule, and release implications to leadership.
Nice to Have:
- Experience with LLM Agentic Safety, Security, Red Teaming, Hallucinations, adversarial assessment, or Responsible AI.
- Experience defining evaluation datasets, rubrics, benchmarks, and release gates.
- Ability to use evaluation results and data to make clear release recommendations and communicate residual risk to leadership.
- Experience crafting or operationalizing multi-turn, hallucination, adversarial, or agentic-safety evaluations.
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 168,000 USD - 258,750 USD for Level 4, and 200,000 USD - 322,000 USD for Level 5. You will also be eligible for equity and benefits.