Description
Software Engineer, Applied Evals
Location
San Francisco
Employment Type
Full time
Location Type
Hybrid
Department
Applied AI
Compensation
- $230K – $325K • Offers Equity
The base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. If the role is non-exempt, overtime pay will be provided consistent with applicable laws. In addition to the salary range listed above, total compensation also includes generous equity, performance-related bonus(es) for eligible employees, and the following benefits.
- Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
- Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
- 401(k) retirement plan with employer match
- Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
- Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
- 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
- Mental health and wellness support
- Employer-paid basic life and disability coverage
- Annual learning and development stipend to fuel your professional growth
- Daily meals in our offices, and meal delivery credits as eligible
- Relocation support for eligible employees
- Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided.
More details about our benefits are available to candidates during the hiring process.
This role is at-will and OpenAI reserves the right to modify base pay and other compensation components at any time based on individual performance, team or company results, or market conditions.
About the team
Applied Evals defines what good looks like for safe, advanced AI systems. We turn complex, high-value workflows into clear, reproducible signals that guide model training and product quality. Our work bridges frontier customers and models, ensuring improvements show up where users experience them. We combine hands-on, unscalable efforts with systems that others can extend, creating a compounding loop of model improvement.
About the Role
We’re hiring product-minded engineers to design and build evals and harnesses that capture real-world quality for advanced AI systems. You’ll own the loop from prototyping with users to building reliable pipelines and integrating signals into training stacks. This role sits at the center of model improvement. The systems you design will directly shape how models behave, accelerate their reliability, and raise the standard for what customers expect.
You’ll collaborate closely with research and product teams and work across the stack, from backend pipelines to user-facing interfaces. The work includes evaluating multi-turn and tool-using systems, designing agent harnesses, and applying reinforcement learning and related methods in production settings. Engineers who succeed in this role bring both a builder’s mindset and the judgment to create reusable systems that others can build on. Many thrive here by operating like founders or founding engineers, taking initiative, moving quickly, and creating structure where none exists.
This role is based in our San Francisco HQ. We use a hybrid work model of 3 days in the office per week and offer relocation assistance.
In this role, you will:
- Define the core evaluation signals that drive model improvement at OpenAI, turning vague product gaps into crisp, defensible measures of quality
- Design agents, harnesses, and eval pipelines that are reliable, reproducible, and extendable
- Prototype solutions with real workflows and convert them into scalable feedback loops
- Connect evaluation signals directly to research and training systems so product improvements show up in what users experience
- Shape model interaction paradigms by partnering with engineering, research, and product teams on how models are deployed and measured
- Build reusable systems and tools that enable contributions from across the company and steadily raise the quality bar
You’ll thrive in this role if you:
- Bring 4+ years of experience in software engineering with strong fundamentals and a track record of shipping production systems end-to-end
- Have experience building AI agents or applications, including designing evals and improving performance through prompting or scaffolding
- Are familiar with evaluation methods for LLMs and have worked with patterns like multi-agent workflows, tool use, or long context.
- Are familiar with deep learning concepts or have prior exposure to training models.
- Communicate clearly across technical and non-technical audiences across levels
- Are motivated by high-impact collaboration with research and product teams and thrive in ambiguity
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.