Staff+ Software Engineer, Inference Runtime

Job Overview

We're seeking a Staff+ Software Engineer for our Inference Runtime team at Anthropic. As a technical lead, you'll shape the architecture and roadmap for our shared runtime in the inference serving stack. This senior IC role involves broad technical ownership, partnering with teams to make key decisions on priorities and trade-offs across various accelerator platforms.

Responsibilities

• Set technical direction for the team, owning the architecture and roadmap for the shared runtime of the inference serving stack
• Own and evolve the accelerator-agnostic runtime, including its interfaces, internal boundaries, and build structure, with hands-on work in Rust and Python
• Ensure the platform's expansion cost remains low by optimizing for new models and deployment targets
• Drive efficient accelerator usage across GPU, TPU, and Trainium
• Develop the runtime's validation surface with mechanisms like partitioned builds and canary/shadow/rollback
• Collaborate with Anthropic's central Infrastructure org on compilers, build systems, and toolchains
• Mentor engineers through design reviews, code reviews, and direct collaboration

Requirements

• Deep background in systems engineering or ML infrastructure
• Experience with performance profiling, latency and throughput optimization, and systems debugging at scale
• Significant software engineering experience in high-performance, large-scale distributed systems
• Track record of defining and using engineering metrics to drive improvement
• Experience driving technical alignment across organizational boundaries

Preferred Qualifications

• 8+ years of software engineering experience as a technical lead or anchor on a platform, inference runtime, or ML infrastructure team
• Experience with ML compiler toolchains or accelerator driver/firmware management at scale
• Background operating production as a validation surface at scale
• Experience with deterministic or simulation-based testing for hardware-dependent systems

Logistics

• Annual Salary: $405,000-$485,000 USD
• Location-based hybrid policy: 25% office time required
• Visa sponsorship available

Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours
• Lovely office space

About Anthropic

Anthropic is a public benefit corporation headquartered in San Francisco, working on reliable, interpretable, and steerable AI systems.