Staff + Sr. Software Engineer, Cloud Inference

Job Overview

The Cloud Inference team at Anthropic scales and optimizes Claude to serve massive audiences of developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers (CSPs). You will design, build, and own backend services and infrastructure that serve Claude across multiple CSPs.

Key Responsibilities

• Design, build, and own backend services and infrastructure for Claude across multiple CSPs, considering differences in compute hardware, networking, APIs, and operational models.
• Collaborate with internal teams and CSP partners to set up the full serving stack on new cloud platforms, resolve operational issues, and influence provider roadmaps.
• Develop and evolve CI/CD automation systems, including validation and deployment pipelines, to reliably ship new model versions to millions of users across cloud platforms.
• Create interfaces and tooling abstractions across CSPs for cost-effective inference management, scaling, and reduced per-platform complexity.
• Contribute to capacity planning, autoscaling, and workload routing strategies that match supply with demand.
• Analyze observability data to identify performance bottlenecks, cost anomalies, and regressions.

Requirements

• Significant software engineering experience with high-performance, large-scale distributed systems serving millions of users.
• Experience building or operating services on at least one major cloud platform (AWS, GCP, or Azure).
• Curiosity about LLM serving; prior inference or ML experience is not required.
• Ability to thrive in cross-functional collaboration with internal teams and external partners.
• Experience working with external partners to align goals and deliver impact.
• Highly autonomous with the ability to take ownership of problems end-to-end.

Preferred Qualifications

• Direct experience working with CSPs to scale infrastructure or products across multiple platforms.
• Hands-on experience with capacity management, cost optimization, or resource planning at scale.
• Solid understanding of multi-region deployments, geographic routing, and global traffic management.
• Proficiency in Python or Rust.

Compensation

The annual compensation range for this role is $320,000-$485,000 USD.

Logistics

• Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience.
• Required field of study: A field relevant to the role.
• Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time.
• Visa sponsorship: We do sponsor visas.