# Staff Software Engineer, Inference

**Company**: CoreWeave
**Location**: Sunnyvale, CA
**Work arrangement**: hybrid
**Experience**: staff
**Job type**: full-time
**Salary**: $188,000 to $275,000
**Category**: Engineering
**Industry**: Technology

**Apply**: https://job-boards.greenhouse.io/coreweave/jobs/4670593006?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_b72ed694-e50

## Description

CoreWeave is The Essential Cloud for AI. As a Staff Software Engineer on the Inference team, you will act as a technical leader driving architecture, performance, and reliability across multiple services and teams.

The Inference team builds and operates CoreWeave's Kubernetes-native inference platform, powering low-latency, high-throughput AI workloads at massive scale. You will work deeply in distributed systems and Kubernetes-based infrastructure, focusing on areas like scheduling, batching, and memory optimization.

Responsibilities:

- Lead cross-team design initiatives and optimize inference performance

- Improve system reliability at scale and work deeply in distributed systems and Kubernetes-based infrastructure

- Focus on areas like scheduling, batching, and memory optimization

Requirements:

- 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms

- Proven experience leading cross-team technical initiatives impacting multiple services or organizations

- Strong programming skills in Go, Python, or C++

- Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design

- Strong understanding of distributed systems, networking, and performance optimization

Preferred Qualifications:

- Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe

- Experience with GPU systems and performance optimization (CUDA, NCCL, RDMA, NUMA, GPU interconnects)

- Experience leading multi-team or org-level technical initiatives

- Exposure to large-scale AI/ML infrastructure or hyperscale cloud environments

Benefits:

- Medical, dental, and vision insurance - 100% paid for by CoreWeave

- Company-paid Life Insurance

- Voluntary supplemental life insurance

- Short and long-term disability insurance

- Flexible Spending Account

- Health Savings Account

- Tuition Reimbursement

- Ability to Participate in Employee Stock Purchase Program (ESPP)

- Mental Wellness Benefits through Spring Health

- Family-Forming support provided by Carrot

- Paid Parental Leave

- Flexible, full-service childcare support with Kinside

- 401(k) with a generous employer match

- Flexible PTO

- Catered lunch each day in our office and data center locations

- A casual work environment

- A work culture focused on innovative disruption

## Skills

### Required
- distributed systems
- Kubernetes
- Go
- Python
- C++
- performance optimization
- inference systems

### Nice to have
- inference frameworks
- GPU systems
- multi-team technical initiatives
- large-scale AI/ML infrastructure

---

Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/coreweave/jobs/4670593006?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)