Staff Software Engineer, Inference

CoreWeave is The Essential Cloud for AI. As a Staff Software Engineer on the Inference team, you will act as a technical leader driving architecture, performance, and reliability across multiple services and teams.

The Inference team builds and operates CoreWeave's Kubernetes-native inference platform, powering low-latency, high-throughput AI workloads at massive scale. You will work deeply in distributed systems and Kubernetes-based infrastructure, focusing on areas like scheduling, batching, and memory optimization.

Responsibilities:

• Lead cross-team design initiatives and optimize inference performance
• Improve system reliability at scale and work deeply in distributed systems and Kubernetes-based infrastructure
• Focus on areas like scheduling, batching, and memory optimization

Requirements:

• 8–12+ years of experience building and operating large-scale distributed systems or cloud platforms
• Proven experience leading cross-team technical initiatives impacting multiple services or organizations
• Strong programming skills in Go, Python, or C++
• Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
• Strong understanding of distributed systems, networking, and performance optimization

Preferred Qualifications:

• Experience with inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe
• Experience with GPU systems and performance optimization (CUDA, NCCL, RDMA, NUMA, GPU interconnects)
• Experience leading multi-team or org-level technical initiatives
• Exposure to large-scale AI/ML infrastructure or hyperscale cloud environments

Benefits:

• Medical, dental, and vision insurance - 100% paid for by CoreWeave
• Company-paid Life Insurance
• Voluntary supplemental life insurance
• Short and long-term disability insurance
• Flexible Spending Account
• Health Savings Account
• Tuition Reimbursement
• Ability to Participate in Employee Stock Purchase Program (ESPP)
• Mental Wellness Benefits through Spring Health
• Family-Forming support provided by Carrot
• Paid Parental Leave
• Flexible, full-service childcare support with Kinside
• 401(k) with a generous employer match
• Flexible PTO
• Catered lunch each day in our office and data center locations
• A casual work environment
• A work culture focused on innovative disruption