Staff Software Engineer, Foundation Model API

At Databricks, you will join an organisation passionate about enabling data and AI teams to solve complex problems. You will be part of the AI team building platforms and products for data apps, AI agents, model training, model serving, and Vector Search.

As a Staff Software Engineer in the Foundation Model API (FMAPI) team, you will help build the unified serving layer for large language models across real-time and batch inference. Your responsibilities will include:

• Building LLM infrastructure for large-scale inference workloads using partner models and self-hosted models
• Shaping the direction of the FMAPI product, from roadmap to execution, by engaging with enterprise users and model providers
• Improving reliability, latency, and efficiency of distributed AI workloads
• Collaborating with platform, infra, and ML teams to deliver seamless end-to-end experiences
• Shaping how developers and data scientists build and interact with AI on Databricks

Requirements:

• 8+ years of experience in backend or infrastructure engineering
• Experience with distributed systems, scalable APIs, or cloud-native infrastructure
• Strong product and ownership mindset, with a focus on shipping user-facing value
• Experience with real-time serving, ML infrastructure, or GPU orchestration
• Familiarity with service-oriented architecture, deployment pipelines, and system observability
• Strong programming skills in Scala, Go, or Python

Bonus points for:

• Exposure to platforms like SageMaker, Vertex AI, or Azure ML
• Built products that support AI workflows

The pay range for this role is $190,000-$265,000 USD. The total compensation package may also include eligibility for annual performance bonus, equity, and benefits.