Description
We're looking for a Senior Software Engineer to help us build Nebius Serverless AI , our GPU-native platform for deploying inference endpoints, batch jobs, and AI workloads without managing infrastructure.
As a Senior Software Engineer, you will:
- Design and build core components of the Serverless platform , the control plane, scheduler, runtime, autoscaler, and the APIs customers integrate against.
- Own the hardest engineering problems on the service: cold-start latency, GPU scheduling under contention, multi-tenant isolation, fair-share quotas, request routing at the edge.
- Set the technical direction for the areas you own , drive architecture decisions, write the design docs, and align the team behind the chosen approach.
- Raise the engineering bar across the team through code reviews, design reviews, and example , the kind of senior presence that compounds.
- Run the service like an SRE: define SLOs, build observability, lead incident response, and make sure every postmortem changes the platform, not just the runbook.
- Work directly with customers on architecture reviews, performance escalations, and the production problems that don't fit a ticket template.
- Partner closely with Product, GTM, and infrastructure teams to translate customer needs into a coherent technical roadmap.
Requirements:
- 7+ years of professional software engineering experience, with a track record of shipping production distributed systems at scale.
- Excellent knowledge of Golang, or you are ready to quickly switch to it.
- Deep experience with Kubernetes and container orchestration , you've operated it in anger, not just deployed it.
- Strong distributed-systems instincts: consistency vs. availability trade-offs, queueing, backpressure, retries, idempotency, multi-tenancy.
- Experience designing and operating high-throughput, low-latency services , you know where the milliseconds go and how to get them back.
- A history of being the engineer others look to on hard problems , driving design discussions, unblocking teammates, and shipping the thing nobody else wanted to touch.
- Ability to write reliable code and dig into complex problems.
- Teamwork-oriented approach.
Nice to have:
- Experience building serverless or function-as-a-service platforms (Knative, AWS Lambda, GCP Cloud Run, Cloudflare Workers, Modal, Replicate, Together, Fireworks, Anyscale, or similar).
- GPU scheduling experience , Kubernetes device plugins, MIG, MPS, time-slicing, NVIDIA GPU Operator.
- ML inference experience , vLLM, TensorRT-LLM, Triton Inference Server, SGLang, model loading and warm-pool strategies.
- Cold-start optimization at the runtime, image, or snapshot level (FireCracker, gVisor, checkpoint/restore, image streaming).
- Experience writing Kubernetes operators (Go + controller-runtime / kubebuilder).
- Contributions to relevant open-source projects in the serverless, scheduling, or inference ecosystems.
We offer:
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams