# Staff Machine Learning Engineer, LLM Inference Optimization **Company**: Nebius **Location**: Amsterdam, Netherlands; Berlin, Germany; London, United Kingdom; Poland; Prague, Czech Republic; Zurich, Switzerland **Experience**: staff **Job type**: full-time **Category**: Engineering **Industry**: Technology **Apply**: https://job-boards.greenhouse.io/nebius/jobs/5012275101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply **Canonical**: https://yubhub.co/jobs/job_a7c04fba-119 ## Description Nebius Token Factory is building production-grade inference for open-source, customer, and internally optimized models. This role owns the path from model artifact to fast, reliable, cost-efficient endpoint. The work sits at the intersection of model internals, inference engines, serving architecture, benchmarking, and production deployment. ## Responsibilities - Own a platform-level optimization pillar such as quantization, speculative decoding, KV-cache optimization, engine selection, or long-context serving. - Create reusable evaluation, benchmarking, and rollout frameworks across multiple model families. - Provide technical direction to MLEs, Applied Scientists, GPU kernel engineers, and infrastructure engineers on shared performance goals. - Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token. - Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, or similar systems. - Build and productionize model-compression workflows, including quantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery. - Implement or integrate speculative decoding, draft-model approaches, KV-cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated prefill/decode serving. - Build reproducible benchmark harnesses for TTFT, TPOT, tokens per second per GPU, p95/p99 latency, GPU memory, reliability, and cost per token. - Partner with GPU kernel engineers and platform engineers to diagnose bottlenecks across model code, kernels, runtime, scheduler, gateway, and cluster layers. - Write clear design docs, performance reports, rollout plans, and customer-facing technical explanations. ## Requirements - Strong Python and PyTorch engineering skills. - Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems. - Practical knowledge of at least one modern inference stack such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent internal systems. - Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving. - Ability to reason quantitatively about latency, throughput, quality, utilization, and cost tradeoffs. - Strong communication skills and ability to collaborate with research, kernel, infrastructure, product, and customer teams. ## Preferred Qualifications - Experience with quantization-aware training, post-training quantization, FP8, INT8, INT4, NVFP4, MXFP4, AWQ, GPTQ, SmoothQuant, or related techniques. - Experience with distillation, speculative decoding, EAGLE, Medusa, multi-token prediction, or other inference acceleration methods. - Experience with agentic workloads, including tool calling, structured outputs, streaming APIs, high concurrency, and multi-step orchestration. - CUDA or Triton familiarity, even if the role is not primarily a kernel-engineering role. - Open-source contributions to vLLM, SGLang, TensorRT-LLM, FlashInfer, LMCache, PyTorch, Triton, Ray, KServe, or related projects. ## Benefits & Perks - Competitive compensation - Career growth and learning opportunities - Flexibility and ownership - Collaborative and innovative culture - Opportunity to work on impactful AI projects - International environment and talented teams ## Skills ### Required - Python - PyTorch - LLM - VLM - transformer inference - inference engines - benchmarking - model compression ### Nice to have - quantization-aware training - post-training quantization - FP8 - INT8 - INT4 - NVFP4 - MXFP4 - AWQ - GPTQ - SmoothQuant - distillation - speculative decoding - EAGLE - Medusa - multi-token prediction - agentic workloads - tool calling - structured outputs - streaming APIs - high concurrency - multi-step orchestration - CUDA - Triton --- Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/nebius/jobs/5012275101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)