# Senior Software Engineer, AI Inference Systems

**Company**: NVIDIA
**Work arrangement**: remote
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/Germany-Remote/Senior-Software-Engineer--AI-Inference-Systems_JR2017366?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_23ebbfca-7aa

## Description

We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency.

You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments.

**Responsibilities:**

- Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware features; profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation.

- Develop, optimize, and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion, autotuning, and memory/layout optimization; build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization.

- Define and build inference benchmarking methodologies and tools; contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite.

- Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds.

- Conduct and publish original research that pushes the pareto frontier for the field of ML Systems; survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products.

**Requirements:**

- Bachelor’s degree (or equivalent experience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience; alternatively, Master’s degree in CS/CE/SE with 5+ years of experience; or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing.

- Strong programming skills in Python and C/C++; experience with Go or Rust is a plus; solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories.

- Knowledgeable and passionate about performance engineering in ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang).

- Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, NCCL; proficiency with profiling/debug tools (e.g., Nsight Systems/Compute).

- Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups.

- Excellent debugging, problem-solving, and communication skills; ability to excel in a fast-paced, multi-functional setting.

**Preferred Qualifications:**

- Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang).

- Hands-on work with ML compilers and DSLs (e.g., Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e.g., CUTLASS) and features (e.g., CUDA Graph, Tensor Cores).

- Experience contributing to containerization/virtualization technologies such as containerd/CRI-O/CRIU.

- Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability.

- Contributions to open-source projects and/or publications; please include links to GitHub pull requests, published papers and artifacts.

## Skills

### Required
- Python
- C/C++
- Go
- Rust
- CUDA
- NCCL
- Nsight Systems/Compute
- Docker
- Kubernetes
- Slurm
- Linux namespaces
- cgroups
- algorithms & data structures
- operating systems
- computer architecture
- parallel programming
- distributed systems
- deep learning theories

### Nice to have
- vLLM
- SGLang
- Triton
- TorchDynamo/Inductor
- MLIR/LLVM
- XLA
- CUTLASS
- CUDA Graph
- Tensor Cores
- containerd/CRI-O/CRIU
- AWS/GCP/Azure
- infrastructure as code
- CI/CD
- production observability

---

Source: [Apply at nvidia.wd5.myworkdayjobs.com](https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/Germany-Remote/Senior-Software-Engineer--AI-Inference-Systems_JR2017366?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)