# Engineering Manager, Deep Learning Inference

**Company**: NVIDIA
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Engineering-Manager--Deep-Learning-Inference_JR2021468?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/job/engineering-manager-deep-learning-inference-20ebaa4e-f69/

## Description

NVIDIA is seeking an exceptional Engineering Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering today's most sophisticated AI systems , from large language models to multimodal generative AI , all accelerated on NVIDIA GPUs.

The Deep Learning Inference team develops and optimizes open-source frameworks that make AI deployment scalable, efficient, and accessible , including vLLM / SGLang, and FlashInfer. Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at every scale, from datacenter clusters to edge devices.

**Responsibilities:**

- Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.

- Drive the strategy, roadmap, and execution of NVIDIA's inference frameworks engineering, focusing on Client AI.

- Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.

- Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.

- Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).

- Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA's broader AI and software strategies.

- Foster a culture of technical excellence, open collaboration, and continuous innovation.

**Requirements:**

- MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.

- 6+ overall years of software development experience, including 3+ years in technical leadership or engineering management.

- Strong background in C/C++ software design and development; proficiency in Python is a plus.

- Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.

- Proven record of deploying or optimizing deep learning models in production environments.

- Experience leading teams using Agile or collaborative software development practices.

**Benefits:**

- Highly competitive salaries

- Comprehensive benefits package

- Equity eligibility

Applications for this job will be accepted at least until August 9, 2026.

For more fresh new jobs, check out [yubhub.co](https://yubhub.co/).

## Skills

### Required
- C/C++
- Python
- GPU programming
- CUDA
- Triton
- CUTLASS
- Deep learning
- AI deployment

### Nice to have
- Open-source contributions
- Multi-GPU communications
- Distributed inference architectures
- Performance modeling
- Profiling
- System-level optimization

---

Source: [Apply at nvidia.wd5.myworkdayjobs.com](https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Engineering-Manager--Deep-Learning-Inference_JR2021468?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)
