Skip to content
Updated How AI-exposed are the skills employers are hiring for? See the data →

From the employer's careers site First seen 31 Oct 2026 · Still live on 15 Nov 2026

Engineering Manager, Deep Learning Inference

NVIDIA Hybrid senior full-time

The employer hasn't listed a salary.

Apply on NVIDIA

Description

NVIDIA is seeking an exceptional Engineering Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering today's most sophisticated AI systems , from large language models to multimodal generative AI , all accelerated on NVIDIA GPUs.

The Deep Learning Inference team develops and optimizes open-source frameworks that make AI deployment scalable, efficient, and accessible , including vLLM / SGLang, and FlashInfer. Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at every scale, from datacenter clusters to edge devices.

Responsibilities:

  • Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
  • Drive the strategy, roadmap, and execution of NVIDIA's inference frameworks engineering, focusing on Client AI.
  • Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
  • Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
  • Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).
  • Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA's broader AI and software strategies.
  • Foster a culture of technical excellence, open collaboration, and continuous innovation.

Requirements:

  • MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
  • 6+ overall years of software development experience, including 3+ years in technical leadership or engineering management.
  • Strong background in C/C++ software design and development; proficiency in Python is a plus.
  • Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
  • Proven record of deploying or optimizing deep learning models in production environments.
  • Experience leading teams using Agile or collaborative software development practices.

Benefits:

  • Highly competitive salaries
  • Comprehensive benefits package
  • Equity eligibility

Applications for this job will be accepted at least until August 9, 2026.

For more fresh new jobs, check out yubhub.co.

Skills

Required

Nice to have

  • Open-source contributions
  • Multi-GPU communications
  • Distributed inference architectures
  • Performance modeling
  • Profiling
  • System-level optimization
Apply on NVIDIA