Senior Research Engineer - Enterprise Products

We are seeking a Senior Research Engineer passionate about Generative AI inference. As a Senior Research Engineer at NVIDIA, you will be working on developing optimized inferencing technologies to support our growing generative AI needs.

Key Responsibilities:

• Design and evaluate routing policies for LLM traffic to best use a mixture of model systems.
• Build and run agentic benchmarks (e.g., Terminal-Bench) to measure algorithm quality, and turn results into calibration data and routing profiles.
• Ship design documents, code reviews, documentation, and community contributions to an open-source repository.
• Collaborate with engineering teams across NVIDIA to ensure seamless integration with the NVIDIA accelerated serving stack.

Requirements:

• Bachelor's or Master's degree in Computer Science or equivalent experience.
• 8+ years of industry experience in Deep Learning frameworks (PyTorch or TensorFlow).
• Experience designing or running LLM evaluations/benchmarks , ideally agentic ones , and drawing statistically sound conclusions from them.
• Understanding of modern techniques in Machine Learning, Deep Neural Networks, Natural Language Processing, or Speech Recognition.
• Empirical research mindset: forming hypotheses about new algorithms, running calibrations, iterating on results.
• Strong communication and interpersonal skills, along with the ability to work in a dynamic and distributed team.
• A desire to constantly grow and learn new things.
• Strong computer science fundamentals - algorithms and data structures, computational complexity, parallel and distributed computing, system software.

Preferred Qualifications:

• Experience architecting or developing large-scale distributed systems for deep learning.
• Agentic benchmark creation and publications.
• Knowledge of CPU and/or GPU architecture.
• GPU programming (CUDA).

You will also be eligible for equity and benefits.