Description
You will lead the engineering team within NVIDIA's Dynamo organization to build and advance the open-source benchmarking platform, AIPerf. AIPerf is the growing standard for assessing LLM serving performance across various inference frameworks. Your responsibility is to drive the technical roadmap for AIPerf's core infrastructure, including load generation, ZMQ-based microservices, GPU telemetry, and Kubernetes-native deployment.
As Technical Lead Manager, you will take ownership for the accuracy and statistical soundness of benchmark results that engineering groups throughout the industry depend on to inform production infrastructure decisions. You will advise upstream engine integrations involving vLLM, TRT-LLM, and SGLang in partnership with NVIDIA's Dynamo and NIM teams to maintain AIPerf's relevance across emerging hardware, workload categories, and inference configurations.
You will hire, mentor, and grow a team of senior engineers operating in a high-velocity open-source environment with active external contributors worldwide. You will have a deep understanding of LLM inference mechanics and the ability to reason about measurement correctness and reproducibility.
The ideal candidate will have a Bachelor's degree in Computer Science, Electrical Engineering, or related field, or equivalent experience. They will have 8+ years of software engineering experience building performance-critical infrastructure, ML tooling, or distributed systems, and 3+ years of engineering leadership experience as a tech lead, TLM, or engineering manager.
Extensive experience with vLLM, TRT-LLM, or SGLang internals along with contributions to their upstream projects is highly desirable. Experience building Kubernetes-native infrastructure, including operators, Helm charts, and GPU observability tooling, is also valued. A background in competitive benchmarking frameworks such as MLPerf or equivalent industry-standard evaluation systems is a plus.
Widely considered to be one of the technology world's most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/