Performance Software Intern for Deep Learning Libraries

We are seeking a Performance Software Intern for Deep Learning Libraries to join our team. As a deep learning library performance software intern, you will be developing optimized code to accelerate linear algebra and deep learning operations on NVIDIA GPUs.

Responsibilities:

• Writing highly tuned compute kernels to perform core deep learning operations (e.g., matrix multiplies, MoE, Attention)
• Following general software engineering best practices, including support for regression testing and CI/CD flows
• Collaborating with teams across NVIDIA:
• Compiler team on generating optimal assembly code
• Deep learning training and inference performance teams on which layers require optimization
• Hardware and architecture teams on the programming model for new deep learning hardware features

Requirements:

• Pursuing Masters or PhD degree in Computer Science, Computer Engineering, Applied Math, or related field
• Demonstrated strong programming and software design skills, including debugging, performance analysis, and test design
• Experience with performance-oriented parallel programming, even if it's not on GPUs (e.g., with OpenMP or pthreads)
• Solid understanding of computer architecture and some experience with assembly programming
• Identify bottlenecks, optimize resource utilization, and improve throughput.

Nice to Have:

• Tuning deep learning library kernel code
• CUDA GPU programming
• Numerical methods and linear algebra
• LLVM, TVM tensor expressions, or TensorFlow MLIR