Software Engineer Intern

NVIDIA is seeking a highly motivated software engineer intern to join our communication libraries and network software team. As an intern, you will be part of a fast-paced team that develops and maintains software for complex heterogeneous computing systems that power disruptive products in High Performance Computing and Deep Learning.

Responsibilities:

• Design, implement, and maintain highly-optimized communication runtimes for Deep Learning frameworks (e.g., NCCL for TensorFlow/Pytorch) and HPC programming interfaces (e.g., UCX for MPI/OpenSHMEM) on GPU clusters.
• Participate in and contribute to parallel programming interface specifications like MPI/OpenSHMEM.
• Design, implement, and maintain system software that enables interactions among GPUs and interactions between GPUs and other system components.
• Create proof-of-concepts to evaluate and motivate extensions in programming models, new designs in runtimes, and new features in hardware.

Requirements:

• Pursuing a Ph.D. in Computer Engineering, Computer Science, or Electrical Engineering with a strong background in computer architecture, operating systems, communication library, and/or AI/ML.
• Excellent C/C++ programming and debugging skills.
• Strong experience with Linux.
• Experience with parallel programming interfaces and communication runtimes.
• Ability and flexibility to work and communicate effectively in a multi-national, multi-time-zone corporate environment.

Preferred Qualifications:

• Deep knowledge of high-performance networks like InfiniBand, RoCE, etc.
• Background with HPC applications.
• Experience with Deep Learning Frameworks such as PyTorch, TensorFlow, JAX/XLA, vLLM/SGLang, etc.
• Experience with AI/DL communication patterns such as Expert Parallelism (EP), TP, DP, PP, and how these patterns can be implemented with NCCL.
• Experience with CUDA kernel optimization and profiling.
• Experience with large-scale model training and production inference software stack.
• Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.