Software Engineer - AI Computing

NVIDIA's Software Architecture (SA) team is seeking a talented engineer to help bring new AI technology into different industries. You will contribute to designing AI computing platforms, analysing AI and HPC applications, and delivering value to customers. Your focus will be on defining and solving computational challenges in LLM inference and training acceleration, as well as network communication and data transfer optimization.

Responsibilities:

• Contribute to developing open-source inference frameworks like SGLang and vLLM, including feature and operator development, performance optimization, and model support.
• Develop and optimize KV cache offloading frameworks for LLM workloads, supporting multi-level cache offloading and reuse across CPU, SSD, and remote storage.
• Drive R&D on compute performance in distributed training and explore methods for performance optimization.
• Study computational challenges in machine learning systems, identify common needs and bottlenecks, and build example code, acceleration libraries, or frameworks.

Requirements:

• Master's degree or above in computer science, mathematics, electrical engineering, automation, or related fields.
• Over 5 years of working experience in the technology industry.
• Strong interest in accelerated computing, parallel computing, and heterogeneous computing.
• Solid programming skills, with a good understanding of data structures and computer systems fundamentals.
• Strong learning agility, adaptability, and ability to analyze technical problems.

Preferred Qualifications:

• Familiarity with heterogeneous computing, distributed training, parallel computing, or high-performance computing.
• Experience in performance analysis, performance modeling, or performance optimization.
• Contributions to open-source frameworks are a plus.
• Strong ability to define new problems and explore solutions.
• Proficiency with AI coding tools.

We offer competitive salaries and a generous benefits package.