# Senior System Software Architect, HPC and AI Networking

**Company**: NVIDIA
**Location**: Shanghai
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/China-Shanghai/Senior-System-Software-Architect--HPC-and-AI-Networking_JR1997487?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_f9401ee3-b43

## Description

NVIDIA is seeking a forward-thinking HPC and AI Inference Software Architect to shape the future of scalable AI infrastructure, focusing on distributed training, real-time inference, and communication optimization.

### What you will be doing:

- Design and prototype scalable software systems optimizing distributed AI training and inference, focusing on throughput, latency, and memory efficiency.

- Develop and evaluate enhancements to communication libraries like NCCL, UCX, and UCC for deep learning workloads.

- Collaborate with AI framework teams (e.g., TensorFlow, PyTorch, JAX) to improve integration, performance, and reliability of communication backends.

- Co-design hardware features in GPUs, DPUs, or interconnects that accelerate data movement and enable new capabilities for inference and model serving.

- Contribute to the evolution of runtime systems, communication libraries, and AI-specific protocol layers.

- Collaborate with customers to understand their needs and provide innovative solutions.

### What we need to see:

- Ph.D, Masters, or Bachelors in computer science, computer engineering, electrical engineering, or a closely related field.

- 5+ years of experience in DNNs, scaling of DNNs, parallelism of DNN frameworks, or deep learning training workloads.

- Deep understanding of inference and training workloads and optimizations like prefill/decode, data parallelism, tensor parallelism, FDSP, etc.

- Experience with AI network parallelism using collective libraries and RDMA/RoCE.

- Background in algorithm design, system programming, and computer architecture.

- Strong programming and software development skills.

- Ability and flexibility to work and communicate effectively in a multi-national, multi-time-zone corporate environment.

### Ways to stand out from the crowd:

- Deep understanding of technology and passion for what you do.

- Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

- Background with designing communication middleware for high-performance computing systems, including RoCE and DPUs.

- Background with CUDA programming and NVIDIA GPUs and programming models for emerging architectures.

## Skills

### Required
- DNNs
- parallelism of DNN frameworks
- deep learning training workloads
- inference and training workloads optimizations
- AI network parallelism
- collective libraries
- RDMA/RoCE
- algorithm design
- system programming
- computer architecture
- strong programming and software development skills

### Nice to have
- designing communication middleware for high-performance computing systems
- RoCE
- DPUs
- CUDA programming
- NVIDIA GPUs

---

Source: [Apply at nvidia.wd5.myworkdayjobs.com](https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/China-Shanghai/Senior-System-Software-Architect--HPC-and-AI-Networking_JR1997487?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)