# Senior AI and ML HPC Cluster Engineer

**Company**: NVIDIA
**Location**: Santa Clara
**Work arrangement**: remote
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Senior-AI-and-ML-HPC-Cluster-Engineer_JR2016887-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_20da04f9-70e

## Description

As a member of the GPU AI/HPC Infrastructure team, you will provide leadership in the design and implementation of groundbreaking GPU compute clusters that run demanding deep learning, high-performance computing, and computationally intensive workloads.

Your key responsibilities will include:

- Providing leadership and strategic guidance on the management of large-scale HPC systems, including the deployment of compute, networking, and storage.

- Developing and improving our ecosystem around GPU-accelerated computing, including developing scalable automation solutions.

- Building and maintaining AI and ML heterogeneous clusters on-premises and in the cloud.

- Creating and cultivating customer and cross-team relationships to reliably sustain the clusters and meet user evolving user needs.

- Supporting our researchers to run their workloads, including performance analysis and optimizations.

- Conducting root cause analysis and suggesting corrective action.

To succeed in this role, you will need:

- A Bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent experience.

- Minimum 5+ years of experience designing and operating large-scale compute infrastructure.

- Experience with AI/HPC advanced job schedulers, such as Slurm, K8s, PBS, RTDA, or LSF.

- Proficient in administering CentOS/RHEL and/or Ubuntu Linux distributions.

- Solid understanding of cluster configuration management tools, such as Ansible, Puppet, Salt.

- In-depth understanding of container technologies, like Docker, Singularity, Podman, Shifter, Charliecloud.

- Proficiency in Python programming and bash scripting.

- Applied experience with AI/HPC workflows that use MPI.

- Experience analyzing and tuning performance for a variety of AI/HPC workloads.

If you have a background with NVIDIA GPUs, CUDA programming, NCCL, and MLPerf benchmarking, experience with machine learning and deep learning concepts, algorithms, and models, familiarity with InfiniBand with IPoIB and RDMA, understanding of fast, distributed storage systems like Lustre and GPFS for AI/HPC workloads, and familiarity with deep learning frameworks like PyTorch and TensorFlow, you will stand out from the crowd.

You will also be eligible for equity and benefits.

## Skills

### Required
- AI/HPC advanced job schedulers
- CentOS/RHEL and/or Ubuntu Linux distributions
- cluster configuration management tools
- container technologies
- Python programming
- bash scripting
- AI/HPC workflows
- MPI
- performance analysis and optimization

### Nice to have
- NVIDIA GPUs
- CUDA programming
- NCCL
- MLPerf benchmarking
- machine learning and deep learning concepts
- algorithms and models
- InfiniBand with IPoIB and RDMA
- Lustre and GPFS for AI/HPC workloads
- PyTorch and TensorFlow

---

Source: [Apply at nvidia.wd5.myworkdayjobs.com](https://nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAExternalCareerSite/job/US-CA-Santa-Clara/Senior-AI-and-ML-HPC-Cluster-Engineer_JR2016887-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)