Principal Software Engineer, At-Scale Reliability and Fleet Intelligence — CSP Engagements

We are seeking a Principal Software Engineer to join our CSP Engagements team, focusing on fleet-scale reliability. You will work directly with key CSP/hyperscale customers' engineering teams to ensure NVIDIA platforms achieve target MTBI in production.

Responsibilities:

• Drive reliability work streams with CSP engineering teams, ensuring shared understanding of MTBI measurement methodology, failure classification, and health monitoring architecture.

• Gather and synthesize CSP fleet reliability data, identifying failure patterns and championing improvements back into NVIDIA's firmware, driver, and hardware teams.

• Define a consistent MTBI measurement methodology that works across different CSP monitoring environments and operational practices.

• Conduct fleet-scale failure pattern analysis using statistical methods to classify failures as systemic, environmental, or configuration-specific.

• Drive fleet health monitoring integration architecture, ensuring NVIDIA's health agents, telemetry, and reporting align with CSP operational workflows and automation.

• Define burn-in reliability test environment and cluster certification criteria in collaboration with quality teams.

• Collaborate with CSPs to ensure reliability-related integration work is complete ahead of at-scale launch.

• Develop predictive failure models using fleet telemetry and validate their effectiveness in customer environments.

Requirements:

• 15+ years of experience in systems software at datacenter scale or reliability engineering with focus on at-scale challenges.

• BS or MS in Computer Science, Electrical Engineering, Statistics, or related field.

• Deep expertise in multi-NUMA, rack-scale system software and firmware, and statistical failure analysis methods.

• Experience with fleet-level telemetry and observability systems.

• Understanding of hardware failure modes in large-scale GPU/accelerator deployments.

• Experience defining or operating burn-in, stress testing, or certification frameworks for complex hardware systems.

• Customer obsession and strong communication skills.

Benefits:

• Equity eligibility

• Benefits package

This is a full-time position, and applications will be accepted until June 30, 2026.