# Site Reliability Engineer

**Company**: SpaceXSI
**Location**: Southaven, MS
**Work arrangement**: onsite
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://job-boards.greenhouse.io/xai/jobs/5266379007?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_56648f34-99f

## Description

SpaceXSI was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceXSI is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.

As a Site Reliability Engineer focused on campus reliability, you will design what the campus watches and trusts, technically command cross-discipline SEVs, and build the guardrails that make the next incident smaller. You are the connective tissue across compute, network, storage, power, and cooling on one of the largest AI training campuses in the world.

Responsibilities:

- Own monitoring architecture and signal quality across GPU and host telemetry and facility telemetry

- Provide SEV command support: technical incident leadership, bridge coordination with the NOC, paging and escalation

- Run blameless postmortems and drive corrective actions to closed

- Gate high-risk change: fleet-scale firmware/BIOS/BMC rollouts and SOP/MOP/EOP-governed work on power and cooling systems

- Lead cross-functional reliability projects spanning compute, network, storage, and facility signal boundaries

- Partner with Hardware Failure Analysis on systemic failure classes

- Build and maintain playbooks, run game days, and keep cross-discipline dependency maps current

- Define error budgets and availability objectives at campus and service boundaries

- Participate in on-call rotations and incident response for SEV-class events

Profile Signals:

- Experience in power plants, nuclear power plants, data centers, or facilities like SpaceXSI

- Proven large-scale incident command experience and calm technical leadership on a bridge

- Demonstrated monitoring and observability design at fleet or campus scale

- Cross-team facilitation and systems engineering depth across software and facility boundaries

Success Measured By:

- MTTD / MTTR trend for SEV-class events

- % of SEVs with a blameless postmortem and closed action items

- Alert actionable ratio

- Recurrence rate of incident classes

- Monitoring coverage of critical dependencies across compute, network, storage, power, and cooling

- GPU fleet availability lost to infrastructure events

Basic Qualifications:

- Bachelor's degree in Systems Engineering, Computer Science, Electrical Engineering, or a related field

- 5+ years of experience in site reliability, systems engineering, plant operations, or large-scale production operations

- Proven large-scale incident command experience and calm technical leadership on a bridge

- Demonstrated monitoring and observability design at fleet or campus scale

- Hands-on experience with at least one of: Prometheus/Grafana, NVIDIA DCGM, or industrial telemetry and controls

- Experience working across at least two of: compute, network, storage, power, and cooling / facilities telemetry

- Experience writing and operating playbooks, runbooks, or SOP/MOP/EOPs

- Proficiency in scripting for automation and analysis

- Excellent problem-solving skills with a data-driven approach to reliability engineering

- Ability to work collaboratively with cross-functional teams

## Skills

### Required
- Prometheus/Grafana
- NVIDIA DCGM
- industrial telemetry and controls
- scripting
- systems engineering
- site reliability
- incident command
- monitoring and observability
- cross-functional teams

### Nice to have
- power plants
- nuclear power plants
- data centers
- AI/ML infrastructure
- supercomputing environments
- fleet firmware and BMC management
- reliability engineering methods
- CMMS
- commissioning / integrated systems testing
- operational turnover
- SLOs
- SLIs
- error budgets
- game days
- dependency mapping
- closed-loop corrective action programs

---

Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/xai/jobs/5266379007?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)