Description
We are seeking an experienced Site Reliability Engineering leader to build and lead a new SRE organization responsible for the production operation of a rapidly scaling AI supercomputing platform.
Responsibilities and Duties:
- Build the SRE Organization:
- Build and develop the team from its initial formation through full 24x7x365 production operations.
- Mentor engineers and team leads, develop successors, and build an organization capable of operating effectively without depending on any single individual.
- Work with leadership to forecast staffing requirements as the platform grows.
- Establish the Production Operating Model:
- Define the operating model for an SRE organization.
- Establish clear operational interfaces with Datacenter Operations, engineering teams, vendors, and other service owners.
- Establish and continuously improve production readiness standards, runbooks, operational procedures, and incident response practices.
- Engineer Reliability Into the Platform:
- Embed with platform engineering teams during development to ensure reliability, serviceability, observability, and operational requirements are incorporated into the platform.
- Lead the development of SLOs, operational health indicators, alerting standards, incident severity definitions, and reliability reporting.
- Build a culture in which recurring operational problems are engineered out through automation and improved observability.
- Lead Production Operations:
- Lead or participate in major production incidents.
- Establish a blameless post-incident review process.
- Serve as the senior operational authority for the SRE organization.
Required Skills and Experience:
- Significant experience leading or building an SRE, Production Engineering, Infrastructure Reliability, or comparable function.
- Experience taking a new or rapidly evolving platform through production readiness, launch, stabilization, and ongoing operation.
- Strong understanding of modern Site Reliability Engineering principles.
- Strong incident leadership experience.
- A strong systems engineering background with a working understanding of Linux, networking, storage, distributed systems, automation, and production infrastructure.
- Ability to operate effectively at both the leadership and hands-on technical levels.
- Demonstrated ability to lead, hire, mentor, develop, and retain a growing team of strong technical engineers.
Desired but Not Required:
- Experience in high-performance networking technologies, large-scale parallel or distributed storage systems, workload schedulers and orchestration platforms, production observability and service-level monitoring, custom or early-generation hardware, firmware, or environments.
What Success Looks Like:
- The SRE organization is staffed, trained and capable of supporting the platform through production launch and subsequent growth.
- Production readiness, observability, incident management and operational engineering practices are embedded into how the platform is developed and operated.
- The team has the technical depth, automation and operating discipline to resolve most production issues independently.
In addition to a competitive salary, Graphcore offers flexible working and a comprehensive benefits package designed to support your health, wellbeing and financial future.