Description
Grafana Labs is seeking a Staff Software Engineer to focus on database reliability engineering for Grafana Cloud. The successful candidate will build and operate systems, write software and automation changes, and handle on-call responsibilities for high-SLA customers.
Responsibilities
- Build reliability into the product by writing production code in services, tooling, and codebases like Mimir, Loki, and Tempo.
- Develop automation and platform tooling to remove toil and scale reliability practices.
- Lead technical design by authoring design documents, reviewing designs and PRs, and shaping feature design for scalability and operability.
- Own the SLO framework by defining and evolving per-tenant SLOs and reliability models.
- Improve observability and alerting to find problems before customers do.
- Be on-call for high-SLA customers, handling incident response and post-incident reviews.
- Mentor engineers and teach reliability practices.
Requirements
- 8+ years of software engineering experience, including significant time building and owning distributed systems in production.
- Strong production software skills, ideally in at least one object-oriented language.
- Deep understanding of distributed systems, performance, scaling, and failure modes.
- Experience running multi-tenant systems at scale on Kubernetes in AWS, GCP, or Azure.
- Hands-on experience designing and implementing SLOs.
- Experience being on-call for production systems.
- Staff-level technical leadership.
Nice to Have
- Experience with Mimir, Loki, Tempo, Pyroscope, Prometheus, or other observability and time-series databases.
- Open-source contributions.
- Customer reliability engineering (CRE) or customer-facing escalation experience.
Compensation and Benefits
- The base compensation range for this role in Germany is €109,709 - €131,651.
- Benefits include equity, bonus (if applicable), and other benefits.