Description
Grafana Labs is seeking a Staff Software Engineer to focus on database reliability engineering for their Grafana Cloud platform. The successful candidate will work on building and operating large-scale databases, specifically Mimir, Loki, Tempo, and Pyroscope, which are used by high-SLA customers across various regions and cloud providers.
Responsibilities
The role involves a combination of building and operating these systems. You will be responsible for writing production code, automation, and product changes to enhance database reliability. Additionally, you will be on-call for high-SLA customers, handling operational tasks, and contributing to incident response and post-incident reviews.
Key responsibilities include:
- Building reliability into the product by writing production code in services, tooling, and codebases like Mimir, Loki, and Tempo.
- Developing automation and platform tooling to remove manual toil and scale reliability practices.
- Leading technical design, authoring design documents, reviewing others' designs, and shaping feature development for scalability and operability.
- Defining and evolving per-tenant SLOs and reliability models.
- Improving observability and alerting to detect problems proactively.
- Acting as a technical escalation point for complex customer environments and leading incident response.
- Mentoring engineers and promoting reliability practices across teams.
Requirements
- 8+ years of software engineering experience, particularly with distributed systems in production.
- Strong production software skills, preferably in an object-oriented language.
- Deep understanding of distributed systems, including performance, scaling, and failure modes.
- Experience with multi-tenant systems on Kubernetes in AWS, GCP, or Azure, and infrastructure-as-code tools like Terraform or Helm.
- Hands-on experience with SLOs and incident response.
- Staff-level technical leadership experience.
- Working knowledge of Linux internals, networking, and cloud storage.
Nice to Have
- Experience with Mimir, Loki, Tempo, Pyroscope, Prometheus, or similar observability and time-series databases.
- Open-source contributions.
- Customer reliability engineering or customer-facing escalation experience.
Compensation and Benefits
- The base compensation range for this role in Sweden is SEK 878,578 - SEK 1,054,294.
- Benefits include equity, bonus (if applicable), and additional perks.