Description
Splunk Cloud is looking for an operations and automation engineer to support Network Operations Center (NOC) responsibilities and manage large-scale Splunk Cloud deployments.
You will monitor and troubleshoot distributed cloud systems, respond to production incidents, and turn manual operational runbooks into safe, repeatable workflows. You will work with operations and engineering teams to improve how we detect, diagnose, and resolve service issues while reducing manual toil.
Responsibilities:
- Monitor and support large-scale Splunk Cloud deployments, ensuring service reliability, availability, and performance.
- Respond to monitoring alerts and production incidents according to defined playbooks and procedures.
- Participate in 24/7 operations and on-call support.
- Investigate complex issues across cloud infrastructure, Linux systems, networking, distributed services, and application code.
- Participate in post-incident reviews and turn findings into operational and automation improvements.
- Identify repetitive, manual, or error-prone operational tasks that are suitable for automation.
- Convert operational runbooks into reliable automated workflows.
- Use Git-based development, peer review, documentation, and cross-functional collaboration to maintain and improve operational automation.
Minimum qualifications:
- B.S. in a related field or equivalent work experience (5+ years).
- 3+ years of experience in systems or network administration in a cloud environment, including incident response and major incident management.
- Proven experience developing maintainable Python/Go automation, operational tooling, or production support utilities.
- Experience using Unix or Linux systems and shell scripting.
- Experience using Git and participating in code review workflows.
- Working knowledge of software engineering practices, including modular design, testing, debugging, exception handling, logging, and documentation.
- Ability to translate a manual operational procedure into a safe, repeatable automated workflow.
- Understanding of production automation risks, including permissions, secrets management, validation, rollback, and human approval points.
- Strong troubleshooting, prioritization, collaboration, and communication skills, including the ability to remain effective during major service outages.