# Principal Engineer

**Company**: Cisco
**Location**: Seattle, Washington
**Work arrangement**: remote
**Experience**: senior
**Job type**: full-time
**Salary**: $241,400.00 to $322,600.00
**Category**: Engineering
**Industry**: Technology

**Apply**: https://cisco.wd5.myworkdayjobs.com/en-US/Cisco_Careers/job/Seattle-Washington-US/Principal-Software-Engineer--Applied-AI---Splunk-Observability-Cloud_2028421?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/job/principal-engineer-a917b67f-03d/

## Description

The Observability Cloud AI team at Cisco builds the intelligent AI SRE experience for Splunk Observability Cloud. As the Principal Engineer, you will define the technical trajectory for agentic investigation, ensuring AI solutions achieve production-grade trust and efficiency. You will architect critical evaluation systems and retrieval layers, establish production standards for AI agents, and mentor team members.

**Minimum Qualifications:**

- 12+ years of professional software engineering experience

- 3+ years of experience designing, building, and deploying production-grade LLM-based systems

- Experience developing evaluation frameworks for AI/ML systems

- Experience engineering retrieval and context pipelines for structured and semi-structured data

- Knowledge of production-grade engineering principles

**Preferred Qualifications:**

- Experience with Observability, SRE, or incident management domains

- Experience working with cloud-native infrastructure at scale

- History of active open-source contributions in AI, agent frameworks, or observability ecosystems

- Experience leading customer-zero or field-deployed programs

- Strong communication skills

**Benefits:**

- Medical, dental, and vision insurance

- 401(k) plan with Cisco matching contribution

- Paid parental leave

- Short and long-term disability coverage

- Basic life insurance

- Paid time away

- Optional 10 paid days per year to volunteer

For more fresh new jobs, check out [yubhub.co](https://yubhub.co/).

## Skills

### Required
- software engineering
- LLM-based systems
- evaluation frameworks
- retrieval pipelines
- production-grade engineering

### Nice to have
- Observability
- SRE
- incident management
- cloud-native infrastructure
- open-source contributions
- communication skills

---

Source: [Apply at cisco.wd5.myworkdayjobs.com](https://cisco.wd5.myworkdayjobs.com/en-US/Cisco_Careers/job/Seattle-Washington-US/Principal-Software-Engineer--Applied-AI---Splunk-Observability-Cloud_2028421?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)
