# Senior Applied AI Engineer (Observability)

**Company**: Komodo Health
**Location**: San Francisco, CA
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Salary**: $230,000-$270,000 USD
**Category**: Engineering
**Industry**: Healthcare

**Apply**: https://job-boards.greenhouse.io/komodohealth/jobs/8512224002
**Canonical**: https://yubhub.co/jobs/job_956ba265-d67

## Description

We're hiring a Senior Applied AI Engineer to join our AI Platform/Observability team. You will be responsible for building the foundation that makes AI systems trustworthy at scale. This is a backend-heavy, systems-focused role in a greenfield environment.

Your starting point will be AI observability, evaluation, and production reliability. From there, your scope expands into broader platform ownership: agent systems, orchestration layers, and shared infrastructure. Observability is the entry point,not the end state.

In this role, you will:

- Build the observability and reliability foundation for AI systems across Komodo (logging, tracing, evaluation pipelines, feedback loops)

- Define how the organization measures LLM performance and quality in production (hallucinations, drift, latency, failure modes)

- Ship production-grade AI systems that improve platform reliability, scalability, and performance

- Lead design and architecture for complex applied AI systems (multi-agent workflows, tool-calling systems, model pipelines)

- Establish evaluation frameworks and experimentation practices (A/B testing, offline + online evaluation)

You will drive experimentation across the organization, set best practices, and integrate new AI techniques into Komodo’s broader engineering ecosystem.

Additional responsibilities include:

- Contributing to reusable infrastructure, patterns, and standards adopted across teams

- Partnering with product, data, and platform teams to shape AI-driven solutions

We're looking for someone with strong expertise in LLMs and prompt systems, agent orchestration, and tool/function calling. You should have hands-on experience with AI observability, evaluation, or monitoring systems, debugging and improving production AI behavior, and strong backend engineering skills.

As a Senior Applied AI Engineer, you will be part of a team that is shaping the future of healthcare intelligence. You will have the opportunity to work on cutting-edge projects, collaborate with experts in the field, and make a meaningful impact on the company's mission.

## Skills

### Required
- LLMs
- prompt systems
- agent orchestration
- tool/function calling
- AI observability
- evaluation
- monitoring systems
- debugging
- production AI behavior
- backend engineering