# Software Engineer II - AI Performance

**Company**: Disney Entertainment and ESPN Product & Technology
**Location**: New York, NY
**Experience**: senior
**Job type**: full-time
**Salary**: $124,900.00 to $167,400.00 per year
**Category**: Engineering
**Industry**: Technology

**Apply**: https://disney.wd5.myworkdayjobs.com/en-US/disneycareer/job/New-York-NY-USA/Software-Engineer-II--AI-Performance_10148555?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_b2a37397-63d

## Description

Disney Entertainment and ESPN Product & Technology is a global organisation of engineers, product developers, designers, technologists, data scientists, and more – all working to build and advance the technological backbone for Disney's media business globally.

The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for their businesses. They are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers.

As a Software Engineer II on the Reliability Insights team, you will design and build data and reporting pipelines that evaluate and prove the real-world performance, precision, and business impact of Disney's operational AI and observability tools. You will bridge backend engineering and data-driven insights, transforming raw telemetry, AI agent interactions, and operational logs into clear metrics around AI reliability, model drift, and incident mitigation.

In this role, you will build automated ground-truth pipelines and model evaluation frameworks that quantify how effectively AI agents and anomaly models perform in production. You will capture end-user feedback, correlate AI signals with incident resolution timelines, and build single-pane dashboards for engineering and leadership.

**Responsibilities:**

- Design, build, and maintain automated feedback pipelines that capture operational outcomes and correlate them directly back to AI model predictions and agent actions.

- Develop programmatic evaluation frameworks to track model precision, recall, and drift over time, establishing automated threshold alerts to proactively trigger model tuning and prevent alert fatigue.

- Build end-to-end data pipelines linking AI outputs to incident lifecycles, quantifying business impact through metrics such as Service to Incident correlation, notification success rates, and Mean Time to Mitigation (MTTM).

- Design and deploy scalable APIs, metrics layers, and single-pane model health dashboards that present clear AI ROI, operational health, and performance narratives to leadership and technical stakeholders.

- Build mechanisms to capture structured feedback from end-users to continuously feed real-world usage signals back into model improvement cycles.

**Basic Qualifications:**

- 3+ years of applicable experience in backend and data pipeline engineering, including building scalable APIs and metrics/insight generation platforms.

- Strong experience with data platform engineering and distributed processing tools to process and analyse telemetry data.

- Practical understanding of model evaluation concepts, such as precision, recall, model drift, and ground-truth labelling systems.

- Hands-on experience with modern development practices, including version control, containerisation, and cloud-native deployments.

- Proficiency with AI-assisted development tools to accelerate engineering velocity.

- Strong analytical skills to translate complex system telemetry and operational logs into meaningful reliability metrics and business insights.

- Excellent cross-functional collaboration and communication skills, with a track record of partnering with operational teams to turn raw data into actionable insights.

**Preferred Qualifications:**

- Experience with observability, telemetry, and monitoring platforms.

- Familiarity with agentic workflows and foundation models and how to evaluate their output accuracy.

- Knowledge of incident management workflows and SRE metrics.

- Prior experience working on data attribution, metrics aggregation, or financial/performance visibility tooling in high-scale environments.

The hiring range for this position in New York, NY is $124,900.00 to $167,400.00 per year.

## Skills

### Required
- backend engineering
- data pipeline engineering
- scalable APIs
- metrics/insight generation platforms
- data platform engineering
- distributed processing tools
- model evaluation concepts
- modern development practices
- AI-assisted development tools
- analytical skills
- cross-functional collaboration

### Nice to have
- observability
- telemetry
- monitoring platforms
- agentic workflows
- foundation models
- incident management workflows
- SRE metrics
- data attribution
- metrics aggregation
- financial/performance visibility tooling

---

Source: [Apply at disney.wd5.myworkdayjobs.com](https://disney.wd5.myworkdayjobs.com/en-US/disneycareer/job/New-York-NY-USA/Software-Engineer-II--AI-Performance_10148555?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)