# Senior/Staff Evaluation Data Analyst (Agentic Search)

**Company**: Nebius
**Location**: London
**Experience**: senior
**Job type**: full-time
**Category**: Data Science
**Industry**: Technology

**Apply**: https://job-boards.greenhouse.io/nebius/jobs/5027907101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/job/senior-staff-evaluation-data-analyst-agentic-dbbca888-3aa/

## Description

We're looking for an experienced Data Analyst to take end-to-end ownership of quality in search built for AI agents. This role combines hands-on engineering, analytical judgment, and mathematical rigor to shape technical decisions and product direction.

In this position, you will:

- Own the quality measurement and improvement cycle end-to-end: identify user needs, define success, build evaluations, diagnose failures, and work with engineers and product managers to improve outcomes.

- Design, build, and evolve evaluation systems, including datasets, benchmarks, metrics, and automated pipelines that support development and launch decisions.

- Evaluate response quality across multiple dimensions, including relevance, completeness, freshness, factual accuracy, and usefulness to downstream agents.

- Investigate quality regressions and metric anomalies, uncover root causes, and identify the improvements with the greatest potential impact.

- Help prioritize the product and engineering roadmap, making quality trade-offs explicit and assessing whether changes deliver meaningful improvements for users.

We expect you to have:

- Substantial hands-on experience designing and building evaluation or quality measurement systems for complex technology products.

- A track record of owning ambiguous problems from initial framing through implementation, diagnosis, and measurable product improvement.

- Strong Python skills and an engineering mindset.

- A solid foundation in mathematical statistics and practical experience with experimental design, sampling, uncertainty, and measurement bias.

- Strong analytical judgment and the ability to reason across user needs, product behavior, and technical systems.

- Strong communication and collaboration skills, with the ability to influence technical and product decisions.

- The independence to make progress in a fast-moving environment where requirements, methods, and definitions of success are still evolving.

Benefits & Perks:

- Competitive compensation

- Career growth and learning opportunities

- Flexibility and ownership

- Collaborative and innovative culture

- Opportunity to work on impactful AI projects

- International environment and talented teams

For more fresh new jobs, check out [yubhub.co](https://yubhub.co/).

## Skills

### Required
- Python
- mathematical statistics
- experimental design
- sampling
- uncertainty
- measurement bias

---

Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/nebius/jobs/5027907101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)