# Senior/Staff Evaluation Data Analyst (Agentic Search)

**Company**: Nebius
**Location**: London
**Experience**: senior
**Job type**: full-time
**Category**: Data Science
**Industry**: Technology

**Apply**: https://job-boards.greenhouse.io/nebius/jobs/5027589101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_b29b4f79-50f

## Description

We are looking for an experienced Data Analyst who has built evaluation systems for complex technology products and is excited to take end-to-end ownership of quality in search built for AI agents.

In this position, you will:

- Own the quality measurement and improvement cycle end-to-end: identify user needs, define success, build evaluations, diagnose failures, and work with engineers and product managers to improve outcomes.

- Design, build, and evolve evaluation systems, including datasets, benchmarks, metrics, and automated pipelines that support development and launch decisions.

- Evaluate response quality across multiple dimensions, including relevance, completeness, freshness, factual accuracy, and usefulness to downstream agents.

- Investigate quality regressions and metric anomalies, uncover root causes, and identify the improvements with the greatest potential impact.

- Help prioritize the product and engineering roadmap, making quality trade-offs explicit and assessing whether changes deliver meaningful improvements for users.

We expect you to have:

- Substantial hands-on experience designing and building evaluation or quality measurement systems for complex technology products, such as search, AI agents, voice technologies, autonomous driving, recommendation systems, advertising, or other similar complex systems.

- A track record of owning ambiguous problems from initial framing through implementation, diagnosis, and measurable product improvement.

- Strong Python skills and an engineering mindset: you can turn analytical ideas into reliable, maintainable pipelines and tools.

- A solid foundation in mathematical statistics and practical experience with experimental design, sampling, uncertainty, and measurement bias.

- Strong analytical judgment: you understand what a metric captures, what it misses, and whether an observed improvement matters to users.

- The ability to reason across user needs, product behavior, and technical systems, and to translate that understanding into actionable evaluations.

- Strong communication and collaboration skills, with the ability to influence technical and product decisions.

- The independence to make progress in a fast-moving environment where requirements, methods, and definitions of success are still evolving.

Benefits & Perks:

- Competitive compensation

- Career growth and learning opportunities

- Flexibility and ownership

- Collaborative and innovative culture

- Opportunity to work on impactful AI projects

- International environment and talented teams

## Skills

### Required
- Python
- mathematical statistics
- experimental design
- sampling
- uncertainty
- measurement bias

---

Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/nebius/jobs/5027589101?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)