# Research Engineer / Scientist, Data & Evals

**Company**: Harvey
**Location**: London, England
**Work arrangement**: hybrid
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://jobs.ashbyhq.com/harvey/6708bb0c-e466-44e4-a992-486ad3300fd6?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_72af6710-252

## Description

At Harvey, we're transforming how legal and professional services operate by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise.

As a Member of Technical Staff, Data & Evals in Harvey Research, you'll build the data and measurement systems that power our post-training, benchmarking, and agent capability efforts.

## Responsibilities

- Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for complex legal reasoning and agentic workflows.

- Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail.

- Build and maintain capability benchmarks like our Legal Agent Benchmark (LAB), along with product-focused evals that track how models perform on real customer workflows.

- Work with legal domain experts to define quality metrics, rubrics, and other grading mechanisms for training and evaluation.

- Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination.

- Close the loop from product usage and customer feedback to new evaluations and training data, so models improve where customers need them most.

- Partner with researchers to investigate model failures, distinguish real gains from misleading metrics, and use findings to guide training and release decisions.

## Requirements

- Experience building data pipelines, ML evaluation systems, or experimentation tooling, with strong Python skills and ownership of production-quality code.

- A rigorous approach to measurement, including statistical reasoning, experimental design, and understanding when a metric can mislead.

- The ability to translate ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria.

- Experience with synthetic data, LLM evaluations, automated raters, or preference data is a plus.

- Strong data and measurement engineering experience close to ML model development is valuable even without prior LLM work.

## Skills

### Required
- Python
- data pipelines
- ML evaluation systems
- statistical reasoning
- experimental design

### Nice to have
- synthetic data
- LLM evaluations
- automated raters
- preference data

---

Source: [Apply at jobs.ashbyhq.com](https://jobs.ashbyhq.com/harvey/6708bb0c-e466-44e4-a992-486ad3300fd6?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)