Description
Every transaction made by a Zopa customer passes through our transaction enrichment pipeline. We tag the merchant and assign a spending category, so customers can understand their money better and the rest of the bank can build on reliable data. This role is about making that classification measurably better.
As an Applied AI Analyst, you will design and tune the prompts that do the categorising, work out where and why they fall short, and help shape what we build next. At heart this is an analytical role, but one where product thinking matters as much as technical ability.
Responsibilities
- Design, test and refine the prompts that tag merchants and assign spending categories, and work out how transaction data should be prepared and passed to the model.
- Analyse enrichment results using Python and SQL to understand where and why enrichment falls short and making evidenced recommendations for improvements.
- Surface opportunities the team would not otherwise: spot patterns in the data that point to a product gap, a recurring failure mode that suggests a systemic fix.
- Design and run evaluations, including LLM-as-judge approaches, to analyse whether a change improves things or introduces new problems.
- Build quick, scrappy prototypes and test harnesses to test a hypothesis with real data before any engineering resource is committed.
- Shape the future of the product, for example how customers could re-categorise transactions, and how enriched data can support a wider range of Zopa products and AI-powered tools.
Requirements
- You think in experiments: you form a hypothesis before looking at the data, design something that could prove you wrong, and interpret results with appropriate scepticism.
- You are curious about messy, imperfect systems: you want to understand why something breaks, and you brainstorm ideas to improve it.
- You can describe what you want and build it together with an LLM, to explore data, clean outputs, call APIs and prototype ideas.
- You have some experience with LLM APIs: calling them, prompting them, structuring their outputs, and understanding why they sometimes give you nonsense.
- You keep the product goal in view: you care about what changes for customers and the business.
- You are comfortable owning ambiguous problems: you can scope the work, prioritise what to test first, and know when good enough is good enough.
- You are a clear communicator who can explain what you built, what you found and what it means to engineers, PMs and stakeholders who don’t care about the implementation details.
Nice to have
- Evaluating LLM outputs, including LLM-as-judge approaches, ground truth sets or evals.
- Prompt engineering and prompt tuning, and comparing LLMs on accuracy, cost and coverage.
- Classification, categorisation, labelling or entity-matching problems.
- Banking, fintech, payments or another fast-paced, data-led environment.
- Side projects or things you have built with LLMs simply because you wanted to.
Additional Information
At Zopa we value flexible ways of working. We value face-to-face collaboration and a good work-life balance. This hybrid role requires you to come to our London office 2-3 days a week. You'll also have the option of working from abroad for up to 120 days a year!