Description
The Personalization team at Spotify makes deciding what to play next easier and more enjoyable for every listener. As a Senior Product Quality Analyst in the Voice Production Team, you will bring a dedicated evaluation perspective to the team, focusing on defining and measuring AI voice quality.
Your goal will be to help determine whether Spotify's AI voice technology serves the company's specific needs. You will own the quality bar for Spotify's AI voice models, defining what 'good enough' means for different products, audiences, languages, and use cases.
Key responsibilities include:
- Developing evaluation rubrics and approaches to measure AI voice quality
- Planning and carrying out evaluation campaigns
- Establishing an ongoing evaluation rhythm
- Benchmarking Spotify's AI voices against third-party models and competitors
- Determining appropriate quality trade-offs for additional languages and regions
- Partnering with voice experts, machine learning teams, and product partners
- Surfacing gaps in model performance and influencing future research priorities
Requirements:
- Experience in product quality, evaluation, or a related field
- Ability to define evaluation criteria and turn subjective qualities into clear rubrics
- Comfort with designing and running qualitative and MOS-style evaluations
- Understanding of voice or audio quality
- Strong judgment and ability to maintain a consistent quality bar
- Autonomy and clear communication across teams
- Curiosity about generative AI and speech technology
This role is based in New York or Los Angeles, with flexibility to work from home. The United States base range for this position is $114,486–$163,552 USD, plus equity.