Description
We're seeking a highly analytical and strategic thinker to take ownership of our model evaluation analysis and insight generation. As an AI Product Manager, Insights, you will be responsible for creating model evaluation from initial hypothesis to final publication, going beyond aggregate metrics to deeply analyse why the model failed and identifying semantic patterns, edge cases, and systemic hallucinations in raw model outputs.
Key responsibilities include:
- Owning the creation of model evaluation from initial hypothesis, data scraping to final publication.
- Deeply analysing why the model failed and identifying semantic patterns, edge cases, and systemic hallucinations in raw model outputs.
- Reviewing raw data sets, meeting transcripts, and research notes to identify the 'so what' and turning these findings into a logical hierarchy.
- Acting as the bridge between the data and the narrative by structuring findings into a logical hierarchy where the most critical 'hook' lands first, followed by the supporting evidence.
As a successful candidate, you will have 5-10 years of experience in DS, ML, AI research and analysis, be a structured thinker, have a high tolerance for ambiguity, and possess executive presence. Experience in Model Evaluation, ML Engineering or Technical Research is a plus.
This role offers a competitive compensation package, including base salary, equity, and benefits. The base salary range for this full-time position in San Francisco is $205,600-$257,000 USD.