# Senior Data Scientist

**Company**: Microsoft
**Location**: Redmond
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Salary**: USD $119,800.00 – $234,700.00 per year
**Category**: Data Science
**Industry**: Technology
**Ticker**: MSFT
**Wikidata**: https://www.wikidata.org/wiki/Q2283

**Apply**: https://microsoft.ai/job/senior-data-scientist-15/?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_5bfa4cb8-172

## Description

As a Senior Data Scientist for LLM Evaluation, you will develop and implement cutting-edge methodologies to help evaluate how well Copilot performs in real-world usage scenarios.

Users turn to Copilot for various tasks, making it crucial to ensure our AI systems effectively assist them.

Your responsibilities will include:

- Developing new methods to evaluate LLMs, train classifiers, and experiment with data collection techniques

- Implementing methodologies to provide real-time signals on Copilot performance

- Collaborating with user researchers and product leaders to build automated evaluation frameworks

The ideal candidate will have experience in social sciences, machine learning, and natural language analysis, with strong problem-solving skills and the ability to work independently.

**Responsibilities**

- Leverage expertise to measure Copilot performance, identify failure modes, and develop mitigation strategies

- Create and implement comprehensive evaluation frameworks across diverse scenarios

- Build automated testing systems and write efficient code for model pipelines

- Maintain a user-oriented perspective and serve as a trusted advisor on AI matters

- Track advances in research and adapt algorithms to drive innovation

**Qualifications**

- Doctorate or Master's degree in Data Science, Mathematics, Statistics, or related field with relevant experience

- Experience with large language models, Python programming, and Responsible AI

## Skills

### Required
- Data Science
- Machine Learning
- Natural Language Processing
- Python

### Nice to have
- Large Language Models
- Responsible AI

---

Source: [Apply at microsoft.ai](https://microsoft.ai/job/senior-data-scientist-15/?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)