Description
As a Data Scientist in Safety Systems, you will establish the data-driven approach for understanding, evaluating, and monitoring the safety of our production systems. You should expect to collaborate with our partners across the company to define north-star metrics, own and implement the statistical methods to productionize those metrics, conduct analysis to understand the impact of our products, and establish source-of-truth dashboards that the entire company can use to answer safety-related questions. Most importantly, you will be a core member of the Safety Systems team, collaborating with researchers and engineers to advance our goals of safe, robust, and reliable AI.
This role is based in our San Francisco HQ. We offer relocation assistance to new employees.
In this role, you will:
- Lead our efforts in understanding and measuring the real-world safety impacts of OpenAI’s current and upcoming products
- Uncovering new ways to improve our approaches to measuring and mitigating harm and abuse
- Develop and implement statistical methods necessary to operationalize safety-related metrics
- Provide direction, guidance, and coordination of projects in the space
- Establish a data-driven culture within Safety Systems by driving the definition, tracking, and operationalizing of feature-, product-, and company-level metrics
- Create and disseminate dashboards, reports, and tools that enable the team and company to answer safety-related questions independently
- Develop safety data flywheel and provide safety research with production insights/data for training and evaluation.
You might thrive in this role if you have:
- 5+ years experience in a quantitative role navigating highly ambiguous environments, ideally as a founding data scientist or team lead at a hyper-growth product company or research org
- Proven leadership skills, including leading multiple data scientists and cross-functional teams
- Expertise in defining and implementing metrics, with a track record of operationalizing new feature and product-level metrics from scratch
- Excellent communication skills with demonstrated ability to communicate with product managers, engineers, and executives alike
- Strategic insights that extend beyond traditional statistical significance testing.
You could be an especially great fit if you have:
- Experience in trust and safety, integrity, anti-abuse, or related fields
- Demonstrated prior experience in NLP, large language models, or generative AI
- Strong statistical background, including knowledge of sampling, regression, causal analysis, and more