Description
Millions of people around the world come to Pinterest to find creative ideas, dream about new possibilities, and plan for memories that will last a lifetime.
At Pinterest, AI isn't just a feature, it's a powerful partner that augments creativity and amplifies impact. The AI Platform team delivers essential tools and infrastructure used by hundreds of AI and ML engineers, powering crucial functions like recommendations, ads, and visual search.
As a Senior Staff Software Engineer on the AI Platform team, you will shape and scale Pinterest's AI/ML evaluation platform into a unified, reliable system adopted company-wide. You will identify modernization opportunities, elevate the roadmap, and execute technical strategy for a durable evaluation system for GenAI and ML workloads.
Responsibilities:
- Identify modernization opportunities and scale the roadmap for a company-wide evaluation system
- Define cross-team technical strategy and drive broad adoption of shared evaluation capabilities
- Make durable architectural decisions that improve the user experience and reliability of Pinterest's AI/ML evaluation platform
- Establish evaluation methodologies and standards that balance automated metrics, expert judgment, human feedback, safety, and business outcomes
- Lead complex cross-team workstreams and deliver impact
Requirements:
- 3+ years of experience designing and operating evaluation systems for GenAI and complex ML systems
- Strong programming skills in Python and solid grasp of distributed systems principles
- Deep familiarity with GenAI and RecSys evaluation libraries and frameworks
- 5+ years of experience building scalable platforms for distributed experimentation, data and model versioning, and production monitoring
- Strong judgment around GenAI risks, including robustness, bias, privacy, security, and limitations of automated evaluation signals
- Bachelor's degree in Computer Science, Engineering, or a related field
Benefits:
- Base salary range: $208,592-$429,454 USD
- Equity eligibility
- Flexible working model
In-office requirement: 1-2 times/quarter Relocation statement: Not eligible for relocation assistance