Description
As Microsoft continues to push the boundaries of AI, we are on the lookout for seasoned engineering leaders to help shape the future of our AI products and organisation. Our vision is bold and broad , to build systems that have true artificial intelligence across agents, applications, services, and infrastructure. It’s also inclusive: we aim to make AI accessible to all , consumers, businesses, developers , so that everyone can realise its benefits.
We are seeking an experienced Senior Engineering Leader to build, scale, and run a high-performing engineering organisation responsible for Copilot AI Evaluation. This is a role for a proven people leader who has built and managed multi-team organisations , someone who has hired and developed engineering managers, set organisational strategy, and delivered large-scale technical programmes across multiple workstreams.
You will own the engineering vision for LLM evaluation at Copilot, partnering directly with senior Eng and Product leadership to define priorities, drive execution, and raise the bar on engineering excellence. We are looking for a leader who combines deep technical judgement with a passion for growing people and building high-trust, high-velocity teams.
Responsibilities:
- Build and lead a multi-team engineering organisation (30+ engineers across multiple teams), including hiring and developing engineering managers who lead their own teams.
- Set the technical and organisational strategy for Copilot AI Evaluation and response quality, aligning with MAI’s broader product and engineering vision.
- Partner with senior Eng and Product leadership (Partner+ level) to define priorities, influence roadmaps, and drive cross-organisational initiatives.
- Own end-to-end delivery of evaluation platforms, novel evaluation techniques, and agentic solutions for measuring and improving Copilot quality at scale.
- Recruit, develop, and retain world-class engineering talent , building a culture of technical excellence, accountability, and continuous learning.
- Drive operational rigor: establish engineering processes, quality bars, and delivery cadences that enable predictable, high-quality execution across multiple concurrent workstreams.
- Navigate ambiguity and make high-judgment tradeoff decisions on technology, staffing, and investment priorities in a fast-moving AI landscape.
- Foster a diverse, inclusive team culture where engineers at all levels can do their best work and grow their careers.
Qualifications:
- Bachelor’s Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, Javascript, or Python OR equivalent experience
- Demonstrated track record of building and scaling engineering organisations (hiring teams from scratch, structuring orgs, growing managers)
- Experience delivering large-scale software systems in AI, machine learning, or related fields
- Experience managing organisations of 30+ engineers across multiple teams and workstreams
- Deep expertise in LLM evaluation, AI quality measurement, or ML infrastructure at scale
- Track record of partnering with senior leadership (VP/CVP level) to set strategy and drive cross-organisational programmes
- Experience recruiting and developing senior engineering talent (principal engineers, engineering managers) in a competitive market
- Proven ability to operate effectively in fast-paced, ambiguous environments , comfortable making decisions with incomplete information and course-correcting quickly
- Strong technical judgement: ability to evaluate architectural tradeoffs, assess technical risk, and guide teams toward sound engineering decisions without needing to write the code yourself
- Experience leading distributed or multi-site engineering teams
Experience Level: senior Employment Type: full-time Workplace Type: hybrid Category: Engineering Industry: Technology Salary Range: $139,900 - $274,800 per year Salary Min: 139900 Salary Max: 274800 Salary Currency: USD Salary Period: year Required Skills: C, C++, C#, Java, Javascript, Python Preferred Skills: LLM evaluation, AI quality measurement, ML infrastructure