Description
As Microsoft continues to push the boundaries of AI, we are on the lookout for individuals to work with us on the most interesting and challenging AI questions of our time. Our vision is to build systems that have true artificial intelligence across agents, applications, services, and infrastructure. We aim to make AI accessible to all , consumers, businesses, developers , so that everyone can realize its benefits.
We are seeking experienced Member of Technical Staff – Copilot AI Evaluation Engineering Manager to lead building our LLM evaluation solutions for Copilot. You’ll work in a deeply iterative, collaborative, fast-paced environment to improve our technology for millions of users around the world. You will lead a team of talented software engineers and applied AI engineers to deliver world-class evaluation solutions that are secure, reliable, and scalable, while collaborating closely with cross-functional teams including Product and Research.
Responsibilities:
- Hire, manage, and lead a team of software engineers, AI engineers, and machine learning engineers responsible for delivering world-class LLM evaluation solutions.
- Collaborate with Eng and Product leadership to prioritize features and improve our world-class AI companion to the world.
- Design and build evaluation platforms, novel evaluation techniques, and agentic solutions for measuring and improving copilot quality.
- Drive implementation of features and systems, breaking down long-term goals into clear milestones, aligning with release plans, and ensuring cross-team coordination.
Qualifications:
- Bachelor’s Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python.
- 3+ years people management experience.
- Hands on experience with ML and/or AI evaluation OR equivalent experience.
- Experience leading engineering teams to deliver large-scale software systems, preferably in AI, machine learning, graphics or related fields.
- Thrive in a fast-paced, collaborative environment and are comfortable making progress in ambiguity.
- Enjoy working closely with cross-functional partners and teammates in an inclusive, curious culture.
- Have strong opinions about best investments to make in establishing the most delightful and performant AI companion engineering system.