Description
As a Model Behavior Architect on the Function Calling team, you will be at the forefront of defining and measuring how Large Language Models (LLMs) use tools, invoke functions, and orchestrate complex agentic workflows.
Your role will involve interacting with models to identify areas for improvement in function calling and tool use behavior, gathering internal and external feedback to scope areas for improvement, designing and implementing evals, data guidelines, data generation, and synthetic tool environments and APIs, identifying and fixing edge case behaviors, and developing robust evaluation pipelines for the function-calling capabilities of our model candidates.
We are looking for individuals with a deep understanding of API design, structured outputs, and schema specification, engineering and code behavior, or LLM agents at work, including reasoning, planning, and multi-step tool use. Prior knowledge in training and optimizing model behavior is also essential. You should be an expert at building robust evaluations and thrive in dynamic and technically complex environments.
In this role, you will work collaboratively with AI Scientists to deliver innovative, out-of-the-box solutions to address real-world constraints.