Principal AI Engineer - Context - Agents and Context

Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale , unleashing the potential of businesses and people.

The Context Engine team builds the knowledge layer that AI agents use to work with enterprise data in Elasticsearch. As a Principal AI Engineer, you own the improvement loop of this product end to end: how agents, automations and skills behave in production, how we observe them, how we evaluate them, and how we ship changes to them safely.

Responsibilities

• Own the production improvement loop for Context Engine: understand how extraction automations, retrieval tools and memory behave, based on offline evaluations and customer conversations and telemetry.
• Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and the guardrails that let us change behaviour without breaking customers.
• Design the telemetry we need to make data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval, how it lands in Elasticsearch, and how it feeds evaluation, dashboards and the feedback loop.
• Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
• Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write the technical proposals that shape the roadmap.

Requirements

• 10+ years of software engineering experience, with the recent years spent shipping and operating AI-driven products on real production traffic.
• A track record of eval-driven product improvement: you have diagnosed agent or LLM behaviour from traces and user feedback, designed the evaluation that exposed the problem, shipped the fix and measured the outcome.
• Direct experience building agents with state and memory, and iterating on prompts, skills and tool behaviour safely in production.
• Familiarity with MCP, including exposing public MCP servers and tools.
• Experience designing telemetry for AI systems, and using it to make engineering and product decisions.
• Strong backend engineering skills in either Python or TypeScript: APIs, stateful workflows, data pipelines and production services.
• Comfort working with data scientists, engineers, and product managers as peers, translating between measurement and shipping, and communicating trade-offs clearly.

Benefits

• Competitive pay based on the work you do here and not your previous salary.
• Health coverage for you and your family in many locations.
• Ability to craft your calendar with flexible locations and schedules for many roles.
• Generous number of vacation days each year.
• Increase your impact - We match up to $2000 (or local currency equivalent) for financial donations and service.
• Up to 40 hours each year to use toward volunteer projects you love.
• Embracing parenthood with minimum of 16 weeks of parental leave.