Principal AI Engineer - Context - Agents and Context

Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale , unleashing the potential of businesses and people.

The Context Engine team builds the knowledge layer that AI agents use to work with enterprise data in Elasticsearch. As a Principal AI Engineer, you own the improvement loop of this product end to end: how agents, automations and skills behave in production, how we observe them, how we evaluate them, and how we ship changes to them safely.

Responsibilities

• Own the production improvement loop for Context Engine: understand how extraction automations, retrieval tools and memory behave, based on offline evaluations and customer conversations and telemetry.
• Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and the guardrails that let us change behaviour without breaking customers.
• Design the telemetry we need to make data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval, how it lands in Elasticsearch, and how it feeds evaluation, dashboards and the feedback loop.
• Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
• Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write the technical proposals that shape the roadmap.

Requirements

• 10+ years of software engineering experience, with the recent years spent shipping and operating AI-driven products on real production traffic.
• A track record of eval-driven product improvement: you have diagnosed agent or LLM behaviour from traces and user feedback, designed the evaluation that exposed the problem, shipped the fix and measured the outcome.
• Direct experience building agents with state and memory, and iterating on prompts, skills and tool behaviour safely in production.
• Familiarity with MCP, including exposing public MCP servers and tools.
• Experience designing telemetry for AI systems, and using it to make engineering and product decisions.
• Strong backend engineering skills in either Python or TypeScript: APIs, stateful workflows, data pipelines and production services.
• Comfort working with data scientists, engineers, and product managers as peers, translating between measurement and shipping, and communicating trade-offs clearly.

Nice to Have

• Experience with agent frameworks such as LangGraph, CrewAI, Claude Agent SDK, or similar.
• Agentic retrieval experience, including knowledge representation.
• Practical Elasticsearch experience.

Compensation

The typical starting salary range for this role is: 369 400 zł-584 300 zł PLN