# Lead AI QA Engineer

**Company**: Kobie
**Location**: Bengaluru, Karnataka
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Category**: Data Science
**Industry**: Technology

**Apply**: https://jobs.lever.co/kobie/3280d456-5db2-4070-a7eb-d879c17c1393?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_5aa6423b-f24

## Description

Join Kobie's India Tech Hub as a Lead AI QA Engineer. You'll design test strategies and evaluation harnesses for production-grade, agentic AI systems, and work with ETL. You need 6+ years of experience, strong Python skills, and hands-on experience testing LLM-powered features.

Responsibilities:

- Design and build evaluation harnesses for agentic systems in Python

- Author automated test suites for prompts, tools, structured outputs, and end-to-end agent workflows

- Validate guardrails around tool execution

- Wire evaluations into CI using Dataiku Evaluations, GitHub Actions, or Jenkins

- Build observability into testing

- Own quality end-to-end

- Partner with data engineers and platform teams

- Help shape internal QA standards for AI & Data engineering

Requirements:

- 3+ years of professional QA / SDET experience

- 1+ years of hands-on experience testing LLM or AI features in production

- Working knowledge of evaluation frameworks such as RAGAS, DeepEval, LangSmith, Langfuse

- Strong Python and PyTest skills

- Solid SQL skills and comfort with at least one cloud platform

- Fluency with Git, Docker, REST APIs, and at least one CI tool

- Solid understanding of data security and responsible AI practices

Nice to Have:

- Hands-on experience with Dataiku DSS

- Experience with Dataiku LLM Mesh, Knowledge Banks, Prompt Studio, or Visual / Code Agents

- Experience with Snowflake, Snowpark, or Snowflake Cortex

- Experience with red-teaming, prompt-injection testing, or adversarial test generation for LLMs

## Skills

### Required
- Python
- PyTest
- SQL
- Git
- Docker
- REST APIs
- Dataiku
- Snowflake
- LLM
- AI

### Nice to have
- Dataiku DSS
- Dataiku LLM Mesh
- Snowpark
- red-teaming
- prompt-injection testing

---

Source: [Apply at jobs.lever.co](https://jobs.lever.co/kobie/3280d456-5db2-4070-a7eb-d879c17c1393?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)