Description
We are looking for a skilled Storage AI Engineer to help ensure the quality, reliability, and safety of our AI-powered products.
As a QA Engineer on our AI product team, you will design and execute testing strategies for both traditional software features and AI-specific behaviors such as model accuracy, hallucination detection, prompt reliability, safety, bias, performance, and user trust.
Key Responsibilities
- Design, develop, and execute test plans for multi-agent systems
- Evaluate model behavior across real-world scenarios, edge cases, and adversarial inputs
- Test prompt flows, agent behaviors, recommendations, summarization, search, classification, and generative outputs
- Identify and document issues related to hallucinations, inaccurate outputs, bias, unsafe content, latency, and poor user experience
- Drive scalability, reliability, and operational excellence for production AI systems
- Build advanced tool orchestration and workflow automation solutions
- Build automated test frameworks for regression testing and continuous quality assurance
- Conduct manual exploratory testing for new AI features and workflows
- Create evaluation metrics and quality rubrics for AI product performance
- Collaborate with ML engineers and product teams to define success criteria for AI features
- Monitor production issues, analyze root causes, and drive quality improvements
- Perform risk assessments for AI safety, privacy, compliance, and user impact
- Maintain test documentation, defect tracking, and reporting to stakeholders
- Partner with cross-functional teams to improve product quality and release readiness
What You Need to Bring
- 7-10 years of software engineering experience.
- 3+ years testing production-scale Generative AI systems.
- Strong understanding of QA methodologies, testing lifecycle, and defect management
- Experience with test case design, exploratory testing, and regression testing
- Hands-on experience with test automation tools such as Selenium, Playwright, Cypress, Postman, or similar
- Familiarity with API testing, SQL/NoSQL basics, and backend validation
- Strong analytical and problem-solving skills
- Ability to think critically about edge cases and user behavior
- Excellent communication and documentation skills
- Experience with bug tracking tools such as Jira, Azure DevOps, or similar
- Experience testing AI, ML, or GenAI applications
- Knowledge of LLM evaluation techniques, prompt testing, RAG systems, or model benchmarking
- Familiarity with Python, JavaScript, or automation scripting
- Experience with AI safety testing, red-teaming, fairness, or policy evaluation
- Understanding of metrics such as precision, recall, latency, user satisfaction, and error rates
- Exposure to analytics dashboards, and user behavior analysis
- Knowledge of privacy, security, and compliance concerns in AI products
- Knowledge of MCP ecosystem expertise.
- Knowledge of Multi-agent orchestration platforms such as LangGraph, CrewAI, AutoGen, and OpenAI Agents SDK.