Description
The Risk & Quantitative Analysis (RQA) group at BlackRock provides independent oversight of the firm's fiduciary and enterprise risks. The AI Validation Associate will perform independent reviews of AI tools and agents used across BlackRock, including agentic systems, RAG applications, and other business workflows.
Key Responsibilities:
- Conduct independent validation of AI usage in workflows, including testing for accuracy, efficiency of guardrails, orchestration, memory management, and token usage.
- Design and implement comprehensive validation strategies and test plans tailored to each AI use case.
- Understand tool architecture, potential vulnerabilities, and weaknesses.
- Assess conceptual soundness, methodology, and alignment with intended business use.
- Review data quality, preprocessing steps, and engineering decisions for AI use cases.
- Execute functional and performance tests.
- Evaluate performance metrics, robustness, and limitations under various scenarios.
- Document validation findings, provide recommendations.
- Stay current on emerging AI technologies, validation techniques, and regulatory developments.
- Communicate validation conclusions to relevant stakeholders, including escalating identified findings, risks, or gaps to senior leadership and approval committees.
- Collaborate with AI Engineering, Data Science, and Risk teams to enhance AI transparency and trustworthiness.
Experience, Skills, and Qualification:
- 3-7 years of work experience in the IT or financial industry.
- Degree or Advanced degree (MS, PhD) in Computer Science, Data Science, Statistics, Applied Mathematics, or related field.
- Strong understanding of generative AI technologies, including large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, prompt engineering, tool use, and modern AI application architectures.
- Experience with Python and common AI development libraries and frameworks (PyTorch, TensorFlow, or similar).
- Excellent analytical, communication, and documentation skills.
- Ability to work independently and challenge stakeholders constructively.
Benefits:
- Strong retirement plan
- Tuition reimbursement
- Comprehensive healthcare
- Support for working parents
- Flexible Time Off (FTO)
Hybrid Work Model:
- Employees are required to work at least 4 days in the office per week, with the flexibility to work from home 1 day a week.