Description
Mastercard's Business & Market Insights (B&MI) group is seeking a Senior AI Engineer, Generative AI & ML Engineering, for the Operational Intelligence Program within B&MI.
In this hands-on role, you'll design, build, and ship production-grade agentic AI systems. These include agents that make extensive tool calls into enterprise services and APIs, handle complex multi-step workflows, and deliver secure, scalable, reliable results.
Responsibilities:
- Architect and build the tool-calling layer that powers our agents: wrap internal services, data platforms, and external APIs as well-defined tools with clear schemas, contracts, and permissions.
- Engineer reliable tool execution at scale, including parallel and chained tool calls, timeouts, retries, idempotency, error recovery, and graceful handling of tool failures or bad model outputs.
- Develop high-performance Python backend services for LLM orchestration, covering async job handling, streaming responses, caching, rate limiting, and distributed workflows supporting high-throughput Gen AI operations.
- Engineer state, memory, and context management subsystems so agents can maintain session continuity across long, multi-step, tool-heavy interactions.
- Architect multi-agent and skills-based agent systems using frameworks such as LangGraph, CrewAI, AutoGen, or similar, enabling autonomous reasoning, tool use, inter-agent coordination, and adaptive decision-making at enterprise scale.
- Build observability for agent behavior, including end-to-end tracing of tool calls, measuring tool-selection accuracy, and debugging multi-step agent runs in production.
- Build retrieval systems (RAG and Graph-RAG) where agents need grounded knowledge, using vector databases (e.g., Pinecone, pgvector, OpenSearch) and knowledge graphs (e.g., Neo4j, AWS Neptune).
- Establish LLMOps and MLOps pipelines on Databricks (AWS), covering MLflow, prompt and agent evaluation, model lineage tracking, CI/CD, and monitoring.
- Design multimodal generative AI pipelines that unify text, image, tabular, and graph data using modern transformer-based and foundation models.
- Guide fine-tuning, prompt engineering, and alignment strategies (e.g., PEFT, LoRA, instruction tuning, RLHF) to adapt foundation models for specialized enterprise use cases, working closely with ML specialists.
- Implement Responsible AI practices, including hallucination mitigation, bias detection, output safety guardrails, explainability, and compliance with data ethics standards.
- Evaluate and productionize advances in agentic AI, multimodal models, and frontier foundation models, benchmarking them against enterprise performance and safety requirements.
- Lead, mentor, and grow a team of engineers, and set the engineering standard for Gen AI development across the program.
Requirements:
- Bachelor's or Master's degree in Computer Science, AI/ML, or Engineering, or equivalent practical experience.
- A strong track record of building and operating production backend systems at scale, including distributed services, APIs, async processing, data stores, and observability.
- Strong API design and integration experience: you know how to build services that other systems, and now LLMs, can call safely and reliably.
- Excellent Python engineering skills, including async programming, API development (FastAPI or similar), and microservices, plus SQL proficiency.
- Hands-on experience shipping at least one LLM-powered application to production that uses tool/function calling, such as agents that call APIs, query databases, or trigger workflows.
- A working understanding of Gen AI engineering patterns, such as prompt design, embeddings, semantic search, and agent memory, or a demonstrated ability to ramp up quickly on new technical domains.
- Experience with at least one cloud platform (AWS preferred; Azure or GCP welcome).
- Experience leading or mentoring engineers, plus strong communication and stakeholder management skills.