Description
Mastercard's AI Innovation at Scale team is seeking a highly motivated and detail-oriented Senior Data Engineer to join their team. This role is critical in ensuring the delivery of high-quality products by assessing requirements, identifying issues, and implementing effective testing strategies.
As a Senior AI Engineer, you will:
- Be an integral part of a creative and innovative team, contributing to collaborative projects and sharing insights to drive engineering and data science excellence
- Work with cutting-edge big data platforms (e.g., Databricks, Apache Spark) at petabyte scale, pushing the boundaries of data processing and model enablement
- Partner closely with Data Science teams to enable seamless R&D, scale models, features, and experimentations into reliable systems
- Support the deployment of model features and model artifacts, ensuring seamless integration into production environments and downstream decisioning systems
- Write clean, testable, and maintainable code, ensuring solutions are robust, efficient, and production-grade
- Design, build, and maintain data pipelines that integrate multiple data sources to support a unified merchant registry and trust profile, enabling richer datasets and unlocking new opportunities for innovation
- Collaborate with data operations and governance teams to move and manage data in compliance with security standards, policies, and regulatory requirements
- Contribute to the design and evolution of scalable, entity-centric data models that support merchant identity resolution and longitudinal profiling
- Automate and maintain data workflows in distributed environments, improving reliability and operational efficiency
- Analyze and optimize ETL/ELT processes to support high-performance data access and model execution
- Implement testing frameworks and monitoring capabilities to ensure production solutions are reliable, observable, and continuously improving
- Support incident response, debugging, and performance tuning of production AI/ML systems
Essential Skills to be successful:
- Proven track record of self-directed learning, demonstrating the ability to acquire new skills and knowledge independently
- Strong independent research skills and resourcefulness, enabling you to find solutions and innovate in data engineering
- Strong understanding of data pipelines and end-to-end ML model development workflows, with exposure to entity-centric data systems
- Experience with Python and SQL, showcasing the ability to write clean, readable, and maintainable code
- Experience with big data technologies (e.g., Spark, distributed compute frameworks)
- Hands-on experience with cloud platforms such as Databricks, AWS, or GCP
- Critical thinking and a drive to produce high-quality work, ensuring all solutions meet rigorous standards
- Strong communication skills, enabling effective collaboration with team members and stakeholders
- Experience collaborating across data science, engineering, and governance teams
- Ability and interest in problem-solving, with a proactive approach to tackling challenges
- Openness to learn and apply new technologies, staying current with industry trends and advancements
- Familiarity with Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration
- Bachelor’s degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical experience
Nice to Have:
- Experience working in hybrid environments (cloud and on-premises)
- Familiarity with ML lifecycle and CI/CD practices for data and ML workflows
- Experience with data governance, lineage, and metadata management
- Exposure to batch, streaming, or real-time data pipelines and production ML monitoring/observability
- Experience supporting high-scale production systems in merchant, fraud, or payment domains
- Understanding of security, compliance, and handling sensitive data
- Experience designing scalable databases and data models such as business registries
- Experience with database updates and maintenance