Description
Mastercard is seeking a Lead Data Engineer to join its team in Pune, India/Toronto, Canada. This is a senior individual-contributor and technical leadership role that drives design standards and guides a team's engineering practices.
The successful candidate will set and enforce design standards, patterns, and best practices for pipelines built on Databricks, Spark, Python, PySpark, and dbt. They will provide technical leadership and direction to a team of engineers, owning architecture and design reviews.
Key responsibilities include:
- Setting and enforcing design standards, patterns, and best practices for pipelines
- Providing technical leadership and direction to a team of engineers
- Leading the design of CI/CD, DataOps, and data observability strategy for the team
- Owning Databricks platform administration standards and deployment governance
- Driving resolution of complex technical challenges across pipelines and the platform
- Mentoring and growing engineers, establishing a strong code review and quality culture
- Partnering with product and stakeholders on roadmap, prioritization, and technical trade-offs
- Owning the deployment, migration, and upgrade strategy for the team's data platforms
The ideal candidate will have:
- A Bachelor's or Master's degree in computer science or a related technical field
- 8+ years of data engineering experience, including demonstrated technical leadership
- Expert hands-on experience with Databricks, Spark, Python, and PySpark
- Deep dbt/TDD and CI/CD expertise
- Proven ability to define engineering standards and lead a team's practices
- Strong architecture and platform ownership across observability, cost, and performance
- Excellent communication and stakeholder management skills
In addition to the above qualifications, experience in banking, e-commerce, credit cards, or payment processing and exposure to both SaaS and premises-based architectures would be an added advantage.