Senior Data Engineer

The Senior Data Engineer will play a key role in designing, building, and evolving large-scale data platforms and services that power reporting, analytics, and data products across Visa.

This role is focused on building scalable, reliable data pipelines and reusable platform capabilities across modern cloud and big data environments. You will work with technologies such as Spark, PySpark, Kafka, Hadoop, and Hive to support batch and real-time data processing at enterprise scale.

Beyond building individual pipelines, you will help improve how data engineering is done across the organization by automating delivery processes, strengthening observability and data quality, establishing engineering standards, and developing reusable frameworks that enable teams to deliver new data products faster and more reliably.

You will work across engineering, analytics, and product teams to solve complex data challenges and help shape the future of Visa's reporting and analytics platforms.

Responsibilities

• Design, build, and optimize large-scale data pipelines and data services supporting batch and real-time processing.
• Develop scalable solutions using technologies such as Spark, PySpark, Kafka, Hadoop, Hive, and cloud-based data platforms.
• Architect data pipelines, data models, and reusable frameworks that support enterprise reporting, analytics, and data products.
• Build shared platform capabilities, services, and automation that can be adopted across multiple engineering teams.
• Improve data pipeline performance and reliability through performance tuning, workflow optimization, monitoring, and observability.
• Automate deployment and delivery processes to improve engineering efficiency and reduce the time required to bring new data capabilities to production.
• Establish automated quality gates and validation practices to improve data quality, reliability, and operational stability.
• Develop reusable blueprints and engineering patterns for new data services and data products.
• Implement and strengthen data governance practices, including data lineage, metadata management, access controls, monitoring, and validation.
• Troubleshoot complex performance and reliability issues across distributed data systems and identify opportunities for architectural improvement.
• Partner with engineering, analytics, product, and other technical teams to design and deliver end-to-end data solutions.
• Evaluate emerging technologies, including Generative AI, and identify opportunities to improve data engineering and platform capabilities.
• Lead technical initiatives from concept through implementation and adoption, influencing engineering practices across teams.
• Mentor engineers and contribute to technical design reviews, architecture discussions, and engineering best practices.

Requirements

• Hands-on experience designing and building enterprise-scale data platforms, pipelines, or distributed data processing solutions.
• Strong experience with technologies such as Spark/PySpark, Kafka, Hadoop, Hive, or comparable big data technologies.
• Experience supporting both batch and real-time/streaming data processing.
• Experience designing scalable data architectures and solving performance, reliability, and data quality challenges.
• Experience building reusable data platforms, frameworks, shared services, or engineering capabilities used across multiple teams.
• Strong understanding of data quality, governance, lineage, metadata, observability, and operational reliability.
• Experience automating data engineering workflows, deployment pipelines, testing, and quality controls.
• Ability to independently diagnose complex distributed-system and data-platform issues and drive them through resolution.
• Ability to influence technical direction and drive initiatives across teams without requiring direct authority.
• Strong ownership and the ability to move technical initiatives from design through production and broader adoption.

Benefits

• Comprehensive benefits package, including Medical, Dental, Vision, 401(k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.

Salary

• Estimated salary range: $118,700.00 to $183,600.00 USD per year.