Data Engineer

We are seeking a talented and motivated Data Engineer with 3–5 years of experience to support the development and operation of our modern cloud-based data platform.

Working within the Global Data Office, the Data Engineer will design, build and maintain scalable data pipelines and data products using Microsoft Azure, Databricks and Delta Lake, while supporting enterprise data governance and quality initiatives.

The role will work closely with the Data Engineering Lead, Data Architects, Analysts, Data Governance teams and business stakeholders to deliver trusted, reliable and accessible data solutions that underpin reporting, analytics, AI and machine learning initiatives across the firm.

Key Responsibilities and Deliverables

Data Pipeline Development

• Design, develop and maintain batch and near real-time ETL/ELT pipelines using Azure Data Factory, Databricks, PySpark and SQL.
• Support the ingestion, transformation and delivery of data from multiple enterprise source systems.
• Develop reusable and scalable data processing components aligned with engineering standards.

Data Platform Engineering

• Build and maintain data lake and lakehouse solutions using Azure Data Lake, Databricks and Delta Lake.
• Support implementation of data models that enable analytics, reporting and self-service data consumption.
• Assist with the development of reusable data products and datasets.

Data Quality and Governance

• Implement data quality controls, validation frameworks and monitoring processes within data pipelines.
• Support metadata management, data cataloguing and lineage activities using tools such as Collibra and Purview.
• Contribute to improving data quality, consistency and trust across enterprise datasets.

Performance and Reliability

• Monitor data pipelines and processing workloads to ensure reliability and operational stability.
• Investigate and resolve data and platform issues in a timely manner.
• Identify opportunities for improving performance, scalability and cloud cost efficiency.

Collaboration and Delivery

• Work closely with Data Architects, Lead Data Engineers, Data Analysts and business stakeholders to deliver data solutions aligned to business requirements.
• Participate in agile delivery ceremonies including sprint planning, stand-ups and retrospectives.
• Support testing, deployment and release activities for data products and platforms.

CI/CD and Engineering Standards

• Contribute to source control, automated deployment and testing processes using Git, Azure DevOps and related tooling.
• Follow established engineering standards, coding practices and documentation requirements.
• Participate in peer reviews and contribute to continuous improvement initiatives.

Documentation and Knowledge Sharing

• Maintain technical documentation, solution designs and operational procedures.
• Support knowledge sharing across the Data Office through collaboration and learning initiatives.
• Contribute to the development of best practices and engineering standards.

Key Requirements

Experience & Background

• 3–5 years of experience in data engineering or a related technical discipline.
• Experience developing ETL/ELT data pipelines in a cloud environment, preferably Microsoft Azure.
• Hands-on experience with Databricks, SQL and Python/PySpark.
• Experience working with enterprise data platforms, data warehouses, data lakes or lakehouse architectures.
• Familiarity with data governance, metadata management and data quality concepts.
• Experience using Git and contributing to CI/CD delivery processes.
• Strong stakeholder engagement and collaboration skills.
• Bachelor’s degree in computer science, Engineering, Information Systems or a related field, or equivalent practical experience.