Description
We are seeking a Lead Cloud Engineer to join our Marketing Services Technology team. As a Lead Cloud Engineer, you will design and deliver scalable, secure, and high-performing data solutions that support advanced analytics, machine learning, and business growth at global scale.
Responsibilities:
- Design, develop, and support large-scale cloud-native data solutions on AWS and Databricks.
- Build and maintain batch, streaming, and real-time data pipelines supporting analytics, reporting, machine learning, and AI workloads.
- Engineer scalable distributed data processing systems using Apache Spark and related Big Data technologies.
- Design and implement modern Lakehouse architectures leveraging Databricks, Delta Lake, and Apache Iceberg.
- Lead modernization initiatives migrating data workloads from Hadoop-based platforms to cloud-native architectures.
- Develop reusable frameworks and engineering patterns that improve scalability, reliability, and operational efficiency.
- Partner with Architects, Product Managers, Data Scientists, and Software Engineers to define and implement cloud data solutions.
- Evaluate technologies and approaches for data ingestion, transformation, storage, governance, and consumption.
- Implement best practices around security, governance, performance optimization, observability, and cost management.
- Drive automation using Infrastructure-as-Code, CI/CD pipelines, and modern software engineering practices.
- Contribute to the evolution of Marketing Services' cloud and data strategy while enabling future AI and machine learning capabilities.
Requirements:
- Strong hands-on experience with Big Data technologies including Hadoop, HDFS, Hive, Spark, and related distributed computing frameworks.
- Deep expertise in cloud data engineering on AWS, including storage, compute, networking, security, and scalable data architectures.
- Experience designing and developing modern data solutions using Databricks, Spark, Delta Lake, and cloud-native technologies.
- Proficiency in Python, Java, or Scala, building large-scale distributed data processing applications.
- Experience building robust batch and streaming pipelines supporting high-volume and high-velocity workloads.
- Understanding of modern Lakehouse architecture principles and open-table formats such as Apache Iceberg and Delta Lake.
- Skilled in data modelling, schema design, partitioning strategies, query optimization, and performance tuning.
- Familiarity with enterprise-scale data governance, security, privacy, and compliance, working with structured and unstructured datasets at scale.
- Experience with Infrastructure-as-Code (e.g., Terraform), modern CI/CD, and cloud-native monitoring and observability practices.
- Strong analytical and problem-solving skills, a passion for automation, and the ability to assess emerging technologies and recommend scalable, cost-effective solutions.
- Excellent communication and collaboration across technical and business teams; comfortable in Agile/Scrum environments managing multiple priorities.
- BS/MS degree in Computer Science, Software Engineering, Information Systems, or a related field.