Description
We are seeking a Lead Engineer – Data Platform to join our team at DWS Group in Madrid. As a Lead Engineer, you will drive the design, build, and continuous improvement of our hybrid data platform. You will lead a small but high-impact team, setting technical direction, mentoring team members, and contributing directly to architecture and code.
Responsibilities:
- Own the end-to-end architecture, design, and evolution of enterprise data platform capabilities across hybrid cloud and on-premises environments
- Define the technical roadmap, engineering standards, and reusable patterns for workflow orchestration, scheduling, dependency management, and data processing
- Lead architectural decisions, proofs of concept, and adoption plans based on scalability, interoperability, security, operability, and total cost of ownership
- Establish reliability, observability, and operational standards for data platform components, including monitoring, alerting, failure recovery, capacity management, and service-level objectives
- Drive platform automation through infrastructure as code, CI/CD, automated testing, and self-service engineering capabilities
- Partner with architecture, security, governance, and application teams to ensure platform solutions meet enterprise standards and regulatory requirements
- Remain hands-on in architecture and code, lead design and code reviews, mentor engineers, and foster a culture of engineering excellence and continuous improvement
Requirements:
- 12+ years of experience in data or platform engineering, including designing and operating enterprise-scale data platforms
- Deep hands-on Snowflake expertise across architecture, ingestion, transformation, workload management, performance, cost optimisation, secure data sharing, and governance
- Strong production experience with Apache Airflow, including reusable DAG patterns, dependency management, scheduling, testing, monitoring, and failure recovery
- Strong expertise in Apache Kafka and event-driven architectures, including topic design, producers and consumers, schema management, Kafka Connect, delivery semantics, and replay strategies
- Proven ability to integrate batch and streaming pipelines across Snowflake, Airflow, and Kafka
- Advanced Python and SQL skills; Java experience for Kafka-based services and connectors is beneficial
- Strong experience with Terraform, Git, CI/CD, and automated testing; OpenShift or Kubernetes experience is desirable
- Strong understanding of reliability, observability, data quality, security, access management, and compliance in regulated environments
- Proven technical leadership, including setting engineering direction, reviewing designs and code, mentoring engineers, and communicating with technical and business stakeholders
- Strong problem-solving and strategic-thinking skills, with fluent written and spoken English
- Proven ability to leverage AI tools to enhance productivity, optimise workflows to solve business problems, while applying critical judgment to ensure responsible and ethical use of data and AI outputs
Benefits:
- Access to a range of benefits, including healthcare, company perks, and retirement plans
- Hybrid working model, allowing for flexibility between in-person collaboration and remote work
- Commitment to creating a diverse and inclusive workplace