Associate Director, Software Engineering Specialist

We are currently seeking an experienced professional to join our team in the role of Associate Director, Software Engineering Specialist.

Principal responsibilities:

• Lead end-to-end engineering delivery for big data ETL and scoring pipelines, from design through build, testing, release, and operational support.
• Design and evolve cloud-native architectures using GCP services (e.g., Dataproc/Spark, GCS, CloudSQL, GKE, VMs) aligned to non-functional requirements.
• Develop and optimise Spark/Scala jobs for large-scale batch processing, data quality controls, enrichment, and feature/scoring outputs.
• Drive effective integration patterns and engineering standards around the Quantexa platform, ensuring robust data ingestion, entity resolution/analytics workflows, and downstream consumption.
• Design and implement efficient indexing/retrieval patterns using Elastic Search to support investigative and analytical use cases.
• Establish best practices for code quality, CI/CD, testing strategies, observability, reliability, and cost/performance optimisation; mentor engineers and conduct technical reviews.

Requirements:

• Strong hands-on big data engineering background, with proven experience building production-grade ETL pipelines and data platforms at scale.
• Advanced Spark + Scala experience (performance tuning, partitioning strategies, handling skew, optimising job execution, production support).
• Strong GCP experience designing and running data workloads (e.g., Dataproc, GCS, CloudSQL, GKE/containers, IAM/security basics, networking considerations).
• Technical leadership experience (solution design ownership, estimation/planning, guiding teams through delivery, mentoring, setting standards and direction).
• Solid engineering fundamentals: distributed systems, data modelling, batch processing patterns, fault tolerance, testing and CI/CD discipline.
• Stakeholder and communication strength: able to translate complex technical topics to non-technical partners, navigate a complex organisation, and drive outcomes across teams.