Senior Data Engineer

The Senior Data Engineer plays a key role within the Audience Data team, designing, building and maintaining the data pipelines and infrastructure that enable teams across the BBC to make effective use of data.

Working closely with Analytics, Data Science, Marketing, Brand and other business functions, the role is responsible for delivering reliable, scalable and trusted data solutions that support audience insight, engagement and decision-making.

The BBC aims to stimulate and delight audiences across the UK with distinctive, creative content. To support this aim, BBC is entering a new charter period of significant and exciting change, developing new capabilities in order to define a BBC fit for the next generation.

Key Responsibilities:

• Design, build and maintain scalable cloud-based data pipelines and data products supporting audience growth, CRM activation, marketing measurement and performance analytics.
• Build and maintain integrations between BBC Account, mParticle, CRM platforms, marketing technologies, advertising platforms and enterprise reporting systems.
• Develop robust API-based integrations to enable secure and reliable exchange of data across marketing, product, audience and analytics platforms.
• Collaborate with analysts, data scientists, engineers, product managers and business stakeholders to identify opportunities for automation, optimisation and improved decision making.

Essential Criteria:

• Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow, Python, SQL and bash/linux.
• Experience developing cloud-native data platforms and solutions on AWS, including working with cloud data warehouse technologies such as Amazon Redshift.
• Good understanding of software engineering principles, including object-oriented development, testing practices, version control, code quality management, CI/CD deployment processes, containerisation, and LLM-assisted coding.
• Experience working with data processing and database technologies, including relational databases, query optimisation, performance tuning, monitoring, backup and recovery.
• Experience delivering data integrations for Customer Data Platforms (CDPs) and external data sources, including technologies such as mParticle, Twilio Segment.

Desirable Criteria:

• Experience implementing layered data architecture patterns (e.g. Medallion Architecture, Data Warehouse or Lakehouse approaches).
• Contributing to the development of area of specialism or industry domain.
• Broadcast, production, start-up or media experience.
• Familiarity with Retrieval-Augmented Generation (RAG), vector databases, embeddings, and other patterns used to support Generative AI solutions on platforms such as Amazon Bedrock.
• Experience developing data-based apps in Streamlit/Shiny.
• Skills in ‘Big Data’ technology tools like Hadoop/Spark.
• Knowledge of Infrastructure as Code tools like CloudFormation/Terraform.

Benefits:

• Fair pay and flexible benefits including a competitive salary package.
• A flexible 35-hour working week.
• 25 days annual leave with the option to buy an extra 5 days.
• A defined pension scheme.
• Discounted dental, health care and gym.