Description
We are seeking a Senior Data Engineer to join our team at Barclays, where you will play a crucial role in building and maintaining the systems that collect, store, process, and analyse data. You will design and implement data pipelines, data warehouses, and data lakes to ensure that all data is accurate, accessible, and secure.
Responsibilities
- Build and maintain data architectures and pipelines that enable the transfer and processing of durable, complete, and consistent data.
- Design and implement data warehouses and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
- Develop processing and analysis algorithms fit for the intended data complexity and volumes.
- Collaborate with data scientists to build and deploy machine learning models.
Analyst Expectations
- Perform prescribed activities in a timely manner and to a high standard, consistently driving continuous improvement.
- Demonstrate in-depth technical knowledge and experience in your assigned area of expertise.
- Lead and supervise a team, guiding and supporting professional development, allocating work requirements, and coordinating team resources.
- Partner with other functions and business areas.
- Take responsibility for end results of a team's operational processing and activities.
- Escalate breaches of policies/procedures appropriately.
Requirements
- Experience with AWS, including S3, EC2, Lambda, RDS, Glue, Athena, data pipelines, and step functions.
- Knowledge of distributed computing architecture and Core Hadoop components (HDFS, Spark, Yarn, HIVE, Impala).
- Expertise in SQL and advanced SQL skills.
- Expertise in shell scripting and programming (Scala/Python).
- Knowledge of ETL technologies, preferably Ab Initio.
- Experience with technical design for ETL/Hadoop/AWS components.
- Knowledge of Data Warehousing.
What We Offer
- Opportunity to work on complex technical problems that require detailed analytical skills and analysis.
- Collaborative environment with fellow engineers, business analysts, and business stakeholders.
- Chance to harness cutting-edge technology to revolutionize our digital offerings and drive innovation and excellence.