Description
We are seeking a dedicated Data Engineer to join our team. As a Data Engineer, you will be at the forefront of our ambitious data initiatives, driving solutions that are both scalable and high-performance. You will lead and participate in impactful data initiatives, analysing complex data challenges and delivering scalable, high-performance solutions. You will build, develop, and maintain robust ETL pipelines and large-scale batch processing workflows using technologies such as Apache Spark and Kafka. You will collaborate with contemporary data platforms such as Microsoft Fabric, Azure Databricks, Snowflake, and Delta Tables in Lakehouse architectures. You will ensure data quality, consistency, and reliability through profiling, validation, and optimisation across diverse data sources. You will translate complex business requirements into innovative, efficient technical solutions, combining deep technical expertise with strong business understanding. You will collaborate closely with data engineers, data scientists, and business collaborators to drive data-driven decision-making. You will integrate AI/ML concepts where relevant to improve analytical capabilities. You will proactively identify improvement opportunities, think creatively to optimise processes, and continuously improve performance and efficiency.
The ideal candidate will have over 4 years of experience working as a Data Engineer. They will have proven experience with ETL (Extract, Transform, Load) processes and tools. They will have strong proficiency in Python for data manipulation and analysis, and a good understanding of SQL and NoSQL databases. They will have hands-on experience with Apache Spark or similar big data processing frameworks. They will have experience working with Microsoft Fabric, Azure Databricks, or comparable large-scale data storage platforms like Snowflake. They will have experience working with Delta Tables. They will have a good understanding of AI/ML concepts and applications. They will have strong analytical and problem-solving skills with a proactive attitude dedicated to making improvements. They will have excellent communication skills and the ability to work effectively with cross-functional teams.