Senior Big Data Engineer

We are looking for an experienced, innovative and highly motivated Senior Big Data Engineer to design, develop, and support data engineering solutions.

The role requires a considerable focus on understanding the data dealt with as an organization, developing a better understanding of the data through profiling, taking ownership of the solution design, working collaboratively with the wider engineering team to implement optimal solutions in a timely manner, and ensuring successful deployment and operation of the platform in production.

The platform helps solve complex business challenges pertaining to healthcare data, such as mapping a wide variety of healthcare drug and diagnosis data to standard coding frames.

Responsibilities

• Oversee and guide junior and mid-level engineers, providing technical direction and mentoring to support their growth.
• Delegate tasks effectively, ensuring clarity of expectations and alignment with delivery goals.
• Take ownership of the delivery of assigned workstreams, ensuring quality, timeliness, and accountability across the team.
• Build pipelines for processing large amounts of medical data.
• Participate in analysis, system design, development, deployment, and maintenance.
• Conduct peer code reviews.
• Contribute to the definition and adoption of technical standards.
• Create comprehensive automated unit and integration tests.

Technical Skills and Knowledge

• 6 years of experience with SQL, Python, Scala.
• 6 years of experience in data engineering.
• Strong understanding of the Databricks platform.
• Experience with Big Data batch processing and Azure Cloud Services.
• Practical knowledge of the Apache Spark ecosystem, including Delta and Parquet data formats.
• Experience with complex ETL design and implementation.
• Experience with data analysis, data profiling, data integration, data quality, data warehousing, and data reporting.
• Experience with CI/CD automation processes and tools such as Jenkins/GitLab.
• Familiarity with AI-powered productivity tools such as Copilot or Cursor.
• Practical experience with FastAPI for building production-grade APIs is a plus.
• Experience working in an Agile environment.
• Experience with database design and modeling.
• Previous experience with healthcare or pharmaceutical data would be an advantage.
• Knowledge of algorithms and design patterns and how to apply them effectively.
• Experience with large data volume processing.

Personal Skills and Behaviors

• Excellent analytical and troubleshooting skills.
• Strong collaboration, written, and verbal skills.
• Familiarity with project management concepts, specifically Agile/Scrum.
• Ability to present complex results in a clear language for both technical and non-technical audiences.
• Experience collaborating with teams from Europe, with an understanding of cross-cultural communication and remote delivery dynamics.
• A great team player mindset.

What We Offer

• Work in a company with offices in over 100 countries around the world, listed on the American stock exchange (NYSE).
• Multi-terabyte databases.
• Quarterly work planning - goals and progress review every 90 days.
• Technical training.
• Possibility to participate in technical conferences within the country.
• Favourable employment conditions and benefits.