Description
Nebius is looking for a Senior Data Engineer who will build and own data pipelines, and drive the design and technical leadership within the data engineering team.
This is a hands-on data engineering role, focused on designing, implementing, and maintaining reliable data flows for analytics and machine learning.
Your responsibilities will include:
- Design, build, and own production-grade data pipelines using Python and SQL.
- Develop stateless, idempotent pipelines that are resilient to retries, failures, and infrastructure interruptions.
- Implement data transformations, validation, and data quality checks.
- Optimize pipelines for performance, reliability, and cost efficiency.
- Collaborate closely with Analytics, Data Science, and ML teams to deliver trusted datasets.
Supporting Infrastructure:
- Orchestrate pipelines using a workflow orchestration framework (e.g., Airflow or equivalent).
- Package and run data workloads using Docker and deploy them on Kubernetes.
- Use autoscaling and Spot / Preemptible compute for efficient pipeline execution.
- Build CI/CD automation for data pipelines.
- Use Infrastructure as Code only to provision and manage the infrastructure required to run pipelines.
Requirements:
- 8+ years of experience as a Data Engineer, primarily focused on building data pipelines.
- 6+ years of hands-on experience with Python and SQL.
- 3+ years of experience running workloads on Kubernetes.
- Strong understanding of stateless system design and idempotent data processing.
- Experience building and operating data pipelines in cloud environments.
- Experience with workflow orchestration frameworks.
- Strong Linux fundamentals and production debugging skills.
- Working knowledge of spoken and written English.
Nice to Have:
- Experience contributing to or working extensively with open-source software.
- Experience building data pipelines using Apache Spark or similar distributed processing frameworks.
- Experience building data pipelines that support machine learning workflows.
- Familiarity with cost-optimized data processing (e.g., Spot / Preemptible compute).
- Experience with relational and non-relational data stores.
- Experience working with large-scale or high-reliability data systems.
- Experience collaborating with strong Data Science and ML teams.
Benefits & Perks:
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams