Description
Our Client is seeking a Data Engineer to join their team. As a Data Engineer, you will be responsible for architecting, developing, and maintaining scalable data pipelines within a medallion architecture. This role is key in enabling high-quality, business-ready datasets by leveraging modern data engineering technologies and orchestration practices.
Role Responsibilities:
- Design, build, and manage end-to-end data pipelines across the medallion architecture,specifically the bronze, silver (base vault with DBT and orchestration tools, business vault), and gold layers.
- Ingest and process raw data using Spark and Amazon EMR for scalable, distributed computation.
- Develop and automate data transformations for the base vault using DBT (Data Build Tool) to standardize and model data efficiently.
Requirements:
- At least 5 years of experience as an Elasticsearch Data Engineer - ELK (Elasticsearch, Logstash, and Kibana) stack Expert knowledge)
- Java Spring Boot
- IBM ACE Programming
- BS in Computer Science, Data Engineering (Big Data, AWS certification), Data Modeling or similar
- Full English Fluency
Skills & Competencies:
- Strong understanding of data modeling, governance, and best practices in modern data architectures.
- Excellent analytical, problem-solving, and communication skills.
Software / Tool Skills:
- Elasticsearch - cluster optimization, query development, data modeling, performance tuning & administration (4-6 Years) (Must)
- Deep experience with Spark, Python and ETLs and Amazon EMR (Must)
- Hands-on experience with DBT for data transformation and modeling.
- Apache Airflow, AWS Step Functions, or similar. (Must)
- Expert knowledge of Amazon S3 and Apache Iceberg for data storage and management.
- Experience with Kubernetes for container orchestration.
- Experience with Dremio, Looker, or equivalent business view/semantic layer technologies
- AWS Cloud – Intermediate, AWS Lambda (Must) , Step Functions, IAM, SNS, API Gateway, VPC, Transit Gateway, Intermediate (3-4 Years)
- JSON - Intermediate (4-6 Years)
- Jenkins – Data Pipeline Intermediate (4-6 Years) PLUS
- CloudWatch - Intermediate (4-6 Years) PLUS
Benefits:
- This position comes with competitive compensation and benefits package:
- Competitive salary and performance-based bonuses
- Comprehensive benefits package
- Career development and training opportunities
- Flexible work arrangements (remote and/or office-based)
- Dynamic and inclusive work culture within a globally renowned group
- Private Health Insurance
- Pension Plan
- Paid Time Off
- Training & Development
This listing is enriched and indexed by YubHub. To apply, use the employer's original posting:
https://jobs.workable.com/view/cW5uAaS9BKcNSiLU33ZA3z/remote-fbs---elasticsearch-data-engineer-(medallion-architecture)-in-pune-at-capgemini