Description
We are seeking a Senior Data Platform Engineer to join our team in Hyderabad, Telangana. This is a full-time position.
The successful candidate will partner with the core Databricks team to support the transition of existing data pipelines to the target enterprise data platform. You will provide hands-on technical guidance for enhancing, stabilizing, and optimizing pipelines after transition, with a focus on performance, scalability, maintainability, operational reliability, and secure AWS deployment patterns.
Key responsibilities include:
- Implementing and promoting repeatable engineering patterns for ingestion, transformation, testing, deployment, monitoring, and support across onboarded datasets
- Ensuring pipelines are designed and operated in a way that supports long-term platform consistency, reliability, observability, and ease of maintenance
- Supporting the design and operation of cloud-native data pipelines using AWS services such as Amazon S3, AWS Glue, AWS Lambda, Amazon Kinesis, AWS Lake Formation, and AWS IAM
- Promoting the use of encryption, environment-level controls, access policies, and platform guardrails to support secure access to platform resources and data products
- Applying lakehouse technologies and concepts such as Delta Lake, Apache Iceberg, Databricks Unity Catalog, metadata-driven pipelines, and governed data access patterns
The ideal candidate will have:
- 6-8+ years of experience in data engineering, data platforms, cloud data architecture, software engineering, or related engineering domains
- Demonstrated experience operating as a senior individual contributor, technical lead, senior data engineer, lead data platform engineer, or equivalent technical role
- Strong hands-on experience with modern data engineering and pipeline development, including batch and/or streaming data workflows
- Experience working with Databricks-based data platforms and supporting migration or transition of pipelines into a lakehouse-oriented architecture
- Strong familiarity with AWS cloud-native data engineering, including services such as Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM
In terms of technical expertise, you should have:
- Working knowledge of technologies and concepts such as Delta Lake, Apache Iceberg, Databricks Unity Catalog, metadata-driven pipelines, data lake governance, lakehouse architecture, and catalog-driven processing
- Experience supporting data integration patterns involving mastering, MDM, reference data, market data, investment data, risk data, or trusted data distribution workflows
- Solid understanding of data quality, schema management, lineage, metadata, access control, encryption, and operational support for production data pipelines
Benefits and perks are not specified in the job posting.
For more fresh new jobs, check out yubhub.co.