Description
We are seeking an experienced BigData-Hadoop/Scala Engineer, AVP to join our team in Pune, India.
As an Engineer, you will be responsible for designing and developing entire engineering solutions to accomplish business goals. You will ensure that solutions are well architected, with maintainability and ease of testing built in from the outset, and that they can be integrated successfully into the end-to-end business process flow.
Key Responsibilities:
- Design, develop, and maintain high-performance, fault-tolerant data pipelines using modern big data technologies.
- Implement data processing solutions to extract, transform, and load (ETL) data from various sources into our data platforms.
- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver solutions that meet business needs.
- Optimize data processing jobs for performance and cost efficiency.
- Ensure data quality, integrity, and security across all data assets.
- Participate in code reviews, contribute to technical discussions, and mentor junior team members.
- Stay current with emerging technologies and best practices in the big data ecosystem.
Benefits:
- Best in class leave policy
- Gender neutral parental leaves
- 100% reimbursement under childcare assistance benefit (gender neutral)
- Sponsorship for Industry relevant certifications and education
- Employee Assistance Program for you and your family members
- Comprehensive Hospitalization Insurance for you and your dependents
- Accident and Term life Insurance
- Complementary Health screening for 35 yrs. and above
Requirements:
- Strong proficiency in Java and/or Scala for big data application development.
- Extensive experience with Apache Spark for large-scale data processing.
- Demonstrable knowledge and hands-on experience with Google Cloud Platform (GCP) services, specifically:
- Airflow for workflow orchestration.
- Dataproc for managed Spark/Hadoop clusters.
- BigQuery for data warehousing and analytics.
- Solid understanding of data warehousing concepts, ETL/ELT principles, and data modeling.
- Experience with version control systems (e.g., Git).
- Strong problem-solving skills and the ability to troubleshoot complex data issues.
- Excellent communication and collaboration skills.
- Understanding of Angular/React frameworks.
Nice to Have:
- Proficiency in Python for data manipulation, scripting, or data science-related tasks.
- Experience with relational databases, particularly Oracle.
- Competency in Unix/Linux scripting for automation and system administration tasks.
- Familiarity with other cloud platforms (e.g., AWS, Azure) is a plus.
- Knowledge of CI/CD pipelines and DevOps practices.
- Understanding of data governance and data security best practices.
- Experience working in the financial services industry.
- Experience developing UI using Angular/React.
What We Offer:
- Training and development to help you excel in your career
- Coaching and support from experts in your team
- A culture of continuous learning to aid progression
- A range of flexible benefits that you can tailor to suit your needs