Description
We are looking for a Python Developer to join a dynamic team for one of our clients in Data Engineering.
As an intermediate-level Python developer in data engineering, you'll design, build, and maintain data pipelines to extract, transform, and load data, ensuring data quality and scalability while collaborating with cross-functional, cross-regional teams.
Responsibilities:
- Design, develop, and implement software applications, scalable data pipelines using Python and related technologies.
- Optimize data workflows to handle large datasets efficiently.
- Write clean, efficient, and maintainable code that meets industry standards.
- Develop and manage databases and applications.
- Implement data extraction, transformation, and loading (ETL) processes.
- Write and document effective APIs.
- Test software at scale to ensure responsiveness and efficiency.
- Ensure data accuracy, consistency, and completeness.
- Identify and resolve data issues or anomalies.
Collaboration and Communication:
- Collaborate with team members, stakeholders, and other developers to define project requirements and develop technical solutions.
- Participate in code reviews to ensure code quality and best practices.
Problem Solving and Debugging:
- Troubleshoot, debug, and upgrade software.
- Identify and resolve performance issues and bugs.
Documentation:
- Contribute to the documentation of code and projects.
Requirements:
- Proficiency in Python, with a strong understanding of object-oriented programming principles.
- Familiarity with data structures and algorithms.
- Experience with data libraries (e.g., Django, Flask, NumPy, Pandas).
- Familiarity with databases (e.g., PostgreSQL, MySQL) and/or NoSQL databases (e.g., MongoDB).
- Familiarity with data storage technologies (e.g., data lakes, distributed file systems).
- Knowledge of ETL tools (e.g., Apache Airflow, Apache Kafka).
- Experience with orchestration tools & cloud computing platforms (e.g., Ansible, Terraform/Kubernetes, AWS, Azure, GCP).
- Familiarity with modern CI/CD frameworks and tools (e.g., Git, Jenkins, Github).
- Hands-on experience with Linux.
- Demonstrated ability to troubleshoot problems and debug to identify root cause.
- 1-4 years of experience in software development with a focus on Python & data.