Description
The Government Technology Agency (GovTech) is the lead agency driving Singapore's Smart Nation initiatives and public sector digital transformation. As the Centre of Excellence for Infocomm Technology and Smart Systems (ICT & SS), GovTech develops the Singapore Government's capabilities in Data Science & Artificial Intelligence, Application Development, Smart City Technology, Digital Infrastructure, and Cybersecurity.
As a Data Engineer, you will be responsible for designing, developing, and maintaining robust data infrastructure that enables analytics and insights across the organisation. You will work closely with business stakeholders, data analysts, and other technical teams to deliver scalable data solutions that support data-driven decision making.
Your job scope includes:
- Requirements Analysis and Solution Design: Collaborate with business users, data analysts, and stakeholders to elicit and document requirements for analytics use cases. Translate business requirements into technical specifications and design optimal data architecture solutions that align with organisational data strategy and governance frameworks.
- Data Pipeline Architecture and Design: Design end-to-end data pipelines from various source systems to different layers within the data analytics platform, including bronze (raw), silver (technical), and gold (business) layers. Ensure pipeline designs follow best practices for scalability, reliability, and maintainability whilst adhering to data governance and security requirements.
- Data Pipeline Development and Implementation: Develop, test, and deploy data pipelines using modern data engineering tools and frameworks. Implement data transformation logic, quality checks, and error handling mechanisms to ensure reliable data processing. Build automated workflows that can handle both batch and real-time data processing requirements.
- Data Infrastructure Maintenance and Optimisation: Monitor and maintain existing data pipelines to ensure optimal performance, reliability, and cost-effectiveness. Troubleshoot data quality issues, resolve pipeline failures, and implement performance improvements. Conduct regular reviews of data infrastructure to identify opportunities for enhancement and modernisation.
- Data Modelling and Architecture: Design and implement appropriate data models including star schema, snowflake schema, and wide table structures based on analytical requirements. Implement Slowly Changing Dimension (SCD) strategies to handle historical data changes effectively whilst maintaining data integrity and analytical accuracy.
- Collaboration and Knowledge Sharing: Work collaboratively with cross-functional teams including data analysts, data scientists, and business stakeholders to ensure data solutions meet business needs. Contribute to the development of data engineering best practices and standards within the organisation.
We are looking for someone with:
- Minimum 3-5 years of experience in data engineering, data analytics, or related technical field, with demonstrated experience in designing and implementing data pipelines and analytics solutions
- Strong knowledge of data modelling concepts including star schema, snowflake schema, and wide table designs
- Proficient in implementing Slowly Changing Dimension (SCD) patterns and understanding their appropriate use cases for maintaining historical data accuracy
- Comprehensive understanding of modern data architecture patterns including Data Lake, Data Warehouse, Data Lakehouse, and Data Mesh concepts
- Proficient in Python programming language with experience in data manipulation libraries (pandas, numpy), data pipeline frameworks, and API development
- Strong SQL skills for data querying, transformation, and database management across various database platforms
- Solid foundation in data engineering principles including data ingestion, transformation, quality assurance, and pipeline orchestration
- Knowledge of data analysis techniques and data management practices including data governance, lineage tracking, and metadata management
- Cloud and Platform Experience: Knowledge of Databricks and AWS cloud services will be an advantage, including experience with cloud-based data processing, storage solutions, and managed analytics services
- Strong analytical and problem-solving skills with the ability to troubleshoot complex data issues and optimise system performance
- Excellent communication skills to effectively collaborate with both technical and non-technical stakeholders, and ability to translate business requirements into technical solutions
We offer you:
- A purposeful career to make lives better
- Continuous and robust learning and development opportunities
- Flexible work arrangements (subject to your job role)
- Leave benefits to meet your work-life needs and employee wellness programs
- A 2-year fixed term contract