Description
The Senior Data Engineer (VP) designs, develops, and supports enterprise-scale data platform capabilities using Airflow, dbt, Python, SQL, cloud-native technologies, and cloud data warehouse platforms to deliver scalable, reliable, and high-performance data solutions.
Responsibilities:
- Actively participate in Agile ceremonies to support successful delivery of platform capabilities.
- Participate in technical design discussions, analyze requirements, and translate complex business needs into scalable data platform solutions.
- Design, develop, and optimize core enterprise data platform capabilities, including data acquisition, ingestion, transformation, workflow orchestration, and reusable processing frameworks.
- Build and optimize large-scale data pipelines supporting data ingestion, transformation, validation, and distribution across enterprise cloud environments.
- Implement metadata-driven processing frameworks and reusable engineering patterns that improve automation, standardization, scalability, and reuse.
- Build and enhance platform capabilities related to metadata management, lineage, dependency tracking, data quality, governance, and workflow control.
- Incorporate Agentic AI capabilities into platform services to enhance automation, intelligent decision-making, developer productivity, and operational efficiency.
- Partner with cross-functional teams to support collaborative development, system integration, and end-to-end testing across platform components and dependent systems.
- Develop and maintain system, integration, regression, and test automation frameworks to ensure platform quality, reliability, and production readiness.
- Support deployment and release activities, ensuring successful rollout of platform enhancements and seamless integration with dependent systems and downstream consumers.
- Contribute to engineering standards, architectural patterns, operational best practices, and framework conventions that improve platform maintainability, consistency, scalability, and supportability.
- Provide L2/L3 production support for data platforms and products, perform root cause analysis, and implement corrective actions to ensure platform stability and operational excellence.
- Drive continuous improvements in platform performance, scalability, resilience, observability, and operational efficiency across data workflows and framework components.
- Lead Proof of Concepts (POCs) and technology evaluations to assess emerging technologies, validate architectural approaches, and identify opportunities for platform modernization and innovation.
- Support product documentation by providing technical expertise, implementation details, and content validation to ensure accuracy, completeness, and alignment with platform capabilities.
- Conduct live product demonstrations and participate in user workshops to showcase platform capabilities, gather feedback, validate solutions, and support platform adoption.
Required Qualifications:
- Bachelor’s degree in Computer Science, Information Systems, or a related technical field.
- 8+ years of experience in Data Engineering, Data Platform Engineering, or a related software engineering role.
- Hands-on experience with Apache Airflow, Python, SQL, cloud-native technologies, and cloud data warehouse platforms.
- Strong proficiency in Python, with hands-on experience developing scalable, production-grade data pipelines and frameworks.
- Hands-on experience designing and building ETL/ELT frameworks, reusable data pipeline components, and workflow-driven data systems.
- Hands-on experience with Snowflake or equivalent cloud-native analytical data platforms.
- Strong hands-on experience writing and optimizing complex SQL.
- Proven experience in query optimization, performance tuning, and workload management for large-scale data environments.
- Extensive hands-on experience with dbt.
- Proven track record of improving the performance, reliability, scalability, and maintainability of enterprise data platforms and pipelines.
Benefits:
- Strong retirement plan
- Tuition reimbursement
- Comprehensive healthcare
- Support for working parents
- Flexible Time Off (FTO)