Description
We are seeking a Sr. Data Engineer to collaborate with stakeholders to identify and clarify requirements for data-driven features and products. You will independently understand the data ecosystem, security, privacy, and retention requirements necessary to support business and product features.
Key Responsibilities:
- Collaborate proactively with stakeholders to clarify requirements and identify key product stakeholders.
- Translate functional requirements into architecture designs for one or more components, leveraging existing architecture design patterns.
- Implement extensible, maintainable, and reusable code using appropriate coding patterns, guidelines, and best practices.
- Provide end-to-end ownership of data products or data pipelines, including mitigation of technical debt and documentation of errors or unexpected bugs.
- Identify and conduct required testing for moderately complex use cases.
- Build tools and standard automation processes to assist in transforming, managing, accessing, deploying, and monitoring data processes.
- Write queries to extract and compile raw data across end-to-end pipelines.
- Ensure adherence to data management principles, governance, and tools to maintain data quality.
- Develop and update troubleshooting guidance and procedures for reviewing, addressing, and fixing advanced problems.
- Manage software upgrades and server patches for security remediation.
Basic Qualifications:
- 3+ years of relevant work experience and a Bachelor's degree, OR 5+ years of relevant work experience.
- Experience in solving data problems using data technologies.
- Experience with Hadoop, Hive, HBase, Spark, Trino, or comparable distributed data systems.
- Experience in designing and implementing data pipelines and large-scale data processing jobs.
- Experience in writing, modifying, and reviewing high-quality, testable, and efficient code.
- Experience in developing unit, integration, and end-to-end tests with a focus on automation.
- Experience in building and maintaining CI/CD pipelines and deploying applications using containerization and orchestration tools.
- Experience in monitoring applications in production and troubleshooting issues using observability tools.
- Experience in integrating AI or ML services into applications.
- Experience in collecting and analyzing metrics to guide optimizations and improvements.
- Experience in partnering with stakeholders to clarify requirements and ensure deliverables meet business needs.
- Experience in applying secure coding practices and complying with regulatory standards.
Visa requires at least 3 days in office, expectations of these days will be confirmed by your Hiring Manager.