Description
Faire is a technology wholesale platform built on the belief that the future is local. Independent retailers around the globe collectively represent a multi-hundred-billion-dollar wholesale market that has historically been fragmented and offline. At Faire, we're using the power of tech, data, and machine learning to connect this thriving community of entrepreneurs across the globe.
Our Engineering organization owns the software that makes our marketplace work. The Data Platform group supports everyone at Faire who depends on data: Product Engineering, Data Science, Machine Learning, Analytics, Strategy, Finance, and Product. We're looking for a Staff Engineer to own the plumbing of our data infrastructure.
Responsibilities:
- Set the technical direction for how data moves from production systems into our analytical stores, and own the roadmap to get there over the next couple of years.
- Build the CDC and streaming ingestion layer: CockroachDB changefeeds and MySQL binlogs into Kafka, then into Iceberg tables on S3.
- Implement data contracts and quality checks throughout our platform.
- Put real ownership and SLAs on the datasets the business runs on, and wire quality checks into the platform with tools like Anomalo and Monte Carlo.
- Run Airflow and Fivetran well, and have an opinion about what we should keep buying versus what we should build.
- Own reliability for the platform: SLOs, on-call, incident reviews, and the follow-through so the same thing doesn't break twice.
- Work with the senior engineers, data scientists, and analysts who depend on this platform, and lead the migration of existing pipelines onto the new one without breaking what they rely on.
- Mentor the engineers around you.
Qualifications:
- You've built and run data infrastructure that other teams depended on, at meaningful scale, and you've been the person setting direction for it.
- Deep experience with change data capture and streaming ingestion from operational databases through Kafka.
- Hands-on experience with lakehouse architectures on an open table format.
- Strong Spark skills, and experience running Databricks and Snowflake against shared storage.
- Experience with data quality and observability in practice.
- Experience operating Airflow at scale and working with managed ingestion like Fivetran.
- Strong SQL, and good instincts for how to model data.
- Solid Python plus at least one of Kotlin, Java, Scala, or Go.
- A working understanding of data governance.
- A track record of leading cross-team data initiatives and migrations, and of mentoring senior engineers.