Reliability and completeness of our multi-provider ingestion pipeline: build failover logic, track daily completion rates per AI provider, and maintain the systems that ensure customers always get accurate, timely data
Data quality and consistency end to end: own the contracts between ingestion, processing, and downstream serving so that what customers see in the product is correct and trustworthy
Extraction pipeline performance: design and maintain async processing patterns that keep our ingestion throughput ahead of demand as we scale
Data readiness for new features: partner with engineering upstream so instrumentation, schema changes, and pipeline capacity are in place before features ship
Pipeline observability: build and maintain health monitoring across the full data stack so the team knows the state of the system in real time
What you'll bring
Built and maintained production data pipelines at meaningful scale, including experience with ingestion, ETL, and distributed job queue systems
Hands-on experience with ClickHouse or a comparable columnar store, including schema design, materialized views, and query optimization
Experience working with third-party data providers or scraping infrastructure at scale
Designed and owned reliability and observability systems: you have built the dashboards and alerting that tell you what is wrong before users do
Comfortable working close to the application layer: our stack is Rails and Postgres and this person will need to understand how backend systems feed the data platform
Nice to haves
Familiarity with AI provider ecosystems (ChatGPT, Gemini, Perplexity) and how answer formats affect extraction and processing
Experience migrating synchronous processing to batch or async patterns
Prior work at a B2B SaaS company where data accuracy was a customer-facing product requirement
Our Guiding Principles
Extreme Ownership
Quality
Curiosity and Play
Make Our Customers Heroes
Respectful Candor
Benefits
Equity in a fast-growing startup
Competitive benefits package tailored to your location
Flexible time off policy
Parental Leave
A fun-loving and (just a bit) nerdy team that loves to move fast!