Scattered sources into one queryable warehouse
A handful of SaaS exports, a production database, and two third-party APIs — all needed in one place, refreshed nightly, and trustworthy enough to build reporting on. The slow part was never the idea. It was the connector-by-connector grind.
- Hand-write an ingestion connector for every source
- Map and reconcile mismatched schemas by hand
- Write the transforms, then tests for the transforms
- Stand up orchestration and backfill logic
- AI scaffolds connectors straight from each API's docs
- Transform models and their tests drafted in bulk, then reviewed
- Schema-mapping boilerplate generated, not typed
- The time freed up goes to the data model and edge cases
The repetitive surface area — connector boilerplate, transform drafts, test coverage. The 80% that's typing, not thinking.
The data model, what "correct" means for each field, how failures are handled, and validating the numbers against reality before anyone trusts them.