The workflow before#
Axim sells data intelligence built on two vendor feeds: retirement plan data from Judy Diamond and market data from Bloomberg. The pipelines that joined them had grown by patching. Specific employer IDs (EINs) were written straight into the code, so adding or dropping a plan sponsor meant an engineer editing and redeploying. The jobs called the Bloomberg API far more often than they needed to. Docker builds pulled images from public registries and Python packages at whatever version was current, so a routine overnight rebuild could fail with no code change at all. And the two vendors share no common key, so matching a sponsor to its market record came down to judgment instead of dependable logic.
What we built#
We ran a focused review and rework sprint on Axim's existing pipelines. The goal wasn't a rebuild. It was to take the surprises out: get fixed values out of the code, make every build reproducible, stop wasted API calls, and write a clear plan for the hardest open problem, matching records across two vendors that don't share an ID.
Plan sponsors managed in configuration
We pulled the hard-coded EINs and other fixed values out of the pipeline code and moved them into configuration. Adding or removing a plan sponsor is now a config change someone can review, not a code release. We kept that configuration in version control next to the code that reads it, so every change has an author and a history.
Builds that behave the same every day
We moved Axim's Docker images into its own container registry and pinned every Python dependency to a fixed version. The same code now builds tonight exactly as it did last week, so the morning team starts with fresh data instead of a failed job. Upgrades still happen, but on purpose, when someone chooses to test and bump a version.
Fewer, leaner API calls
The pipelines were asking Bloomberg repeatedly for data they already had. We consolidated those calls so each run requests what it needs once. Runs get shorter, the API sees less load, and there's one less place for a nightly job to stall.
A plan for matching vendor records
Judy Diamond and Bloomberg describe the same companies with no shared key. We documented the data quality gaps and specified a matching approach, along with the data assertions that catch a bad match before it reaches a customer. We designed the checks first because in a data business a wrong match shipped to a client costs more than a missing one. This piece is a plan ready to build, not yet in production.
How we delivered it
A small Blue Orange team worked inside Axim's existing code base as a single sprint: review first, then rework in order of risk, starting with the build failures that hit every morning. We changed the pipelines in place instead of replacing them, so Axim's engineers kept working in familiar code and could review each change as it landed. The sprint closed with written documentation of the data gaps and the matching plan, handed to Axim's team as the next piece of work.
Before and after#
| Measure | Before | After |
|---|---|---|
| Daily build failures from dependency drift | Recurring | Down about 80% |
| Hard-coded EINs in pipeline code | In the pipelines | 0, moved to configuration |
| Docker image source | Public registries | Axim's own registry |
| Cross-vendor record matching | Ad hoc, no shared key | Documented approach with validation assertions |
What's next#
Next is building the cross-vendor matching from the plan, with the assertions running on every load. Once plan and market records line up reliably, Axim can put a question-answering tool on top of the combined data, so its analysts can ask about a sponsor's plans and market position in plain language.
