Mailgun redesigned its revenue pipeline so no invoice history is ever overwritten

The workflow before#

Mailgun sends email at scale for developers and businesses, and its revenue data is what finance closes the books on. Each billing cycle, data engineers locked invoices and worked revenue exceptions by hand. The revenue pipeline updated records in place, so a bad load could overwrite an invoice's earlier state with no way back. The Matillion jobs that moved the data had no version control, so nobody could see what changed between runs. Loading and querying competed for the same Snowflake compute, and machine learning models were trained and versioned without proper orchestration. None of it had failed badly yet, but finance was relying on care and memory rather than design.

What we built#

We audited Mailgun's Matillion and Snowflake setup end to end and designed a revenue pipeline that keeps every state an invoice has ever been in. Around that design we laid out the changes that would make the rest of the stack safe to work on: version control, separated compute, test copies and a proper home for machine learning. This was an architecture and advisory engagement, with a clear plan for Mailgun's team to implement.

An append-only revenue ledger

In the new design, a change to an invoice is written as a new row with its timestamp, and the current state is worked out from that history. Nothing gets overwritten. We chose this over patching the existing update logic because it removes the data-loss path entirely and gives finance an audit trail for free. It's also what lets invoice locking and exception handling become rules instead of manual steps.

Version control for every job

We set out the path to Matillion Enterprise, which brings Git version control and high availability. Engineers can see who changed a job and when, review changes before they run, and roll back a bad one in minutes instead of rebuilding it from memory.

Separate compute and safe test copies

We recommended separate Snowflake warehouses for loading and querying, so a heavy load never slows the analysts and finance users reading the data. Zero-copy clones give engineers a full copy of production to test against without paying to store it twice, so changes to revenue logic get tested on real data before they go live.

A home for machine learning

For Mailgun's models, we set out a move to a cloud ML environment with automated training runs and versioned models. The data science team can retrain on a schedule and know exactly which model version produced which result.

How we delivered it

We reviewed every Matillion job, the Snowflake configuration and the revenue logic with Mailgun's data engineers, and walked finance through how invoices moved through the pipeline today. The recommendations came back as a roadmap ordered by risk: the revenue ledger first, then version control and compute separation, then machine learning operations. Each step was sized so Mailgun's team could take it on without pausing the billing cycle.

Before and after#

What's next#

Once the revenue history is reliable, Mailgun can put it to work. The natural next steps are anomaly checks that flag revenue exceptions before close, and an assistant that lets finance ask questions of billing data in plain English instead of waiting on a query from engineering.