PingThings cut grid analytics compute cost 90% and queries to seconds

The workflow before#

PingThings runs PredictiveGrid, a service that gives utilities access to synchrophasor (PMU) data from the electric grid. Each sensor reports 30 to 120 times a second, so the history grows to petabytes fast. On AWS EMR, a single analytic query cost $200 to $500 and took 30 to 40 minutes. That's tolerable once. It breaks down when a utility's engineers want to explore a disturbance, rerun a window or train a model across years of data. Customer contracts were losing money on compute, and PingThings' engineers spent their days on infrastructure workarounds instead of the service customers were paying for.

What we built#

We moved PingThings off general-purpose cloud compute and onto infrastructure built for one job: storing and querying high-rate sensor data. At the center is the Berkeley Tree Database (BTrDB), running on on-premise Hadoop and Spark clusters sized for this workload. Queries that took 30 to 40 minutes now return in seconds, and compute cost dropped 90%.

A store built for time series

BTrDB keeps time series in a tree structure designed for fast time-range queries, so pulling one sensor's readings across a week of disturbances takes seconds. It compresses the data 10x or more and versions corrections, so a cleaned reading never erases the raw one. We chose it after benchmarking it against InfluxDB and TimescaleDB on PingThings' own data.

Streaming in, not batch

Kafka takes in PMU streams as they arrive and Spark processes them on the way in, while PostgreSQL holds sensor and asset metadata. We chose streaming because grid events play out in seconds, and an engineer looking into a disturbance shouldn't wait for the next batch load.

Models that clean data and flag disturbances

TensorFlow models clean the raw sensor data and flag grid disturbances. Cleaning sits at the front of the pipeline because PMU feeds drop out and drift, and every analysis downstream inherits whatever errors get through.

APIs for customer applications

REST and WebSocket APIs give customer applications direct access to the data, and Jupyter supports engineers doing their own analysis. Utilities' tools read from the same store PingThings' team uses, so everyone works from one version of the grid's history.

How we delivered it

The program ran 36 weeks. We benchmarked candidate stores first, then built ingestion and storage, then moved customer workloads off EMR in stages so contracts kept running throughout. Applications run on Kubernetes with Prometheus and Grafana monitoring. By the end every customer workload had migrated, and the new infrastructure was on track to pay for itself in about six months.

Before and after#

What's next#

With fast, cheap access to years of grid history, PingThings can run far more models over it. Next up are equipment failure prediction and disturbance classification, moving from analysis engineers run by hand into alerts that utilities act on as events happen.