Ingesting 2.1 million vehicle events an hour at 42 percent of the previous cost

The situation

A commercial fleet operator running roughly eleven thousand vehicles across Germany, Austria and Poland had built a telematics platform four years earlier for a fleet of about fifteen hundred. It had grown by a factor of seven and the architecture had not changed.

The symptoms were the usual ones. Dashboard queries that timed out. An ingestion pipeline that dropped events during morning peak when most of the fleet started within the same forty minutes. A storage bill growing faster than the fleet. And a driver mobile app that most drivers had stopped using because it did not work in the underground loading bays where they spent a third of their shift.

The initial request was to scale the ingestion pipeline. That turned out not to be the problem.

What we found

Each vehicle emitted a full telemetry frame every two seconds: position, speed, engine parameters, fuel, door states, temperature in refrigerated units, roughly sixty fields. Every frame was transmitted over cellular and written to Postgres at full resolution and kept indefinitely.

That is 1.7 billion rows a month, growing, with no expiry, in a database being asked to serve both real-time operational queries and twelve-month analytical ones.

The important observation was this: of those sixty fields, most change slowly or not at all between frames. A parked vehicle transmits an identical frame every two seconds for nine hours. We measured it across a week of production data and found that about eighty-four percent of transmitted bytes carried no new information.

So the problem was not ingestion capacity. It was that the system was faithfully storing an enormous quantity of nothing.

What we built

Edge aggregation on the device. We rewrote the telematics unit firmware module in Rust to transmit on change plus a heartbeat, rather than on a fixed interval. A field is sent when it moves outside a per-field threshold, with a full frame every sixty seconds regardless so that gaps are always bounded and a missing device is detectable.

Position is handled separately, because position always changes when moving and never when parked. We used a corridor algorithm: transmit a point only when it deviates from the line predicted by the previous two points by more than a configurable distance. A vehicle on a motorway sends far fewer points than one manoeuvring in a depot, which is exactly the right distribution.

Transmitted volume fell by seventy-nine percent with no loss of information anyone could detect in reconstruction. We validated that by replaying a fortnight of full-resolution historical data through the new encoder and comparing reconstructed tracks against originals. Median position error was 1.4 metres, which is well inside GPS accuracy.

That change alone paid for a substantial part of the engagement in cellular data charges.

A tiered storage model. Rather than one table holding everything forever, four tiers with explicit retention:

The critical decision was what the operational dashboards actually query. We instrumented every query for three weeks. Ninety-four percent touched only the last seven days. Of the remainder, almost all were analytical questions asked at minute granularity or coarser, where per-second resolution was never used.

So per-second data past ninety days is retained only in cold archive, for the two cases that genuinely need it: accident investigation and regulatory requests. Those are rare, latency-tolerant, and now cost almost nothing to keep.

Kafka between ingestion and storage. The old pipeline wrote directly from the HTTP endpoint to Postgres, so a database slowdown became dropped events at the edge. Kafka decouples them. Ingestion accepts and acknowledges, consumers write at their own pace, and a storage incident becomes consumer lag rather than data loss. Retention on the topic is seventy-two hours, which is enough to replay through any outage we have planned for.

An offline-first driver app. The old app assumed connectivity. We rebuilt both platforms natively around a local SQLite store as the source of truth, with a sync queue that reconciles when a connection appears. Drivers complete inspections, log delivery events and capture signatures with no signal at all, and the data reconciles when they surface. Conflict resolution is last-writer-wins on scalar fields and append-only for event logs, which suits the domain because two people rarely edit the same record.

We went native here rather than cross platform. Background location with the reliability and battery profile this needed is one of the four cases where cross platform stops being cheaper, and we made that call in the architecture sprint.

Predictive maintenance on the rollups. Once minute-level rollups existed with two years of history, the maintenance analytics the client had wanted for years became straightforward. Engine temperature trend against ambient, brake wear inferred from deceleration profiles, battery health from cranking voltage. Nothing exotic. The reason it had not been possible before was not the modelling, it was that no queryable history existed.

What was harder than expected

Firmware rollout. Eleven thousand devices across three countries, several hardware revisions, updated over the air on unreliable cellular links. We built the pipeline to accept both the old and new encoding for the entire rollout period and ran dual format ingestion for five months. The last few hundred devices required physical depot visits. Plan for the tail, because it is longer than the schedule suggests.

Threshold tuning. The on-change thresholds are a direct trade between bandwidth and fidelity, and the right value differs by field and by vehicle type. Refrigerated units needed much tighter temperature thresholds than we initially set, and we found that out from a cold chain compliance report rather than from our own testing. We now derive thresholds per vehicle class from historical variance instead of setting them by hand.

Convincing the client to delete data. The hardest conversation in the project was the retention policy. Deleting per-second history past ninety days felt dangerous to everyone, despite the query data showing nobody used it. What resolved it was the Glacier archive: nothing is actually destroyed, it just stops being expensive and fast. Once "we can still get it, it takes four hours" was on the table, the objection dissolved.

Results

Ingestion capacity went from about 700,000 events an hour, where it started dropping, to a tested 2.1 million with headroom. Peak load no longer produces loss.

Infrastructure cost fell fifty-eight percent, from roughly 34,000 euros a month to 14,300, while handling three times the throughput. Cellular data charges fell separately by about sixty-two percent.

Dashboard p99 query latency across twelve months of history is under 200 milliseconds, against a previous state where those queries frequently timed out at thirty seconds.

Driver app usage went from thirty-four percent of drivers active weekly to eighty-eight percent, which we attribute almost entirely to it working in loading bays.

Unplanned downtime hours fell thirty-one percent in the first year of predictive maintenance, measured against the previous year's baseline for the same fleet composition.

What we would tell another team

Before scaling an ingestion pipeline, measure how much of what it ingests carries information. On this project the answer was sixteen percent, and everything that mattered followed from finding that out.

Instrument what your dashboards actually query before you design retention. The gap between the data people say they need and the data they read is usually large, and it is the single biggest lever on storage cost.

Plan the firmware tail. The first ninety-five percent of an over-the-air rollout is a schedule. The last five percent is a logistics project.