S1 · Episode 02

Late vs Wrong: When to Stream and When to Batch

Runtime 14:35 AI-narrated

About this episode

A concert lets out and last night's prices are already useless. Friday night, the same company pays drivers, and a messy number is a lawsuit. This episode teaches the fork that decides most architectures: what happens if the number is twenty minutes late, versus wrong by a little.

Eve and Surya use Uber's public H3 work and a DoorDash-shaped Tuesday without inventing anyone's production diagram. You will learn why a grown-up platform runs a conveyor and a truck, how an ordered log is like a ship's log, what partitioning and replay actually do, and why a merge on a unique event ID turns duplicate delivery into a boring failure. Payroll does not belong on the live belt because it feels modern.

Two clocks on the same Tuesday

Eve wants the fork: when does speed beat exactness, and when is it the other way round? Surya answers with two scenes from one ride company. A concert ends, a thousand people open the app together, and a price built on yesterday's data is already useless. That is a late problem. Later the same week the firm pays its drivers, tips and tolls and tax rules included; get that wrong and you have a lawsuit. That is a wrong problem.

Eve names the fashion she keeps hearing: streaming is for grown-ups, batch is a leftover from 2012. Surya calls it the costliest fashion in the trade. The real dividing line is which failure hurts more.

Hexagons, and where the public record ends

Eve checks that this is not Uber's kitchen imagined from a tweet. Surya cites Uber's own engineering blog: an open-sourced hexagon grid called H3, used to measure supply and demand per hex when setting dynamic prices. Hexes beat blocks because a square's diagonal neighbor is farther away than its side neighbor; a hex has six even ones. A concert is one tile lighting up, and the live job is counting cars and requests in that tile inside a moving window. Surya is explicit about what he will not claim: no production diagram for tonight, no millisecond budget that was never published.

Twenty minutes late and the crowd is already underground. Some noise in the count is fine; a stale map is fatal. The payout is the opposite clock, a whole day joined together and then checked. Hours late is a shrug. A few dollars short is a war.

Same fork, a DoorDash-shaped kitchen

Eve asks for a real Tuesday and gets 12:17 on a wet sidewalk. The restaurant list needs three live facts: where you are right now, whether this kitchen is genuinely open, and whether anyone can cook and deliver within a wait you will accept. A stale list shows the Thai place open when its lights are off; the customer experiences a broken app. Friday morning, the same company pays the restaurant and the courier for Thursday; short by a little and the owner rings before coffee. Eve's phrase: not two features, two physics. Surya adds a limit on his own storytelling: he will not describe anyone's specific nightly job as if he had sat in their office. Only the pattern matters: live decisions ride a stream, money and board packs ride a batch, and a mature platform runs both.

Belts, partitions, and the ship's log

The belt is the Kafka-style log from last episode. Phones and services append events to it, from request through accept, pickup, and cancel, instead of calling pricing or payroll directly. Partitioning keeps the kitchen upright: many ordered belts side by side, more belts and cooks as load grows. Eve worries about events arriving out of order; the fix is to pin related events to one belt by a chosen key (trip, driver, hex), while unrelated keys never wait for each other. Order is local.

Why a log? Like a ship's log: written once, in sequence, never edited. A wrong price stays and the correction is appended after it, because a tape anyone can tidy is a rumor. Amazon offers Kinesis and managed Kafka, Microsoft offers Event Hubs (which speaks Kafka), Google offers Pub/Sub. A direct call from one service to another is a wish; the log is the shock absorber that lets pricing die and payroll sleep while the concert keeps writing.

Replay, duplicates, and twice equals once

A restarted consumer resumes from the last ticket it stamped; every reader keeps a bookmark. Events remain for hours to days as a general idea; Surya declines to invent any company's retention figure. The catch: replay can cook a ticket twice, and that is the deal, not a bug. The default promise is at least once; treat a repeat as a surprise and a driver gets paid twice.

Eve: “So the belt is allowed to be sloppy, and the cook is not.”

The remedy is a write where a repeat does nothing: a merge on a unique event ID. If 8841 was already applied, the second copy is ignored, and duplicate delivery turns into a boring failure, the word Surya says you want in this job. Merge on something sloppy, though, and a real second event, and someone's tip, disappears. Idempotence also permits deliberate replay after a bad deploy: rewind to the bookmark before the mess and rerun.

Why payroll stays off the conveyor

Eve pushes once more: the trip is done, so why not pay from the belt? Because pay is a judgment over a pile of lines that trickle in all afternoon, some deliberately late. So the day closes, a batch job joins the pile, and reconciliation follows: fares less refunds plus tolls must equal the cash about to be released plus the company's cut. If the check fails, nothing is paid and someone, or a gate, takes a look. That pause is the point; a conveyor cannot stand pausing.

Surya: “If late hurts more, stream it. If wrong hurts more, batch it and reconcile it.”

Eve closes by listing the moves: partitioned belts, a log that absorbs shock, replay from the last stamp, a merge on the event ID, and payroll kept off the conveyor. Everything else, Surya says, is just the logo on the door.

Figure: the fork — if wrong hurts more the number rides the truck (batch), the day closes, reconciliation runs and someone takes a look when the check fails; if late hurts more it rides the belt (stream). Below, the moves on the belt: partitioned belts, replay from the last bookmark, a merge on the event ID so twice equals once.

Takeaway

Takeaway: technology preference is the last reason, not the first.