What broke before the smoothie word
People say lakehouse as if it were a smoothie, so Eve asks what broke. Two buildings, Surya says, each a little. The cheap, dumb lake kept everything but could not update a row, erase someone who asked to be forgotten, or tell two cooks which files made up a table without a hopeful folder name. The clever, fussy warehouse had transactions, types, and SQL finance trusts, but was historically closed and expensive.
The real damage was two copies of one truth drifting: last week's pile in the lake, last night's plate in the warehouse, a third number invented by whoever joined them. The lakehouse is that complaint turned into a building.
Same three verbs, different order
Eve still confuses the two acronyms; Surya says the order is the entire point. In the old order, food was cooked in a side kitchen en route to the pantry, because warehouse seats were expensive, so only dessert was loaded, and a changed recipe meant calling the Informatica-class machine in the closet, or the farm.
Now storage is cheap and the pantry cooks for itself: load raw, transform inside the warehouse with dbt, recook when the recipe changes. His example: active customer means anyone who ordered within ninety days, until legal or finance moves the definition. With the raw shelf intact you edit the SQL and rerun; with only dessert, you are phoning a system that has forgotten January. But verb order alone, he cautions, is not yet the smoothie word.
Three brains on the same files
Iceberg, Delta, and Hudi are not a boy band. They are three metadata brains on ordinary files, usually Parquet in the cheap fridge, that know which files constitute the table right now. That gives the fridge warehouse manners: transactions, so nobody reads a half-written load; column changes without rewriting the ocean; time travel, which rolls a poisoned Tuesday back to Monday; and updates or deletes without photocopying every file.
Why three? Different kitchens raised them. Iceberg began at Netflix and moved to Apache; its hidden partitioning keeps the slicing in the brain rather than folder names, so the fridge can be recut without every cook learning a new hallway. Spark, Snowflake, and others read it. Delta Lake came from Databricks and suits cooks living in its notebooks. Hudi came from Uber, is Apache too, and is known for streaming insert-or-update all day long. Surya declares no winner; choose for the day you have and for whoever must read the table a decade on.
Surya: “Lake prices. Warehouse manners. One copy of the data instead of two.”
The files live in S3, Blob Storage, or Cloud Storage. Compute becomes a rental car; the data stays the house.
Two kitchens, one belt, or a hybrid
Lambda came from wanting two things that hate each other: a number now, and a number you would defend. It built two kitchens, fast-and-messy and slow-and-correct, merged at the pass. Robust, but every rule is written twice and drifts: the stream picks up Tuesday's new tax rule while the batch job keeps last quarter's. Not a tooling problem, Surya says. Two kitchens.
Kappa treats everything as a stream, with the log as truth and batch as a replay from ticket one. One kitchen, simpler logic, but it needs mature streaming, long retention, and people who can live that way; and since most logs forget, the file fridge still matters. Most grown-ups are hybrids: stream what needs freshness, batch the rest, and keep bronze replayable for the morning you make a mistake.
Contracts that fail the right deploy
Data contracts sound like lawyers to Eve. Surya calls them a versioned promise: the producer states what a field means, how fresh it arrives, that it is not half null, and that renaming it breaks their deploy rather than his 3 a.m. Slack. The example: the orders team renames order-total to total-amount for tidiness. Without a contract, gold goes null overnight; with one, their pipeline fails at four in the afternoon while they are awake.
The promise covers meaning (tax or returns included?), quality (null rates, unique IDs), freshness as a service level rather than a vibe, and evolution rules: add fields freely, never rename silently, and ship breaking changes as a new version with a warning. Not in a wiki, which fails no deploys, but in a schema registry on the belt, a git spec the producer's deploy reads, and dbt tests on the warehouse side.
Steal the medicine, leave the religion
Every consultant has sold Eve a mesh. Surya credits Zhamak Dehghani and grants that the underlying problem is real: in a huge company, a single central data team drowns in tickets. The first pill is domain ownership: the Orders team treats orders as a product, meaning a name, an owner who answers Slack, a freshness service level, and a table a stranger can find. A paved road is the platform team supplying templates, a self-provisioning warehouse, Airflow as a service, and a catalog, with global rules enforced by that platform rather than a committee.
The warning: mesh costs duplicate skills, and a smaller company that renames its org chart usually ends up with the same exhausted platform team plus half-finished products. So take the medicine without converting: owners, service levels, a paved road, a strong central platform, domain dbt projects, and contracts that fail in daylight.
Surya: “Owners. Service levels. A paved road. Don't rename your company around a blog post.”
The recap: a contract fails whoever changed the recipe, not whoever is eating at 3 a.m.; Lambda's kitchens drift; most teams stream the minute, batch the truth, and keep a raw shelf. It was never a smoothie, Surya says, only a complaint about photocopying ourselves.
