The System Design Pattern

Early in my career I was the data guy in a fledgling e-commerce and merchandising group at Continental Airlines.

We were trying to prove that the airline's website was more than a low-fare distribution channel—slicing bookings and revenue by channel, member status, local/flow, fare class and cabin and displaying it all as a booking curve—all to challenge the conventional wisdom about who our direct customers were and what they wanted.

My boss was a revenue management veteran who'd built our group from nothing. He forwarded some analysis I'd done to a wide audience as evidence for something he was pushing. An exec questioned the numbers. And I, an eager analyst who had no business commenting on the thread, replied all. I said something about data silos—implying his group had a different version of the truth than ours.

His Director of Revenue Analysis called me into his office the next day.

He really didn't like the word “silo” and made me explain what I meant. Then he dressed me down for sowing doubt with leadership. He was probably right—we were both reporting off the same data warehouse after all. But I'd stumbled into a truth I didn't yet have the language for: two teams, same data, different answers, and the political minefield waiting for whoever points it out.

Twenty years later, I've seen that same pattern at every airline I've worked with. It's the first of five patterns I'll share as I launch this newsletter, because everything else I've learned about making data work starts here.

Finance double counts revenue because they can't see exchanges. Revenue Management forecasts from incomplete history—blind to precise booking curves. Marketing is off in another world—tags, events, carts. Everyone is delivering reports that look good but never answer the next question fast enough or without a change request.

Three departments. Three results. Same airline. It isn't like this everywhere. But it's certainly more prevalent than we want to believe.

And the problem isn't the dashboards or the database. It's that nobody has the same, precise, point-in-time definition of revenue and the attributes you need to slice it by. And no amount of visualization or supercharged query times fixes data you can't trust.

At many airlines, data services reports into IT. The people who build data products are measured on technical metrics—uptime, CPU and storage cost, delivery against timelines. None of those metrics ask: did the business get an answer it needed in the time it needed to make a difference?

So the commercial teams build workarounds. Shadow databases. Spreadmarts. That one analyst who knows how to get the right number before every earnings call, because they've been manually reconciling sources for years.

These workarounds become load-bearing. Invisible to the leaders making the investments.

In too many places, every report has its own pipeline—an end-to-end path from source system to finished output. Each pipeline pulls only the slice of data that one specific report needs, transforms it along the way and unknowingly bakes a business rule into the code.

Finance has a revenue pipeline with Finance's definition of revenue. Revenue Management has a different pipeline with a different version. Marketing has a third. Each one works. None of them agree. And when the CEO asks a question that doesn't fit any existing report, someone has to build a new pipeline from scratch. That's why new analysis takes months. And that's why the business stops asking new questions.

No single person is failing here.

Every person in this system is behaving rationally given their incentives, their information and the org structure they operate within. This isn't a people problem. It's a system design problem—and no tool migration, no tech modernization fixes a structural flaw.

This is what I mean when I talk about focusing on outcomes over technology. The instinct is to solve a business problem by buying something. But that won't work.

The teams I've seen solve this didn't start with a platform migration or a new vendor. They started by getting the right people together and making them uncomfortable.

Sales & Distribution, Revenue Management, Digital and Marketing—in the same room, answering the same question: what do we mean when we say customer, sales channel, revenue? Not philosophically. Operationally. When is something sales and when is it revenue? How do we handle exchanges? No-shows? How are we going to prorate ancillaries?

Every one of these we think is obvious. But these conversations are slow and uncomfortable. People discover they've been defining things differently for years and blaming the tech instead of the process. Working through it together—agreeing on shared definitions, arguing about edge cases, forcing clarity where ambiguity has been the default—is how we make data work.

What emerges from these rooms every time I've seen it work is a unified data layer that replaces all of those siloed pipelines at the foundation. Instead of each report pulling its own slice, you bring everything from the source systems at the lowest grain, with full history. Every booking event, every ticket issued—created, modified, canceled, exchanged, lifted, flown, accrued, redeemed—timestamps turned into versions. Not a summary. Not a snapshot. The complete record, with shared definitions agreed upon by the teams that use it.

This layer isn't owned by one department. It's the foundation all of them build from. When anyone queries revenue, they're starting from the same events, the same history, the same agreed-upon definitions. The team that built it together trusts it because they defined it together.

And here's the real payoff: when the CEO asks a new question, you're not building a new pipeline back to the source. The data is already there. Already clean. Already defined. And that analysis that used to take months now takes hours.

Seasonality adjustment recommendations worth millions in RASM—ready when you need them. Premium seat price sensitivity analysis—powering your take rates into double digits. Finance forecasts your board trusts.

We know it's working when the business doesn't just ask more questions, they ask different ones.

It was just a simple A/B test running for a few weeks. Clean random holdout, directionally positive, but the significance wasn't there yet. We'd undersampled and needed more data.

We spent the next couple of weeks looking for a way around it. Found a clever diff-in-diff that produced a bigger number with 80% confidence. Defensible for what we were doing but dang was it going to be hard to explain. So we didn't.

By the time we gave up on diff-in-diff, we'd collected twice as much data as we'd originally pulled in and we simplified the analysis on the single outcome we were looking to prove.

And that simple test—the same one we ran on day one—came back significant. The clever method was doing what clever methods do: producing a bigger answer by introducing assumptions the design wasn't built to carry and we couldn't explain in plain language.

I spend a lot of time writing about trusting the foundations over the tech. This week I had to write it back to myself.

—

And in the actual kitchen this week—my youngest asked me to make tavern-style pizza for her birthday dinner. The thin, crispy Chicago kind, not deep dish.

I grew up in Texas but had an uncle who worked for United in Chicago, managed the charter sales for the Bears, Blackhawks, etc. My first flight was DFW–ORD for his funeral. Years later during the CO/UA merger I spent months in the same Elk Grove offices he had worked in, then had one of my own in the Willis Tower downtown. The pizza in Chicago stuck with me—both kinds.

A couple years ago J. Kenji López-Alt did a NYT series on tavern-style and I got obsessed. Now I make them on the outdoor pizza oven. 5-day dough. Homemade Italian sausage. Giardiniera. 3-minute cook time.

That senior analyst who replied all about data silos twenty years ago? He wasn't wrong. He just didn't know enough to be useful about it yet. The pattern was real and it took another decade of seeing it everywhere to understand why—and what to do about it other than "reply all."

Three departments and three answers. Three vendors and one unsolved problem. One clever statistical method too tangled to explain. Same pattern, every time—our systems design the outcome.

Next week: The Shared Definitions Pattern. Where good enough satisfies us but quietly kills our innovation.

Thanks for being here as we push back from the gate.

—Ben