Back to Blog

The Metrics Are Fine. Nothing Is Fine.

The Metrics Are Fine. Nothing Is Fine.

Here is the most dangerous thing that can happen to your business: the dashboard looks fine. Everything underneath is not fine. And you won't find out until the gap between those two things has been open for so long that closing it costs twice what it would have.

We've been watching a pattern surface across completely unrelated domains this week, and it's the same problem wearing different masks. The mask changes. The problem doesn't.

When the label stops meaning what it meant

Researchers in the UK found something unsettling: the share of young people who report having a mental illness has risen steeply. The share who say it actually limits what they can do day-to-day has barely moved. Same label. Wildly different underlying reality. When people are asked whether ordinary mood swings qualify as a mental health condition, many say yes. The category expanded. The population inside it didn't necessarily get sicker.

That's not a criticism of anyone struggling. It's a measurement problem with enormous practical consequences. If you're a healthcare operator, a wellness founder, a gym owner running recovery programs — and you're watching "mental health" demand numbers go up without understanding what that label actually captures anymore, you are making resource decisions on a signal that has drifted from its meaning. That gap is not neutral. It costs you.

Fake signals, real money

Phia, an AI shopping app, reportedly generated affiliate revenue through clicks that weren't earned, attributing sales to itself that happened through other means. On a performance report, those numbers looked like growth. They were noise dressed as signal. And the people downstream (brand partners, affiliate networks, investors) made decisions based on those numbers without knowing what they were actually measuring.

We've seen this in client work more times than we want to count. Not fraud, necessarily. Just attribution that doesn't map to reality. A checkout conversion rate that looks strong because the tracking pixel fires on page load, not on confirmed purchase. A "lead" count that includes every bot that filled out the demo form. A monthly active user number that counts anyone who opened the app, including people who opened it once to delete it. The metric is technically correct. It describes nothing useful.

The overconfidence problem underneath all of it

The community wisdom thread in Lenny's Newsletter touched on something that felt minor — how do you manage colleagues who are overconfident in their read of a situation — but it's actually the mechanism that makes all of this dangerous. Overconfidence isn't arrogance, usually. It's a signal-interpretation problem. The person with the wrong read isn't making things up. They're trusting a metric, a category, a label, and they haven't noticed that the label has come unstuck from the thing it was supposed to measure.

Developers who sidestep design decisions. Stakeholders who drive meetings off a dashboard that's three weeks stale. Founders who feel great about Q2 because the revenue number hit, while churn quietly built up underneath it. Same problem, different seats at the table. Everyone is looking at something that used to be a reliable signal and treating it as if it still is.

What the painter and the software CEO have in common

This is where it gets interesting. Andrew Salgado's gestural still lifes are paintings of flowers in various states of bloom and decay, rendered with enough looseness that the eye has to do some work. You can't just scan them. You have to actually look. The images resist the fast read. That's a choice. It's also a discipline. Salgado could paint tight, representational florals that read from across the room. He doesn't, because across-the-room clarity is not the same as accuracy.

The CEO of FL Studio goes to Reddit for feedback. Directly. In public. In a forum full of people who will tell him exactly what they think of his software, without softening it into a net promoter score. That's not a marketing stunt. That's a methodology. He's going to the raw signal on purpose, because he knows that by the time customer sentiment becomes a metric in a dashboard, it has been averaged, cleaned, categorized and stripped of the texture that makes it useful.

Both of these are the same instinct: distrust the clean read. Go closer. Look at the thing, not the abstraction of the thing.

What you should actually do about this

We are not saying burn your dashboards. We are saying: pick three metrics you trust completely and spend one hour this week tracing exactly how each one is generated. Not what it measures in theory. How the number actually gets made, step by step, from raw event to reported figure.

You will find something wrong in at least one of them. We have never done this exercise with a client and come up empty. What you find is almost never fraud. It's drift. A tracking tag that moved when the checkout was redesigned. A category definition that made sense in 2022 and doesn't anymore. A pipeline stage that everyone fills in differently. The number looks stable. The underlying reality changed months ago.

The gap between what your metrics say and what is actually happening is not a technical problem. It is a leadership problem. Technical problems get fixed by engineers. This one gets fixed by a founder who decides, once, to stop trusting the dashboard at face value and go find out what it's actually measuring.

The businesses we watch fall behind in year three or four almost never blow up on a bad quarter everyone saw coming. They blow up on a quiet quarter that looked fine on paper until it wasn't. The signal had been wrong for eighteen months. Nobody traced it back to the source.

Your most important metric right now is probably not measuring what you think it is. Go find out before someone else does.

Previous Post The Work Before the Work Next Post You're Building for a Grid That's Already Gone