The Problem Didn’t Change. Who Has It Did.

• 6 min read
The Problem Didn’t Change. Who Has It Did.
putting the pieces together in a new way

For most of the last year, almost every conversation I had with a customer or prospect turned into a conversation about AI.

It didn't matter where it started. We'd be talking about pipelines that broke, or a team that couldn't get the data they needed out of Salesforce, and within ten minutes we'd be talking about models and agents. What are you doing with AI? Isn't this something AI can just fix now?

It was frustrating but understandable. For all of us, the ground has shifted every few weeks: a new model, a new protocol, a board member who read something over the weekend. Budgets earmarked for "data" got reallocated to "AI," and nobody is quite sure yet what that money should buy. There's a ton of churn, uncertainty, and crazy pressure to look like you have a plan.

All of that put us in an awkward spot.

I didn't want to "slap AI on it".

The obvious move was to rebrand. Put "AI" in the tagline, change our domain to "matterbeam.ai", add a chatbot, call ourselves an "AI-native data platform" and try to ride the wave. Plenty of companies did exactly that. We had lots of internal team discussions about it. We even took some half-steps. One person at a startup event told me I was being dumb - "get that sweet, sweet, AI money before it's all gone!".

But that's not what I started Matterbeam for. We built Matterbeam to solve a real problem, one that I'd personally lived through for most of my career, and pretending it was suddenly an AI product because that's what the market wanted to hear felt like a betrayal. It felt cynical. I think most people can smell it too.

So for a while we just kept doing the work, and kept hearing the objection: isn't this AI? Can't AI just fix data? And honestly, I didn't have a great answer.

It probably took too long, but it clicked.

I started Matterbeam because humans have had a terrible time using data for decades. Every time I wanted to use data for something at a company, especially something novel, the first challenge was just finding it: which system has it, who owns it. Then you'd get access, stare at a pile of random stuff, and start the ritual. Where did this come from? Is anything still flowing into it? Does it even make sense?

Then getting it into a shape you could actually work with was its own mind-numbing slog, usually starting with a ticket to a grossly overworked data team.

So we built for those two halves. First, understand the data: its shape, its flow, what it's connected to, where it came from. Second, make it easy to take that data and put it wherever you need it, in whatever shape you need, without fear of breaking something.

At some point I was listening to yet another conversation about how to make agents work with enterprise data, and it hit me, they were describing the exact same problem. An agent wants to know what this data is. What shape it has. What feeds it, what it links to, what the distribution looks like. Which of the seven metrics tables is the real one, and is it really the one named gold_metrics_website_v6_USE_THIS_ONE?

That's not a new problem. That is the problem. It's the one we've been working on the whole time.

The vision didn't change: make it easier to understand and use data across an entire organization. What changed is who needs it most. If anything, it matters more for agents than for people.

Agents are becoming the main users of data

I'm not the only one seeing this. In a recent conversation, Joe Reis and Tristan Handy talked about agents becoming the primary consumers of data systems. Not analysts, not dashboards. Agents.

MotherDuck published another fascinating post, Agents Don't Query Like Humans Do, comparing queries from their UI (humans) with queries through their MCP server (agents). As of that post agents were running 29x more queries than humans - nearly doubling each month for the previous few months.

The shape of the work is very different, too. Humans ask a big question, then sit and think about the answer. Agents work in bursts: schema lookups, a quick distribution check, a test join, back out, try again. The queries are the agent thinking, and a lot of that thinking is just orientation, relearning the landscape of your data nearly every time it shows up. That burns tokens, burns warehouse credits, and trips metering models designed around how people query.

A happy accident

Almost every design decision we made for humans turns out to be exactly what agents need. We didn't plan it. It just fell out of solving the original problem honestly.

Understanding. We statistically profile every record that flows through Matterbeam, from every source. The profile lets people better find and understand their data and also lets us learn schemas on the fly. Having schemas then allows us to translate the data into whatever target shape you want. A continuously updated statistical profile of everything in your org is also precisely the ground truth to build the context an agent is missing.

Fearless use. We built Matterbeam on an immutable log so humans could spin up a new view of their data without touching production. This enables rapid and safe experimentation and the ability to easily react to change. Agents are more aligned with this same pattern. Even more important, these views can be a prefiltered, locked-down slice: PII stripped, only the fields it's allowed to see, tailored to the job at hand. This goes a long way to solving one of the legs of the "lethal trifecta".

Replay and audit. Agents are stochastic and might do something different every time. Immutability means you can go back and see exactly what an agent read and exactly what it built. That used to be a nice-to-have. With agents it's critical.

People cope with messy data because they carry around a huge amount of tacit knowledge about how the company works. Agents don't have that. They need it handed to them.

The experiments that got me excited

Theory is nice. What actually changed my mind was trying it.

Working with a couple of our existing customers, I started sending our metadata through an early version of our agent tooling. The models got only our statistical profile. No sample records. No schema. The tooling treats the statistics as ground truth and builds chains of evidence on top of them, so I could always ask: how did you get there?

I picked datasets more or less at random. The first one came back with: this is test data. Really? How? It flagged a suspiciously round number of records. Value distributions that were obviously artificial. And no steady flow of new data, just occasional pulses. All correct, and nobody had told it.

The second one is the one that really got me. I threw in a CRM object, not cherry-picked. It said this was a Salesforce opportunity object, and that the stream was change-data-capture, not an export. Fine, you could guess that from field names and structure. Then it said this was a company selling digital services to flooring vendors and flooring manufacturers, running a SaaS model that tracks MRR. Okay, way more interesting.

Then it told me the company had been through a series of recent acquisitions, and named the companies and the dates. That was true. How on earth did it figure that out? I walked the evidence chain. There was an admin email field. The profile had its distribution over time. On one date the email domain flipped from one company to another. On a later date it flipped again. The model recognized those domains as companies and read the pattern as an acquisition chain.

Warehouse statistics will tell you a column has 4,000 distinct values. What I was looking at was something closer to understanding: what this data is, where it came from, whether to trust it. And it came out of machinery we'd already built, for a completely different reason.

That's when I stopped worrying about whether we were "doing AI."

To be clear about where we are: the core platform (collect from any source into an immutable log, profile everything continuously, rematerialize into whatever shape you need) runs in production today. The agent-facing tools layer on top of it is early. We're building it with a small number of teams who are running agents against real, messy data and feeling the pain, not just running demos on a hand-curated table.

It may take time, but the AI churn will settle. I strongly believe the underlying problem won't go away just because the consumer changed from a person to a program. Data is still scattered across dozens of systems. It's still messy. Something still has to understand it and get it into the right shape for the job. A while back I wrote a reaction to "can't AI just fix this data?": if humans, with all our context, struggle to use data effectively, why would agents magically do better? That's still true. I just didn't see then that it's the same problem.

We've been working on that for years. I've felt it in my bones, but it's still pretty exciting that it seems like it was the right problem all along.


If any of that sounds interesting, or you're working in this space, I'd love to compare notes: mikepk@matterbeam.com

Share This Post