ad hoc data literacy

Can I Just Get the Data?

4 min read
Can I Just Get the Data?
A DATA EXPORT! TO THE BACKLOG WITH YOU.

"Can I just get the data? Can I just get a dump? Can I please just connect to the database?"

If you've worked in data in any organization for more than five minutes, you've heard this plea. Usually it comes from someone who just wants to do their job but has been told they need to wait six months for a proper data pipeline to be built. Maybe it's a data scientist who needs to test a hypothesis. A product manager trying to understand user behavior. A content editor with a master's degree in cognitive psychology who's been waiting three years to get access to learning analytics data.

The answer is almost always no. Not because the data doesn't exist, but because getting it out is "complicated." Because there are governance concerns. Because the data team is backlogged. Because building another pipeline will break something else.

So they wait. And wait. And they accept that this is just how things work. And often they give up.

The Pain We Accept

There's a fascinating dynamic that happens in organizations. We encounter a problem so consistently that we stop seeing it as a problem. We accept it as the cost of doing business.

This is what's happened with data access. We've become conditioned to the idea that getting data is hard that we've stopped questioning why it has to be that way. We've built organizational structures around this assumption. Data teams become gatekeepers not by choice, but by necessity. Business users become supplicants, waiting months for simple requests.

But what if the fundamental assumption is wrong? What if getting data doesn't have to be the hard part?

From Five Months to Two Days

At Broadlume, Josh Pendergrass had earmarked five months to get catalog data out of their old COBOL ERP system for a new product initiative. Five months to extract, transform, and load data that already existed in their systems.

With Matterbeam, they did it in two days.

Not because we're magicians, but because we'd already solved the hard part. The data was already in Matterbeam's intermediate format, ready to be transformed into whatever shape they needed. They didn't need to build a new pipeline. They didn't need to wait for the data team. They just needed to specify what they wanted, where they wanted it, and Matterbeam delivered it.

Josh told us later that they had projects in their backlog they thought were impractical or impossible. Now they're doing them because they can actually get the data.

The Shift from Getting to Using

Most data infrastructure today is built on the assumption that getting the data is the hard part. So we centralize everything into massive warehouses or lakes, creating data gravity where everything gets pulled toward the center because moving it is so expensive.

But what if it were flipped? What if getting data was easy?

Everything changes. Instead of spending months planning a pipeline that locks data into a specific shape for a specific use case, you can just ask for what you need when you need it. Instead of building shadow copies because you can't get access to the real data, you can work with the actual source.

A data scientist at Pluralsight stopped me in the hall once. She'd gotten data in 30 minutes that she'd been struggling to access for three months. That early prototype was gnarly, but it gave her what she needed to do her job.

Making Data Useful

The insight is both technical and philosophical. We've been so focused on moving data that we've forgotten the goal: to use it.

Matterbeam doesn't eliminate the complexity of data transformation. It makes that complexity visible, mechanical and predictable. Instead of building brittle point-to-point pipelines that break when requirements change, we create an intermediate format that can be transformed into any shape you need.

Want your Salesforce data in a graph database? Easy. Need to blend clickstream data with opportunity records? Done. Want to replay three months of data to test a new algorithm? No problem.

The data exists in an immutable log, and transformations become fast, reusable operations that you can compose and recompose as needed.

The Flywheel Effect

When getting data is the easy part, something interesting happens. People start asking for things they never would have asked for before. Use cases emerge that you never would have planned for. Projects that seemed impossible suddenly become trivial.

At Broadlume, this started with a COBOL reporting problem. Their initial thought was getting the data was the hard part. But once they saw what was possible, it cascaded into Salesforce integration, customer lead flow consolidation, and a complete rethink of their data strategy.

That's the flywheel effect. When you remove the friction of data access, innovation accelerates exponentially.

Stop Accepting Pain

The most dangerous assumption in data infrastructure is that the current pain is inevitable. Waiting months for data access is just how things work. Building brittle pipelines and praying they don't break is the best we can do.

It's not. We can do better. We can build systems where "Can I just get the data?" has a simple answer: "Yes."

The question isn't whether your organization needs better data access. The question is how much longer you're willing to accept that it doesn't.

Share This Post