Sometimes you build the source first

Almost every BI agency website carries the same promise. We extract insight from the data you already have.

That sounds reassuring and it is usually true. But not always. At a contractor I walked into last year it simply was not true. Not because the data was messy or scattered. It did not exist.

What was there

An invoicing package that produced their invoices neatly and gave up nothing else. No usable integration, no export you could schedule. I had to write to their software vendor to ask whether there was any way at all to get structured data out of it, in any form: an API, a UBL file, a CSV on a server. That is the level the conversation started at.

An accountant working in a different package, as these things go.

And beyond that: nothing. No stock registration. No record of what happened on a job site. No schedule that lived anywhere other than in the owner's head and on a board on the wall.

That is not an exceptional situation. That is what it looks like at a great many SMEs as soon as you look past the invoicing.

The wrong reflex

At a moment like that the reflex is to propose a bigger system. An ERP that catches everything. Then within a year you will have data.

That is usually a bad idea, and not for the reason you think. It is not too expensive. It is that you are buying a package to answer a question you have not asked yet. At that point you do not know which numbers you are going to use, so you do not know what has to be captured either. What you get is a system that registers everything and explains nothing, and staff filling in fields without knowing why.

The other reflex is just as wrong: waiting. Get the house in order first, data afterwards. That waiting always takes longer than planned and rarely comes to anything.

What we did instead

We built three small things, each around one question the owner was genuinely stuck on.

An app to count stock. When a material drops below its minimum stock level, an order list rolls out as a PDF. Nobody has to interpret anything, you get a list you forward.

An app for site surveys. Four salespeople document a job site on location with photos, measurements and notes. Out comes a site survey sheet as a PDF, and at the same time everything goes to a database.

A planning board. Four crews of three people, job sites you drag around per day. The office and the owner work in it.

Each of those solves a practical problem on its own. That is why they get used. But the real effect sits somewhere else.

The side effect is the data

Every count in that stock app is a measurement point. Every site survey is a dated file with photos and measurements attached. Every scheduled day is a record of who was where and for how long.

After a few months you have something that was not there before: a series. And with a series you can start comparing. Which job sites structurally overrun. Which materials keep running out. Which crew gets which kind of work done fastest.

That is not a by product you discover afterwards. It is a design choice. Each of those apps writes to the same database, with the same keys, so the three hang together later. During the build that costs almost nothing extra, and afterwards it is the difference between three loose tools and a model of how that business runs.

Why this goes faster than it used to

Being honest: ten years ago I would not have proposed this. Having three apps built was a project of months and a budget a contractor with four crews does not take on.

That has changed. A working app with a database behind it, hosted and reachable on a phone, is now a matter of days. Not because I type faster, but because the tooling is different. The hosting is a push of a button, the database is up in minutes, and the writing goes many times faster than it used to.

That changes the arithmetic completely. Building a source yourself used to be a last resort. Now it is often simply the shortest route.

Where it does not work

If you do have data and it is only messy, this is not your solution. Then you just clean up and connect, and that is faster.

And if the process on the shop floor is not right, an app does not fix that. A count nobody performs produces no stock figure. The question of whether people will really use this comes before the question of what to build, not after it.

The claim

Working with the data you have is a good rule until they have no data. Then the honest question is not how to get more out of what exists, but what you can bring into being today so that in six months you will know.