I lost the same model twice

This is not an article with a tidy conclusion. It is one about something that went wrong, twice, and about what I have done differently since.

I was working on a data model in a project format, the kind where your model sits on your disk as separate files instead of in one large combined file. That has advantages: you can follow the changes, you can use version control, you see what moves.

That model crashed completely twice. Not damaged, not partially lost. Gone.

What probably happened

I was working on the same model in two ways at once. I was changing things in the design tool, and at the same time I was writing to it programmatically through the interface that allows for that.

Those two both do the same thing: they write to the model. And the design tool is meanwhile also synchronizing to the files on your disk.

Three parties writing to the same state without anyone being in charge. That goes fine for a while and then one time it does not.

I cannot prove that was the cause, so let me say it as it is: this is my best explanation, based on when it happened. But since I have stuck to the rules below, it has not occurred again.

What I do now

Back to the ordinary file format. That is a step backwards in terms of version control and I did it reluctantly, but it holds up. The advantages of the project format did not outweigh starting over twice.

Never work in the design tool and through the programmatic route at the same time. One of the two at a time, and the other closed. That sounds obvious and it is exactly what you forget when you want to change something quickly.

Bundle write actions. Instead of twenty small changes one after another, one block that goes through in one go. Fewer moments where something can be left halfway.

Limit recalculations. Every change that forces the model to recalculate everything is a period in which it is vulnerable. Do three of those in a row and you have that risk three times.

The broader point

What stayed with me from this is not the technical lesson. It is how rarely this kind of thing gets written down.

You find thousands of articles online about how to build something. You find very few about what breaks and why. That is partly because it is not flattering to write that you lost two days of work, and partly because it only happens once you have been doing something long enough.

While that is exactly what somebody needs. How you build a model is in every manual. That you should not be working on the same model from two sides at once is nowhere, and it costs you an afternoon to find out for yourself.

What you can take from this

If you work with a model that several tools write to, agree on which tool is in charge at which moment. That applies when you work alone as well, because then you are the one interrupting yourself.

Make a copy before you start something structural. Not as a procedure, just as a reflex, the way you save a document before making a big change.

And write down what went wrong when it went wrong. I did that, and it is the only reason I can tell you this now instead of wondering whether it was me.

The claim

The manuals explain how you build something. What makes it a craft is knowing where it breaks, and you only learn that by losing it once.