Model Thinking When All Models are Wrong

In the late 1970s, statistician George Box created the famous wisdom, “All models are wrong, but some are useful.” The earliest version of the sentiment comes from Box’s 1976 article, “Science and Statistics”:

Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary, following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so over-elaboration and overparameterization is often the mark of mediocrity.

While often used to explain that statistical, and especially econometric, models are not designed to replicate reality, merely help explain it. And this is true, but it turns out the sentiment extends well beyond statistics.

Models generally fall into one of two categories, prescriptive and descriptive. That sentence is itself a model describing models, and that’s more than a metanotation. It is itself critical to understanding models.

Descriptive models are those designed to explain reality, after the fact. Statistical models are one type, but so are the electron shell model of chemistry and Newton’s gravitational model. Notably, both are wrong. There are no electron shells and Newton’s gravitational model breaks down beyond certain speed thresholds. Though these models are wrong, they are useful. They provide a simplified version of their underlying systems that help understand the systems. And that’s why they are useful.

Prescriptive models are a different beast. They explain reality, but before the fact. I think the best example of a prescriptive model is an architectural drawing. Such a document does not, within perfect precision, nor complete fidelity, explain how to build a building. There is still engineering, design, even basics such as painting that happen before a building is done. So it is wrong, but it is still useful. An architectural drawing is sufficient to start a project.

As we teach model thinking, and its close cousin, systems thinking, we need to be cognizant of reminding students that the model is not the end, but only the beginning of thinking about a project. It helps you understand the underlying system but it is never the system itself.