Counting the uncountable

“Not all things worth counting are countable and not all things that count are worth counting.” — Albert Einstein (Or was it?)

Chris Dillow reviews The Tyranny of Metrics by Jerry Muller, a book about “how the obsession with quantifying human performance threatens our schools, medical care, businesses, and government.”

The Tyranny of Metrics: a review
Muller provides lots of examples of this, mostly from the US. But you’ll all have examples of your own. In universities the Research Assessment Exercise (now the REF) contributed to increased administration costs and perhaps to the replicability crisis by incentivizing the publication of mediocre research. In schools, targets can encourage teaching to the test, endless revision and a focus upon the marginal student to the neglect of both the strongest and weakest. Waiting-time targets might distort clinical priorities. Immigration targets deter foreign students and lead to the harassment of people who have lived here for decades. Sales targets encourage workers to mis-sell financial products, cook the books, or increase risk by encouraging “liars’ loans. And so on.

It’s not all bad news, though. It’s just a question of balancing the quantitative with the qualitative.

The Tyranny of Metrics is not, however, a diatribe against targets. Muller points to the experience of some US hospitals to show that metrics can work. They do so, he says, when they are “based on collaboration and peer review”:

Measurements are more likely to be meaningful when they are developed from the bottom up, with input from teachers, nurses and the cop on the beat.

In other words, metrics can succeed when they are complements to knowledge: when they organize the tacit and dispersed professional judgements of people who know ground truth.

Author: Terry Madeley

I work with student data and enjoy reading about art and design, data, education and technology.