A week inside the algorithm, on camera
Five days with the team that decides what a recommendation system is allowed to optimise for, and the editors who audit it afterwards.
The audit room: every recommendation the system made last Tuesday, projected where the team could argue with it.
Watch: inside the lab where the recommendation team reviews the week's decisions.
The room has no windows, three whiteboards, and a projector that never seems to cool down. For five days, this is where a dozen engineers, two ethicists and one very tired editor decide what three million people will be offered to read next.
The system in question recommends local news. Its owners invited us in after a year of criticism that it amplified outrage and buried council coverage. Our condition: film everything, publish the footage, and let an independent editor annotate the decisions afterwards.
What the system is allowed to want
Every recommendation engine optimises for something. This one used to optimise for predicted reading time — a metric that quietly rewarded anger, because anger keeps people scrolling. The team's first decision, made on our second morning, was to replace it with what they call “completed understanding”: did the reader finish, and did they then read something else?
It sounds abstract. In practice it meant demoting the stories people rage-clicked and promoting the ones they finished. Council budgets rose. Shouting matches fell. Nobody in the room would call it censorship; the chief engineer called it “choosing the customer we want to deserve.”
The editors who audit afterwards
The second half of the week belonged to the auditors: two editors with no stake in the engineering, a spreadsheet of two thousand recommendations, and veto power. They overturned eleven percent of the system's choices — mostly local stories the model had scored as “low interest” because too few people had clicked them yet.
That is the loop the team now runs weekly: the system proposes, the editors dispose, and the differences become next week's training data. It is slower than pure automation and, by every metric they showed us, better.
The week in numbers
- 2,140 recommendations reviewed by the audit desk
- 11% overturned in favour of local and civic coverage
- 1 metric retired: predicted reading time
- 1 metric adopted: completed understanding
On Friday the projector finally cooled. The team ordered noodles, the editors filed their notes, and the system — oblivious, as systems are — kept recommending. Only now it recommends a little more like an editor, and a little less like a slot machine.
Related stories
Get the next feature first.
One considered edition every weekday morning. Free to read, free to leave.