John Stokvis

March 25, 2022

On understanding second order effects

Dunking on politicians is easy because everyone can agree that although they are given a lot of power, they almost never know what they're talking about. 

Putting aside truly malignant politicians, they can seem like Lenny from Of Mice and Men: powerful, meaning well, but ultimately ignorant of the ramifications of their power. The appropriate role for them seems to be figureheads - we should elect them to do absolutely nothing.

For example, when Nadine Dorries, the UK's Digital, Culture, Media and Sport Secretary asks a big tech company to "get rid of algorithms," it's easy to 🙄:

Dorries arrived at a meeting with software giant Microsoft and immediately asked when they were going to get rid of algorithms, according to an official given an account of the meeting. She also raised the same issue in a separate stakeholder meeting, a lobbyist familiar with the exchange said.

But I find it helpful assume good faith (as an exercise at least) and to look beneath the words she used to try to understand the meaning behind them.

Let's assume “algorithms” is code for something else, like “people feeling manipulated.” This sounds very similar to governments’ constant refrain of “we need access to encrypted messaging, but in a safe way.” 

Governments are concerned by their inability to fight bad actors (which is one of their basic responsibilities), but because technology has evolved faster than our cultural understanding of it and how it works, the solutions they propose get all jumbled.

This reflects a fundamental misunderstanding of the second order effects of product choices. And this is why it so important for anyone involved in building a product to be VERY CONSCIOUS of the second (and third and fourth...) order effects of the choices they make.

In this case, without algorithms, everything would be manually organized. People don’t have time for that. Algorithms help make things easier for people. In taking care of choices so people don't have to make them, algorithms also make people feel like they’re in less control. 

So maybe the real ask behind "get rid of algorithms" is how do we make it so people feel more in control of the algorithms that power their software.

But now we’ve got politicians getting into the details of product design which is:
  • way outside of politicians’ expertise
  • subtle, when the primary tools that politicians have at their disposal are blunt (regulations)
  • REALLY HARD, even for people who specialize in this and think about it all day

But hard things are still worth doing. When you're building products and you've got a million things to attend to, try to make a little time to think through the second order consequences of what you're building. The small choices you make now can have huge effects down the road.