Your very own Grima Wormtongue

The current explosion in startups developing software underpinned by social recommendations and similarity scoring has brought to the fore what I think is a worrying unintended consequence. All these algorithms are in one form or another continually reinforcing our confirmation biases and narrowing our view of the world. And these effects are subtle and progressive.

Confirmation bias in a nutshell is the effect that we are more ready to accept and internalise information that agrees with our personal and in-group biases, and avoid or discard information that does not.

Newspapers, for instance, are well aware of this tendency and the editorial slant and stories are mainly selected in order to appeal to their target market's intrinsic viewpoints, either positively or negatively. Similarly most modern advertising is founded on segmentation targeting and is designed to appeal to the perceived biases of the segment it is being pitched to.

We have enough experience of these idioms that it is relatively straightforward to separate these influences from our day to day life in the main. However, these approaches are now pulling through into many social sites and it is much more difficult to keep a clear distinction between product and real discourse in this setting. A large amount of concomitant information in our online social interactions will inevitably become mediated by advertising segmentation and ideas presented via network analysis that is disguised as social peer communication.

While I doubt the companies rushing into this space are considering this angle, I think it is a progressive force pushing us towards an online existence where we are increasingly blind to information outside our immediate circle. Indeed as this process continues we are in danger of being funnelled into an experience of recycled information and ideas that are tailored never to challenge our biases, and we will most likely be largely unaware of this process.

All humans have in-group and identity biases, it is an instrinsic part of our mental model, however, we should maybe reflect how this constantly tailored view of information and product by a third party interwoven with the information and biases passed around during our normal peer interactions can affect us.

Imagine in the real world a constant companion; whispering ideas, recording our conversations and friendships, and subtly promoting products in all our social interactions. Should we over time come to accept this as normal, or be concerned about a Grima Wormtongue we are inviting into our online life?