Your Mind Plays Tricks on You: The relevance of Congitive Biases in Data Science

The diffusion of digital technologies has brought data into countless areas of our professional and private lifes. As this big shift also impacts all types of businesses, a lot of of them take measures and, for instance, try to facilitate a data-driven culture, formulate data strategies or invest in technology. Data Analysts and Data Scientists play key roles in many modern organizations. It can be assumed that a large number of people – this includes of course managers and all kinds of „data people“ – perceive themselves as rational and logical decision makers. The unconvenient truth is: They / we are not! A lot of human thinking is influenced by so called congitive biases. We will take a closer look at them in this blog post.

​​Cognitive biases are systematic patterns of deviation from rationality in judgment. These biases are subject to research interests in fields like psychology and behavioral economics. What we call cognitive biases are mechanisms that have developed within an evolutionary process. They already helped our ancestors in making fast decisions when needed and with limited information processing capabilities. These biases are not only an essential building block of our "gut feeling" but also our intuition to a ceratin degree. This is what Daniel Kahnemann, nobel prize winner for economics in 2002, has called System 1, the area of unconscious and fast decision making in our minds. The speed and ease of this sytem comes with a price as biases can lead to irrational and counter-factual decisons. Biases can affect human power of judgment in a professional context and in personal life.

Presumably rational and fact-oriented people like analysts and data scientists are not save from cognitive biases either. Some authors even argue that they are even more prone to be to biased due to the experimental and research-oriented nature of their work. As biases are essentially part of human nature and they are everywhere, it is important to be aware of them. This might enable us to give better advice to others and take more informed decisions ourselves. We will try to provide a light introdcution, some hints for prevention and some interesting sources for further reading. Let us look at the most relevant cognitive biases one by one.
​

Confirmation Bias

The challenge: One could say that the Confirmation Bias is the „flagship“ of cognitive biases. The underlying idea is that we favor data that confirm our existing beliefs and hypotheses. As everybody wants to be right, confirmation bias is literally everwhere. The mechanism is highly relevant for data scientists. One will tend to interpret results of analyses or model predictions as support for prior assumptions. It will not be possible to avoid confirmation bias completely – but being aware of it will help a lot.

What you can try: Write down any of your relevant hypotheses, ideas etc. before you run an actual analysis. As soon as you have results, go back to your notes and cross-check them with prior beliefs.
​

---

Anchoring Effect

The challenge: As the metaphorical name suggests, the anchoring effect implies that the first quantitative judgment a person makes has impact on subsequent judgements. When a salesperson offers you a more expensive product or tells you a higher price at the beginning of your talk, he or she tries to anchor you the expensive price point. You are expected to use this anchor as benchmark for the subsequent prices to make them look cheaper. 

Anchoring was researched intensively by Kahnemann and Tversky in the 1960ies and 1970ies. However, there must have been awareness of this bias for centuries - just visualize the bargaining on ancient markets.

Data scientists might be influenced by anchoring when groups or the impact of several new features are compared. Also here the very first interpretations will set the frame for the following ones.
​
What you can try: Define an anchor level a priori. If you can for instance define a 5% lift in accuracy as a significant improvement, you can go back to this unbiased benchmark as soon as you have results. 

---

Availability Bias

The challenge: We tend to take cognitive shortcuts by relying on information and prior experience that is available and easily accessible. This over-reliance in what is there may result in neglecting additional sources of data that would potentially improve predictions. If you are a data scientist developing a churn model for regular gifts, it is straightforward to take sociodemographic and behavioural data into consideration. The existence of the availabilty bias makes it worthwhile to think outside the box particulary in terms of feature selection, knowing that there will be relevant constraints such as data protection rules, availabilty of data in the company of on the market and last not least data quality.

What you can try: Invite others who potentially add different perspectives and ideas. Organize brainstormings and jointly formulate hypotheses.

​

---

Curse of Knowledge

The challenge: Knowing to much can be a challenge when you try to communicate your knowledge to others. The curse of knowledge is when someting is completely clear to you and you assume that it is obvious to everyone else. For data scientists, this bias can be a big obstacle when they are supposed to present results to stakeholders. Data scientists often invest lots of time and energy to build analyses, models and lines of argumentation step by step. In many cases, everything finally makes sense and you have developed a big picture. Others have not gone through this process and did not have the chance to develop the level of understanding that you have. The curse of knowledge comes in many forms such as using to many unexplained technical terms or jargon as well as too little elaboration and „story telling“.

What you can try: „If you can't explain it simply, you don't understand it well enough.“ is a quote attributed to Albert Einstein. For the context of data science projects, this could mean investing time and effort in developing comprehensible, concise and well-structured presentations, reports etc. Focus on actionable information and key results and provide additional background information if necessary or asked for.
​

---

Narrative Fallacy

The challenge: We all need stories. Humankind has used them to convey information wrapped in plots for thousands of year. As story telling is so deeply human, we look for them literally everywhere. Narrative fallacy refers to our limited ability to look at sequences of potentially unrelated and random facts, events etc. without finding logical links between them. Analysts and also decision makers might tend to connect dots, i.e. analytical insights, in a way that seems plausible. Nassim Taleb, author of „Black Swan“ says that Explanations bind facts together. These facts make more sense to us then and get more easily rememberd. The key question is whether these presumed relationships are really fact-based.

What you can try: Discuss whether a signal in data is strong enough not be noise. If this is the case, you might quickly develop a story. Try to formulate an „alternative narrative“ that leads to your results and is consistent with your data. Be aware that your story is one of many interpreations and that there might be unobserved or even unobservable relationships.

---

So what?

You can get it if you really want. But you must try, try and try.
Jimmy Cliff, You can get if you really want.

Overcoming cognitive biases completely might be almost impossible. However, raised awareness of how our minds try to trick us will already lead to noticeable improvements in judgment. If you are interested in the topic, we can recommend the following readings.

Books

• Thinking, Fast and Slow by Daniel Kahneman
• The Art of Thinking Clearly by Rolf Dobelli
• Predictably Irrational. The Hidden Forces that Shape Our Decisions by Dan Ariely

Blog Posts

• Practical Psychology for Data Scientists via Towards Data Science
• Dealing with Congitive Biases ​via Towards Data Science
• YourBias

All the best and read you soon!