Daniel Kahneman talks extensively about how we make reasoning errors because we tend to use mental shortcuts. One mental shortcut is ‘substitution‘. Substitution is what we do when we (often unconsciously) answer an easier question than the one being asked. I find that I sometimes do this in my own research. For instance, when I set out to answer the question, “How can X be rational?” I sometimes end up answering easier questions like, “How does X work?”. In an effort to avoid such mistakes, I will (1) explain the question substitution error, (2) give an example of how we can distinguish between questions, (3) give a personal example of the substitution error, and (4) say what we can do about it.
In case you’re not familiar with Kahnemen’s notion of ‘substitution’, here is some clarification. In short, substitution is this: responding to a difficult question by (often unintentionally) answering a different, easier question. People use this mental shortcut all the time. Here are some everyday instances:
|Difficult Question||Easier Question|
|How satisfied are you with your life?||What is my mood right now?|
|Should I believe what my parents believe?||Can I believe what my parents believe?|
|What are the merits/demerits of that woman who is running for president?||What do I remember people in my community saying about that woman?|
For further discussion of mental shortcuts and substitution, see Part 1 of Kahneman’s Thinking Fast and Slow (2012).
Now, how does this mental shortcut apply to research?
2. Some Research Questions
Regular readers will know that I am interested in — among other things — the differences between reflective and unreflective reasoning (Pennycook et al 2015). Philosophers ask lots of questions that involve reflective reasoning:
- Must I do this? (Korsgaard 1996)
- Should I believe this? (Sosa 2009)
- Why do I believe that? (Kornblith 2012)
- Why did I do that? (Doris 2015; Vargas 2005)
- What do we know about our own minds? (Cassam 2015)
- How is my relationship to my own mind different than my relationship to other minds? (Moran 2001).
Obviously there are other questions that involve reflective reasoning. These are just a few. And, obviously, there are also hybrid questions that involve reflective reasoning. For instance, there are normative-explanatory questions like “Why is it wrong to X?” And there are normative-explanatory-self-knowledge questions like, “Why do I believe that it is wrong to X?
The point of this section is just to point out that we can — when we are careful — discern clear differences between similar questions.
3. Falling Prey To Substitution
Like any researcher, I don’t want to make an erroneous mental shortcut in my research. Alas, I sometimes find myself doing exactly that. For instance, I will substitute an answer to an easier question for an answer to my more difficult, research question.
|Research Question||Easier Question|
|How can reflective reasoning be rational?||How is reflective reasoning related to our behavior?|
|How does reflective reasoning work?|
|How does reflective reasoning fail?|
I sometimes mistake answers to these easier questions for answers to the target question. (Based on my reading of the literature, I do not think that I am the only one who does this. But that’s a much longer conversation.)
Don’t get me wrong. We might have to answer the easier questions as a means to an answer to the target question. But we shouldn’t mistake the means for the end. That is, we can’t think that the answers to the easier questions just are answers to the target question.
4. General Take-aways
Consuming research. When a researcher tells us that they plan to answer a certain question, we should take note of that question. And then we should pay attention to see whether their research, argument, etc. can and/or actually does answer their target question. If the research can’t or doesn’t answer the original question, then we should take note of it. Heck, we might want to do more — e.g., tell the researcher.
Doing research. Once we pick a research question, it is important to regularly remind ourselves of that question throughout our enquiry. Otherwise we might lose sight of the target question. And when we do that, we might end up substituting the target question with an easier question. And, worse, we might then erroneously think that our answer to the easier question — fruitful, though it may be — is an answer to the original, target question.
Cassam, Q. (2015). Self-Knowledge for Humans (1st edition). New York, NY: Oxford University Press.
Doris, J. M. (2015). Talking to Our Selves: Reflection, Ignorance, and Agency (1st edition). New York, NY: Oxford University Press.
Kahneman, D. (2013). Thinking, Fast and Slow (1st edition). New York: Farrar, Straus and Giroux.
Kornblith, H. (2012). On Reflection. OUP Oxford.
Korsgaard, C. M. (1996). The Sources of Normativity. Cambridge University Press.
Moran, R. (2001). Authority and Estrangement: An Essay on Self-Knowledge. Princeton, N.J: Princeton University Press.
Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2015). Is the cognitive reflection test a measure of both reflection and intuition? Behavior Research Methods, 1–8.
Sosa, E. (2009). Reflective Knowledge: Apt Belief and Reflective Knowledge. OUP Oxford.
Vargas, M. (2005). The Trouble with Tracing. Midwest Studies In Philosophy, 29(1), 269–291.