What Intelligence Tests Miss: The Psychology of Rational Thought

by Keith E. Stanovich

An engaging discussion of the important cognitive characteristics missing from IQ tests

Critics of intelligence tests—writers such as Robert Sternberg, Howard Gardner, and Daniel Goleman—have argued in recent years that these tests neglect important qualities such as emotion, empathy, and interpersonal skills. However, such critiques imply that though intelligence tests may miss certain key noncognitive areas, they encompass most of what is important in the cognitive domain. In this book, Keith E. Stanovich challenges this widely held assumption. Stanovich shows that IQ tests (or their proxies, such as the SAT) are radically incomplete as measures of cognitive functioning. They fail to assess traits that most people associate with “good thinking,” skills such as judgment and decision making. Such cognitive skills are crucial to real-world behavior, affecting the way we plan, evaluate critical evidence, judge risks and probabilities, and make effective decisions. IQ tests fail to assess these skills of rational thought, even though they are measurable cognitive processes. Rational thought is just as important as intelligence, Stanovich argues, and it should be valued as highly as the abilities currently measured on intelligence tests.

Genres: Psychology, Nonfiction, Science, Education, School, Popular Science

308 pages, Paperback

First published January 27, 2009

About the author

Keith E. Stanovich is Emeritus Professor of Applied Psychology and Human Development at the University of Toronto and former Canada Research Chair of Applied Cognitive Science. He is the author of over 200 scientific articles and seven books. He received his BA degree in psychology from Ohio State University in 1973 and his PhD in psychology from the University of Michigan in 1977.

Reviews

[Michael Connolly · Rating 5 out of 5]

This is a book about cognitive psychology. Its main points are that:

(1) Good thinking requires more than Intelligence; it also requires rationality,
(2) Intelligence tests do not measure rationality, they measure only intelligence,
(3) There is a poor correlation between intelligence and rationality
(4) There is currently no test for rationality,
(5) Rationality is a purely cognitive skill and is distinct from emotional intelligence and social intelligence

The author describes a tripartite framework of the mind: autonomous mind plus algorithmic mind plus reflective mind.

Autonomous Mind:

(1) facial recognition,
(2) kin recognition,
(3) gaze direction detection,
(4) reading the intentions of others.
(5) depth perception,
(6) proprioception,
(7) language ambiguity resolution,
(8) syntactic processing.

The autonomous mind is unconscious. There is little difference between individuals in the autonomous mind.

Algorithmic Mind:
The algorithmic mind contains intelligence as measured by intelligence tests. The author argues that it is best to restrict the use of the word intelligence to intelligence as measured by intelligence tests, and then to use the word rationality for the other cognitive skills. Historically, a distinction has long been made between fluid intelligence and crystallized intelligence. Fluid intelligence is cognitive decoupling and the size of working memory. Cognitive decoupling is hypothetical thinking. Crystallized intelligence is declarative knowledge. Intelligence tests measure mainly fluid intelligence. There are large individual differences between individuals when it comes to fluid intelligence.

Reflective Mind:
The reflective mind contains rationality, a cognitive ability that is missed by intelligence tests. Stanovich distinguishes epistemic rationality and instrumental rationality. Rationality has overlap with Robert Sternberg's practical intelligence. The reflective mind may override the algorithmic mind. Antonio Damasio has shown that the reflective mind, but not the algorithmic mind, requires emotional input from the autonomous mind.

Thinking Dispositions:
Reflective mind contains thinking dispositions, such as:
(1) superstition,
(2) dogmatism,
(3) loyalty to beliefs versus loyalty to reality,
(4) overconfidence,
(5) insensitivity to contradictions,
(6) need for closure,
(7) thinking about future consequences,
(8) hierarchy of goals, and
(9) reflective versus impulsive personality.

Cognitive Miser:
The cognitive miser is a part of the mind that tries to use the reasoning method that requires the least effort. Examples of the cognitive miser:
(1) Affect substitution, that is, making a choice based upon the emotions associated with each choice,
(2) Accepting the given framing of the question, rather than searching for the proper framing,
(3) Avoiding the exhaustive enumeration of all possible cases (called disjunctive reasoning) by using the easier serial associative cognition (considering only the cases that come immediately to mind)

Mindware Gap:
Examples of mind ware gaps:
(1) not having a critical thinking strategy,
(2) failure to seek falsifiability,
(3) not considering alternative hypotheses,
(4) mistaken causal inferences,
(5) not understanding independent probabilities,
(6) not understanding conditional (Bayesian) probability,
(7) inverting probabilities (confusing A given B with B given A),

Lack of education and experience are causes of mind ware gaps.

Contaminated Mindware:
Examples of contaminated mindware:
(1) Albanian Ponzi scheme,
(2) ritual satanic abuse panic,
(3) parasitic meme-plexes (cf. Richard Dawkins),
(4) faith-based mindware (which disables the critical faculty).

[Esteban del Mal · Rating 2 out of 5]

Dear Psychology,

I really wanna be friends. You ALWAYS have the answers and are so much smarter than me.

But, well, with your "cognitive miser," "fluid intelligence" and "myopic loss aversion," you come off as kinda smug. And how many times are you going to say the same thing? Over and over and over again? Don't you know when to shut up? Or is just that you have nothing else to say? You're like my grandmother who goes on about her recipes. Don't get me wrong -- the food is great, but no matter how much she talks, I'll never be able to cook like that. In fact, I don't want to! I've got other stuff to do! Besides, her house doesn't even look like anyone lives in it and it smells funny.

I still dig you and all, so I'm sure I'll come hangout again sometime. And maybe you're right, I am just (Moby) Dick-whipped, but let me work through it on my own, please?

Fully Disjunctive Reason This,

Esteban

[Nebuchadnezzar · Rating 4 out of 5]

What is "intelligence"? How does reasoning go wrong? Why do smart people do the dumbest things? How did Dubya get such a high IQ?

Keith Stanovich has the answers in what is the most satisfying work of IQ revisionism I've read. While I agree with many of the criticisms of the psychometric construct of general intelligence (or "g") made by critics such as Howard Gardner and Robert Sternberg, I always found their alternative models of multiple intelligences to be too reliant on loose semantics and circular definitions. Then there is Daniel Goleman's concept of "emotional intelligence," which suffers the same problem in addition to the research literature on it being riddled with methodological flaws. Stanovich makes much the same criticisms and then some.

Stanovich argues that these critics are actually undercutting their own goals by inflating the importance of the construct of "intelligence" by appending it to their ideas. He identifies IQ testing as measuring a construct he calls "MAMBIT," which stands for (wait for it...) "Mental Abilities Measured By Intelligence Tests." As he notes, this may sound like E.G. Boring's dictum that intelligence is "whatever IQ tests measure," but this is not so. Unlike those who would dismiss intelligence research out of hand, he shows that IQ measurements do relate well to working memory capacity and certain abstract verbal, logical, and mathematical skills. (Academics design a test to measure faculties they consider important -- quelle surprise!) However, as other IQ critics have noted, these are only a small subset of human mental faculties. Contra other IQ critics, though, Stanovich sticks with a narrow definition of "intelligence" as MAMBIT.

The differentiation between intelligence and rationality is Stanovich's core argument. He draws heavily on the research program into cognitive biases and rationality initiated by Daniel Kahneman and Amos Tversky to make his case and does so convincingly. He illustrates how we are all prone to act like "cognitive misers," putting as little critical thinking into most of our decisions as possible.

Ultimately, Stanovich demonstrates that even critics of IQ testing have inflated the importance of the faculty of intelligence. This is an indictment not on the very existence of IQ tests, but on their abuse and misuse. Anyone with an interest in intelligence, rationality, and IQ testing has no excuse not to read this book.

[Luc Beaudoin · Rating 5 out of 5]

Great book.

Some of the key concepts are:

* Mindware: The software of the mind (following David Perkins).
* Fluid rationality:
* Thinking dispositions: habits and motivation to make use of one's fluid intelligence.
* Crystalized rationality: Crystallized facilitators and Crystallized inhibitors. Facilitators are reasonable skills and knowledge of rationality (e.g., logic, math). Inhibitors include false beliefs and belief in authority.
* The architecture of the mind in three parts: The reflective layer, the algorithmic layer (what IQ tests measure) and the autonomous layer. (He calls each layer "minds").
* Cognitive miserliness: being too lazy to adequately engage one's mindware. (Stanovich Claims George Bush was smart, but cognitively lazy.)

Stanovich claims that smart people do dumb things either because they lack mindware or the thinking dispositions to make use of their algorithmic layer.

He claims the concept of rationality is more global than intelligence: It encompasses two things (thinking dispositions of the reflective mind and algorithmic-level efficiency (fluid intelligence).

I use a lot of these ideas in my own book, Cognitive Productivity: Using Knowledge to Become Profoundly Effective. However, our model of mind is augmented with "motive generators", attention filters and other mechanisms. Also, I focus on productive practice. Whereas Stanovich's book characterizes rationality in general, I focus on the more specific problem of using knowledge. A major source of irrationality is superficially knowing something but not realizing that it is applicable. This is not merely due to laziness. It's also due to not having the right systems and software to master knowledge gems (such as the one in Keith Stanovich's excellent book).

[Maher Razouk · Rating 4 out of 5]

الذكاء والتفكير العقلاني!
.
.
هناك اتفاق كبير على أن تفكير الرئيس (جورج بوش) تعيبه عدة جوانب إشكالية: الافتقار إلى المشاركة الفكرية ، وعدم المرونة الإدراكية ، والإصرار على الاعتقاد ، والتحيز التأكيدي ، والثقة الزائدة ، وعدم الحساسية للتناقض.
هذه كلها خصائص معرفية تمت دراستها من قبل علماء النفس ويمكن قياسها على الأقل ببعض الدقة. ومع ذلك ، فهي كلها أمثلة على أساليب التفكير التي لم يتم استغلالها في اختبارات الذكاء. وبالتالي ، فليس من المستغرب أن يعاني شخص ما من العديد من هذه النواقص المعرفية ولا يزال لديه معدل ذكاء مرتفع بشكل معتدل.

إن أوجه القصور الإدراكي لدى بوش لا تؤثر على الأداء في اختبارات الذكاء ، لكنها تؤثر على عملية اتخاذ القرار العقلاني. وبدلاً من ذلك ، فإن عيوبه المعرفية هي أسباب "عسر القراءة" ، وهو مصطلح صُغته في منتصف التسعينيات من أجل لفت الانتباه إلى ما هو مفقود في اختبارات الذكاء.
أعرّف الخلل العقلي بأنه عدم القدرة على التفكير والتصرف بعقلانية على الرغم من امتلاك الذكاء الكافي. في الواقع ، الرئيس ليس غبيًا ، لكنه قد يكون يعاني من اضطراب عقلي.
و هو ليس بمفرده في هذا . يظهر الكثير من الناس عدم القدرة المنهجية على التفكير أو التصرف بعقلانية على الرغم من حقيقة أن لديهم أكثر من معدل ذكاء كافٍ. أحد الأسباب التي تجعل الكثير منا يعاني من خلل العقل إلى حد ما هو أننا ، لأسباب متنوعة ، وصلنا إلى المبالغة في تقدير أنواع مهارات التفكير التي تقيسها اختبارات معدل الذكاء والتقليل من شأن المهارات المعرفية الأخرى ذات الأهمية الحاسمة ، مثل القدرة على التفكير بعقلانية.
على الرغم من أن معظم الناس قد يقولون إن القدرة على التفكير بعقلانية هي علامة واضحة على تفوق الذكاء ، إلا أن اختبارات معدل الذكاء القياسية لا تخصص أي قسم للتفكير العقلاني .

التفكير بعقلانية يعني تبني أهداف مناسبة ، واتخاذ الإجراء المناسب وفقًا لأهداف الفرد ، والتمسك بمعتقدات تتناسب مع الأدلة . على الرغم من أن اختبارات معدل الذكاء تقيّم القدرة على التركيز على هدف فوري في مواجهة الإلهاء ، إلا أنها لا تقيّم على الإطلاق ما إذا كان الشخص لديه ميل لتطوير أهداف منطقية في المقام الأول. وبالمثل ، تعد اختبارات معدل الذكاء مقاييس جيدة لمدى قدرة الشخص على الاحتفاظ بمعتقدات في الذاكرة قصيرة المدى والتلاعب بهذه المعتقدات ، لكنها لا تقيّم على الإطلاق ما إذا كان الشخص لديه ميل لتكوين معتقدات عقلانية عند إطلاعه على الأدلة. ومرة أخرى ، وبالمثل ، تعد اختبارات معدل الذكاء مقاييس جيدة لمدى كفاءة الشخص في معالجة المعلومات التي تم توفيرها ، لكنها لا تقيّم على الإطلاق ما إذا كان الشخص هو فعلاً مقيِّم نقدي للمعلومات كما يتم جمعها في البيئة الطبيعية.
.
Keith E. Stanovich
What Intelligence Tests Miss
Translated By #Maher_Razouk

[Michael Hirsch · Rating 4 out of 5]

this book convincingly shows that there is a big difference between intelligence and rationality. intelligence tests like the SAT test whether the subject can successfully manipulate symbols in a test taking situation, but makes no attempt to answer the more interesting question as to whether the subject would be likely to make good decisions in real life. It turns out that rationality is, at best, only slightly correlated to intelligence.

The author does a good job of making the case, but in a not very approachable style. He users too many big words in ways that are not common in everyday English. He also has an axe to grind that gets in the easy of writing an educational book.

For a much better book on a similar topic, read Daniel Kahneman's wonderful book "Thinking Fast and Slow"

[Shea Levy]

Gave up mid-way through chapter 6. Seemed to just be "Intelligence is what's measured by IQ but it's not everything, there's also rationality" over and over and over, plus stuff you can get from Kahneman, and Stanovich's style is pretty mind-numbing. I was also really skeptical of the certainty with which he expressed his model, especially identifying (fluid) g with cognitive decoupling. Overall, did not come out of this impressed with Stanovich at all and I have no idea what he wants me to get out of this (beyond, again, that there's more to minds than intelligence and we can measure rationality).

[Curtis Gagliardi · Rating 4 out of 5]

A little repetitive, particularly in the beginning, but once it gets into all the individual cognitive biases it's fascinating. I was surprised how often I got the example problems wrong even knowing I was reading a book about cognitive biases. I got this book from the Center For Applied Rationality's reading list http://rationality.org/reading/, I'll probably go back and finish LessWrong's core sequences now: http://wiki.lesswrong.com/wiki/Sequences.

[Massimo Redaelli · Rating 3 out of 5]

Why, oh lord, why, does every popular science book repeat over and over again the same two concepts, filling well over 200 pages with slightly different wordings of the same thing?

It happens here, too: the (interesting) stuff could have been comfily put in 50/70 pages, making it more readable.

Anyway, it seemed to me a good - although somewhat vague - introduction.

[Ian · Rating 5 out of 5]

In What Intelligence Tests Miss, Keith Stanovich provides an important and fascinating analysis of intelligence and rationality—what they mean and how they relate to each other. He explores how IQ tests of various types are overvalued, and how the lack of a similar test on rationality hurts society in significant ways. He does so using understandable diagrams, interactive examples and accessible complexity. This makes for an engaging and applicable experience that can provide insight not only into the societal view of these subjects but your own rationality as well.
As someone with little to no prior knowledge of psychology, What Intelligence Tests Miss was mostly understandable. The first chapter or so was relatively difficult, but with focus and by reviewing I could gain an understanding. The most fascinating stretch of the book for me came after that first introduction, during which Stanovich goes into detail about the specific downfalls of “cognitive misers”, and why humans evolved to be that way. I enjoyed learning about how evolution developed such flawed predispositions, which he explains is a function of the fact that natural selection is not based on truth, but rather on survival and reproduction. For this reason humans in the modern world are plagued with logical fallacies and biases, preventing us from understanding everyday probabilities and logical concepts, and keeping us from utilizing our “algorithmic” minds (intelligence) to their full extent. Such insights as these are plentiful in Stanovich’s book.
The details of What Intelligence Tests Miss are also extremely applicable to life, especially in the contexts of belief, politics, awareness of manipulation (advertising), and decision-making. For example, Stanovich outlines various “mindwares” (conceptual knowledge bases) that are critical to various aspects of rationality. He demonstrates them using examples given to the reader, and then explains the most common answers and what causes them to be wrong. One such mindware is the tendency to examine alternative hypotheses when considering a claim. If a restaurant owner tells you that “95% of my customers don’t complain”, you might deduce a certain expectation of quality. You might put the probability that the food is good at 80% given that 5% of people complained. But by considering the probability that an alternate hypothesis is true—that the food is bad given that 5% of people complained—might give insight into another perspective. Even if the food is bad, most people are passive in nature. Is a 5% complaint rate actually low? This is a relatively simple example of mindware, but it’s useful to know. It demonstrates that understanding is often a matter of looking at things from the right angle—even when that angle may seem obvious when presented.
I recommend What Intelligence Tests Miss to anyone interested in the phenomenon of “smart people doing dumb things”. If you are ever surprised by the disconnect between perceived intelligence and actual action, the insights Stanovich provides are beyond fascinating. You don’t need to have a background in psychology to understand this book, and you may even find it’s more applicable to your own life than expected.

[Zhijing Jin · Rating 5 out of 5]

Motivation: Smart people (by IQ) can have logical fallacies easily.
Overview: Let's put three books in a row.
(1) Thinking, Fast and Slow
(2) What Intelligent Tests Miss: The Psychology of Rational Thought
(3) How Not to Be Wrong: The Power of Mathematical Thinking

Book (1) introduces the psychology discovery of two thinking patterns: system 1 which deals with intuitive tasks, such as recognizing people's faces, and system 2 which deals with deeper thinking such as arithmetic calculation.

Book (2) introduces that people good at the system 2 mentioned above can be classified into (a) algorithm-wise smart, but weak reasoning, and (b) algorithm-wise smart, and strong reasoning. For example, people who are good at math, or who have published lots of papers in science, can be considered as algorithm-wise smart. However, we found that these people can make silly logical fallacies, such as hard to even identify contradiction in real-life events, or failing at "if p then q <=> if not q then p" deduction.

This book suggests "Rational Thought" tests. I strongly support this training and examination.

Note that (a) is reasonable because people tend to save mental effort when thinking, so system 1 > system 2(a) > system 2(b) in terms of frequency of use.

Book (3) directly talks about several most useful logical reasoning skills that we can use. See my summary here.

Critical Thinking: My own theory for people who are algorithm-wise smart, but lack reasoning skills is that their islands of intelligences in mind (e.g., mathematical intelligence can be one island, EQ can be another island) are disconnected. So they could not generalize their theories from one island to another. The intelligence they achieve on one island cannot be transferred out. Admittedly, these people cannot be considered as smart in a strict way.

[YHC · Rating 3 out of 5]

If you have read Thinking, Fast and Slow then this book doesn't seem to be so difficult to understand.
First of all, our IQ test is basically : General intelligence + Fluid intelligence. But this is crystallized
intelligence as well. ("Crystalized rationality: Crystallized facilitators and Crystallized inhibitors. Facilitators are reasonable skills and knowledge of rationality (e.g., logic, math). Inhibitors include false beliefs and belief in authority." ) So far there is no test for rationality, it can be learned. Rationality is a cognitive skill and differs from emotional intelligence and social intelligence.

Extending Daniel Kahneman's 2 methods, Stanovich introduce third one (Reflective Mind and Algorithmic Mind:what IQ tests measure and autonomous Mind). The interaction among these 3 was shown as images in the book.

He mentioned a lot about cognitive miser: too lazy to engage one's mindware. ( he used George Bush as example.), making a choice based upon the emotions, accepting what is given framing as possible answer, not searching or thinking over, avoiding the exhaustive enumeration.

He also mentioned about:
1. Mindware Gap: no critical thinking, not considering alternative possibilities, fail to understand independent probabilities.
2. Contaminated Mindware: such as Ponzi scheme, parasitic meme.

We can see this kind of High IQ but low RQ or EQ people ruling our world ...sadly.

[Matthew Leslie · Rating 5 out of 5]

How does everyone not talk about this book? Pours so much of society into perspective. IQ is like computer hardware, but rational thinking is like software. What good does a speedy computer do if it runs garbage programs?

The first few chapters might scare away more casual readers, but I would encourage sticking with the book. The later chapters focus on more specific examples of how the mind can fail to achieve rational thought processes. I fell for several of the diagnostic questions, but was delighted to learn what my mistake had been. The odds form of Bayes’s formula was particularly enlightening. Finally, the chapter on faulty mindware has a grim relevance to life in 2023, with news of anti-vaxxers and MAGA morons afflicting our daily lives.

If only our society would prioritize teaching rationality as highly as other ideals. Maybe we’d have a more utopian society for our progeny.

[Sumeet Kapoor · Rating 4 out of 5]

Pros:
The concepts of rationality covered in this book, if applied in real life (especially in investing), could assist you in making very good decisions. The concepts could also be used to understand where we may have gone wrong in our past decisions.

Cons:
The author used language that was very difficult to follow. And as the title suggests, there was a lot of energy spent in proving that intelligence tests miss to test the rationality of humans but the book does not offer any solution to correct the problem.

[Hairuo · Rating 5 out of 5]

The author talked about the importance of rationality and how it is ignored and could not be concluded in traditional intelligence tests.

To understand what the author talks about, I highly recommend you preparing to finish reading the author's previous book " How to think straight about psychology" which clarify what falsifiability is. and book "The selfish gene" by Dawkins to understand the essence of evolution well.

[Lawrence Chen · Rating 4 out of 5]

It is funny to find a joke about my professor as I was a Freshman in the fox school of Temple. Prof. Paulos for Math. Lol. Generally, good book. But not that good compared to Richard Thaler's Behavioral Economics research books I read earlier this year. Recommend write down some notes as you read it, it is not a simple book to understand. You need to think and you need a draft.

[Matt Danner · Rating 4 out of 5]

I read this as background for a thesis. It's very well researched and documented and the author does a fine job of making his argument. For the layperson, though, there are a lot of terms that are new and challenging to understand (technical). Readers with at least some familiarity in behavioral science will fare better.

[Carl Stevens · Rating 4 out of 5]

Novelists need to know rational thought well and dysrationalia even better.

[Heathyr · Rating 5 out of 5]

Excellent.

[Marika Rostvall · Rating 4 out of 5]

An interesting book about the concept of rationality and it's importance. I liked it, but the book loses some points for being somewhat repetitive.

[Vidur Kapur · Rating 5 out of 5]

Unlike most detractors of IQ testing, Stanovich does not subject such testing to blatantly false criticisms ("everybody is intelligent in their own way", "IQ tests aren't important in the real world). Rather, he states that while IQ is important to a certain extent and is a reliable and accurate measure of intelligence, there is far too much emphasis placed on intelligence and not enough placed on rationality, and goes onto show how those with higher IQs will tend to be more rational when prompted to do so, but engage in rational thinking at the same rate as those with lower IQs spontaneously.

The book also has a lot of interesting rationality puzzles, some of which also come up in Daniel Kahneman's excellent book Thinking, Fast and Slow.

[Eleanor · Rating 3 out of 5]

loved the content and think i will actually get a copy at some point because I would want to re-read parts of it (my thesis was related to heuristics so I was really interested in the subject). but it took me so long to read/was such a slog to get through. well written and clear and very readable at the sentence level... but so repetitive, lots of summarizing what had been said previously (without adding to it--just repeating). if there were a long article available by the same author, I would say read that instead.

[Ben · Rating 5 out of 5]

This is a book that changed my mind.

It's very common for intelligent people (and intelligent people interested in rationality) to associate intelligence with rationality. "Look how smart people are who are interested in this! Surely, intelligence must correlate with rationality!"

Interestingly, it doesn't.

This book was a good summary of some of the basic ways people make bad decisions (cognitive miserlyness and "lacking mindware").

[BookSweetie · Rating 2 out of 5]

The key point faults current thinking about intelligence as being unduly influenced by what is tested on IQ tests rather than incorporating ways of expanding our ideas to include rational thinking. Tests could be devised that would assess this key aspect of our thinking resulting in a fuller way of thinking about thinking! Somewhat technical.

[AJ · Rating 3 out of 5]

I didn't make it through this one, though I was intrigued by the first several chapters.

[Frank]

Not written in a very readable style

[Veronica · Rating 3 out of 5]

Explained why smart people make stupid mistakes, use a diagram to depicts how those irrational mistakes generated.

[Jack Zhao]

The writing is too dry. Don't want to finish.

[Daniel Frank · Rating 4 out of 5]

I wish this book was written for the popular audience, because it discusses such an important idea that nobody else is talking about. The book is fascinating and salient.

[Melissa · Rating 2 out of 5]

Read for one of my psych courses and YIKES what a dry read. Honestly, this book caused me so much grief.