By Sigurd Arild, Ryan Smith, and Eva Gregersen
Regular readers know the drill by now: We have no affiliation with the MBTI; we just don’t like seeing lazy and uninformed critiques misguiding the public. The latest example of such a critique is the journalist Joseph Stromberg’s article: Why the Myers-Briggs test is totally meaningless, as featured in Vox.com. The article is basically non-news, lazily re-hashing older articles that we have previously debunked on the site. But what’s worse, the article is also factually incorrect in several places. Let’s go over Stromberg’s points:
1
“The test is completely meaningless. … ‘There’s just no evidence behind it,’ says Adam Grant, an organizational psychologist at the University of Pennsylvania … ‘The characteristics measured by the test have almost no predictive power on how happy you’ll be in a situation, how you’ll perform at your job, or how happy you’ll be in your marriage.’”
If the test is “completely meaningless,” why does it have “some” predictive power? It can’t be both.
Adam Grant should not be treated as a credible source. As we have previously documented, Grant quotes unfaithfully from the scientific source material that he bases his argument on. He says that the test is “completely meaningless,” but the scientific study that Grant himself has referenced concludes that: “… the four MBTI indices did measure aspects of four of the five major dimensions of normal personality…” and that “even critical reviewers … see promise in the instrument.”
It is true that the MBTI has little predictive power with regards to happiness in marriage or job performance, but the MBTI doesn’t claim to measure those things, so why should that be a problem?
2
“Most of the faithful think of it primarily as a tool for telling you your proper career choice.”
It is true that some people think that, but there is no basis for claiming that “most” of the people who are sympathetic to the MBTI think that it can meaningfully determine which careers you will thrive in. Some people also think that microwave ovens can be used to dry their pets, but that doesn’t really reflect badly on microwave ovens.
3
“But the test was developed in the 1940s based off the untested theories of an outdated analytical psychologist named Carl Jung, and is now thoroughly disregarded by the psychology community.”
Jung’s theories weren’t out-dated in the 1940s; on the contrary, the Allies used him as an intelligence agent, crafting psychological reports on Hitler, which were read by Eisenhower, among others. Also the matter of Jung’s relation to mainstream psychology is not as black and white as the writer lets on. More about that later.
4
“Several analyses have shown the test is totally ineffective at predicting people’s success in various jobs, and that about half of the people who take it twice get different results each time.”
It is true that the test is ineffective at predicting job performance, but again, the MBTI itself says that it should not be used to determine job performance, so the MBTI is like a microwave with a big yellow warning sticker that says: “WARNING – SHOULD NOT BE USED TO DRY PETS,” but which nevertheless attracts scores of detractors who voice their offense at the fact that the microwave isn’t a suitable way of drying your pet.
5
“In 1921, Jung published the book Psychological Types. In it, he put forth a few different interesting, unsupported theories on how the human brain operates.”
Not brain – psyche. Anachronisms are unseemly. It was important to Jung that his theories were not materialistic or Aristotelian, which is why he avoided giving them a physiological basis.
6
“Among other things, he explained [in Psychological Types] that humans roughly fall into two main types: perceivers and judgers.”
This is not correct. Judging and Perceiving are Myers’ and Briggs’ inventions, based only loosely on Jung’s original concepts of rational and irrational. If you are going to report on the contents of a book, do yourself the trouble of reading it.
7
“[The theory of types] didn’t come out of controlled experiments or data. ‘This was before psychology was an empirical science,’ says Grant, the Penn psychologist. ‘Jung literally made these up based on his own experiences.’ … None of this came out of controlled experiments or data – it was all theoretical.”
Neither Darwin’s theory of evolution nor Einstein’s theory of relativity came out of “controlled experiments or data.” Like Jung’s theory of types, they were initially made up and later confirmed to some degree by experiments. In the same way, Jung’s theory of the elements that constitute the personality have also been confirmed to some degree by later experiments. It seems disingenuous not to inform the reader that the basic gist of Jung’s postulates have been confirmed in experiments to some degree.
8
“Jung’s principles were later adapted into a test by Katherine Briggs and her daughter Isabel Briggs Myers, a pair of Americans who had no formal training in psychology.”
Myers and Briggs were not psychologists, but they nevertheless succeeded in doing what academically certified people in their own time failed to do. If someone runs a marathon and finishes ahead of the pros, should he then be stripped of his victory because he isn’t a professional runner?
9
“Myers and Briggs gave titles to each of these types, like the Executive, the Caregiver, the Scientist, and the Idealist.”
This is factually incorrect. The writer is confusing the official MBTI material with fan-generated material that he has found online. If a Richard Dawkins fan believes in astrology, does that constitute a sound basis for claiming that Richard Dawkins believes in astrology? The logic seems to be limping here.
10
“But the fact is that [the indices measured by the MBTI] come from the now-disregarded theories of a early 20th century thinker who believed in things like ESP and the collective unconscious.”
Actually, the two areas of Jung’s theories that are widely regarded as being non-mystical and compatible with empirical science are his studies in word association and his theory of types. It seems disingenuous not to inform the reader of this.
11
“Actual data tells psychologists that these traits do not have a bimodal distribution. Tracking a group of people’s interactions with others, for instance, shows that as Jung noted, there aren’t really pure extroverts and introverts, but mostly people who fall somewhere in between.”
This point will be dealt with in conjunction with point #15 below.
12
“All four of the categories in the Myers-Briggs suffer from these kinds of problems, and psychologists say they aren’t an effective way of distinguishing between different personality types. ‘Contemporary social scientists are rarely studying things like whether you make decisions based on feelings or rational calculus — because all of use both of these,’ Grant says. “
It doesn’t seem like the writer has any sources for his claim that “psychologists” are of this opinion, and Adam Grant’s assertion is incorrect. Grant hasn’t done even the most basic research on the MBTI and mistakenly thinks that the MBTI’s “Feeling” category is about actual feelings. But the very first hit on Google for “MBTI thinking feeling” takes you to the official Myers & Briggs Foundation and tells you that it is not.
Again, we must urge journalists not to use Grant as a serious source. The fact that Grant misconstrues scientific source material to make his point should be a huge disqualifier in and of itself. Grant is basically someone with a fancy academic title who keeps insisting that microwaves should be able to dry pets.
13
“This is why some psychologists have shifted from talking about personality traits to personality states — and why it’s extremely hard to find a real psychologist anywhere who uses the Myers-Briggs with patients.”
This point seems like a non sequitur. The MBTI doesn’t purport to measure any kind of psychopathology, so why should clinical psychologists use it? On the other hand, it is easy to find organizational psychologists who use it.
Also, one might get the idea from the article that the very practice of testing static personality “traits” is on the retreat. The state of the art in personality testing is still the Big Five test and so the practice of measuring personality in static terms rather than fluid ones is not at all on the retreat.
14
“There’s also another related problem with these limited choices: look at the chart above, and you’ll notice that words like ‘selfish,’ ‘lazy,’ or ‘mean’ don’t appear anywhere. No matter what type you’re assigned, you get a flattering description of yourself as a ‘thinker,’ ‘performer,’ or ‘nurturer.’”
Again, the writer is confusing the official MBTI material with the fan-generated material found online. But disregarding that, it is generally true that the MBTI has a positive bias. We have dealt with that criticism here. Yes, the MBTI is generally too positive, but one reason for the positivity was that it was developed to be used in organizations where the practitioners had to get people to accept their results. The Big Five was developed to be used by researchers and did not need to win the acceptance of the test takers.
Now that psychologists are seeking to use the Big Five in organizations, they have found it necessary to adopt a positive bias which is akin to the one used in the MBTI, and even to postulate personality types, which is one of the things that the adherents of the Big Five started out criticizing the MBTI for. The process can be compared to a child who criticizes his parents for the slights in their parenting. The child vows to himself that when he grows up and becomes a parent, his parenting will be much better. But then, when he actually is a parent, and stands there with the crying children, he finds himself repeating many of the “mistakes” that his own parents made.
Additionally, if people want negative type descriptions, they need to look no further than Jung’s Psychological Types. Since the writer has ostensibly read that book (see point #6 above), it seems like an oversight that he doesn’t mention these non-positive type descriptions to his readers.
15
Returning to the criticism cited in point #11 above: “Actual data tells psychologists that these traits do not have a bimodal distribution. Tracking a group of people’s interactions with others, for instance, shows that as Jung noted, there aren’t really pure extroverts and introverts, but mostly people who fall somewhere in between.”
The criticism ties into another observation furnished by the writer, namely that: “Research has found that as much as 50 percent of people arrive at a different result the second time they take a test, even if it’s just five weeks later.”
This criticism is true, as we have examined here. It is the single biggest problem with the MBTI.
<Stromberg’s article then continues for some time, recycling the points he has already made. We avoid insulting the reader’s intelligence by repeating the rebuttals.>
16
Stromberg then writes:“Apart from the introversion/extroversion aspect of the Myers-Briggs, the newer, empirically driven tests focus on entirely different categories. The Five Factor model measures people’s openness, conscientiousness, extraversion, agreeableness, and neuroticism — factors that do differ widely among people, data has told us.”
- The writer seems to be saying that the scales measured by the MBTI are normally distributed, while the scales measured by the Big Five are bi-modally distributed. In reality, both the scales measured by the MBTI and the scales measured by the Big Five are normally, rather than bi-modally, distributed.
- The writer says that apart from the E/I dimension, the other dimensions of the Big Five are entirely different from the MBTI. But in fact, Sensation-Intuition on the MBTI is related to the Big Five’s concept of Openness, Thinking-Feeling on the MBTI is related to the Big Five’s concept of Agreeableness, and Judging-Perceiving on the MBTI is related to the Big Five conception of Conscientiousness. While there are differences in overall methodology (the MBTI is Platonic, while the Big Five is Aristotelian), the basic gist of the four dimensions mentioned above is the same.
Ironically, the people who fault Jung for just making up the scales tend to be oblivious to the fact that modern science has largely ended up confirming the scales he came up with. The criticism is like faulting Darwin for only having had a vague idea of evolution without being able to work out the genealogy of every species in detail. According to that manner of measuring results, Darwin should actually be criticized for postulating a theory that was largely right, because he didn’t have all the details back in 1859. Later scientists, who then confirmed his theory in the lab using data, should then snub their noses at Darwin and claim exclusive credit for the theory of evolution.
17
Stromberg then writes: “The Myers-Briggs is useful for one thing: entertainment. There’s absolutely nothing wrong with taking the test as a fun, interesting activity, like a BuzzFeed quiz.”
So apparently a test which has been demonstrated to measure actual elements of the personality to an acceptable degree is no better than a test which has no validity to back it up. It seems a bit hasty on Stromberg’s part to say that because something is somewhere in the middle of the road in its field, then it’s no better than something at the very bottom. This is somewhat like saying that because silver isn’t gold, then silver is really worthless. It seems that Stromberg is overstating his case at the end. The MBTI has known issues and has had so for years. But the MBTI has never been found to be devoid of validity, as the writer insinuates, or to be “totally meaningless” as the writer’s headline says.
18
Stromberg then critiques the pricing policies and certification process of the MBTI (a criticism that we agree with), but then continues to state that: “Because once you [are certified] you can sell your services as a career coach to both people looking for work and the thousands of major companies … Once certified, test administrators become cheerleaders of the Myers-Briggs, ensuring that use of the outdated instrument is continued.”
As mentioned, we agree with the point that there are problems regarding the pricing and certification process that surrounds the official use of the MBTI instrument. But how does Stromberg know that certified MBTI practitioners are “cheerleaders” for the MBTI? One of the authors of this article was first typed by a certified psychologist and MBTI practitioner who introduced the instrument by saying that she didn’t see the value of it. It seems that Stromberg is presuming to know the motives of thousands of people without having anything solid to base his claims on. Didn’t he just say that it was important to base your claims on data?
Finally, Stromberg ends with an injuction: “Let’s stop using this outdated measure — which has about as much scientific validity as your astrological sign — and move on to something else.”
It is really tiresome that writers who set out to criticize the MBTI can’t be bothered to acquaint themselves with the most basic facts concerning the instrument. As we covered on the site long ago, almost every scientific study ever conducted has concluded that there is some validity to the MBTI, while there is no validity to astrology. Hence there is really no basis at all for claiming that their validity is “about the same” and it is an incontrovertible mistake on the writer’s part to suggest otherwise.
Article Series: CelebrityTypes Debunks Bad MBTI Criticisms:
- Drake Baer’s Lazy Critique of the MBTI [Business Insider]
- Why Adam Grant’s Critique of the MBTI is Useless [Business Insider]
- MBTI for Skeptics [General criticisms of the MBTI answered]
***
Myers-Briggs Type Indicator and MBTI are trademarks of the MBTI Trust, Inc.
CelebrityTypes.com is an independent research venture, which has no affiliation with the MBTI Trust, Inc.