Critical Thinking Tests and the Limitations of Statistical Reasoning

Many businesses and educational institutions use famous critical reasoning tests to appraise employees’ or candidates’ critical thinking skills.

Compared to our own concept of how critical thinking skills are actually used in the many complicated situations we face in real life, the situations described in these tests are usually somewhat limited in scope, and the number of possible answers is so limited that many candidates could score correctly from a lucky guess.

Most critical thinking tests assume there is only one correct answer and all the other options are plain wrong. When the situation can be reduced to mathematics then there often is just one correct answer. But our own approach is to acknowledge that in most circumstances the correct answer for one person is not necessarily the correct answer for another person, as each person has different aims and most of our fundamental decisions in life are not simply a question of mathematical reasoning. If only they all were!

These formal thinking tests can however be very useful in sharpening one’s logical skills – a central part of our arsenal for dealing with decision-making! But we must remember that many decisions in life cannot be boiled down to a small number of options of which there is only one correct answer.

Here is some information about two of the most popular tests, both widely regarded as valid, yet clearly limited in scope and sometimes even confusing in the choice of language used within the questions.

Watson-Glaser Critical Thinking Appraisal (WGCTA)

The most famous critical thinking test of all, the Watson-Glaser contains 80 multiple-choice questions over 5 sections and is usually completed in an hour or less. Successful candidates can infer, recognize assumptions, evaluation whether an argument is strong or weak, deduce, and intepret logically. It was first introduced as a test of fair-mindedness back in 1937 and has undergone several updates since including internationalisation.

The first section concerns inferences which are conclusions drawn from observed or supposed facts. Inferences can be correct or incorrect. For example, if your phone does not switch on you may infer that the battery has been used up, but that is not necessarily the case as there are other possible explanations such as that your power button is faulty. In the test, the answer options regarding inferences in given situations are true, probably true, more information required, probably false, and false.

The second section is concerned with assumptions which are things presupposed or taken for granted. The only answer options in the test are assumption made or assumption not made. After reading the statements, one must decide if various assumptions are logically justified or not. For example, if a friend says he will see you at 7 o’clock this evening, the assumption is made that you will indeed meet later.

Section three is all about deductions, and whether or not a particular conclusion follows logically. Section four is about interpreting information and the answer options are identical to the previous section, with the conclusion either following logically or not.

And the final section is concerned with the analysis of arguments with the evaluation options of either strong or weak argument. This final section can be easily misunderstood, with the question stating that a strong argument is one which is both important and directly related to the question, though what that exactly means is unclear and potentially confusing.

Advanced Numerical Reasoning Appraisal (ANRA)

Published by Pearson, the ANRA (RANRA in the UK) is a companion test to the Watson-Glaser Critical Thinking Appraisal and aims to assess how well a person can recognize, understand and apply logical skills in mathematical or statistical contexts. It is much more than a mathematical competency test. It measures how well someone can deduce, interpret and evaluate based on two interdependent tests – comparison of quantities based on comparative data, and sufficiency of information based on presented statements.

Scoring is automatic, which well sums up the limitations of the test and how the skills it focuses on assessing will not alone necessarily help a person make a good decision for themselves or solve a complex problem effectively in the real world.

The main issue with both of these tests, and many others like them, is that they do not encompass a large enough percentage of complex real life situations in which we need to draw on creative thinking methods to consider different perspectives and make decisions to the best of our ability.

Should Person A accept a job offer in South Africa? Should Person B get divorced? There is not always one ‘correct answer’ like on a multiple-choice test, or at least not without the tool of hindsight. The vast majority of life decisions we face involve many layers of inductive reasoning in addition to deductive reasoning, and what might be a good answer for Person A could make Person B incredibly unhappy.

In addition, we almost always lack sufficient information of one sort or another, information which can simply be inaccessible to us regardless of our skills. Yet we often still need to make a decision under such circumstances regardless of the uncertainties in order to meet set deadlines.

Despite the limitations, and whilst the skills assessed in these tests are effectively blunt tools for many real-life situations, they do offer the opportunity for good practice of some of the core elements needed for successful critical thinking, decision-making and problem-solving.