IQ Tests - What's the Deal?

"Yes, James. I get it...'Hi, Q'...you're freaking hilarious."

The Book That Never Was
    Over the years I've hopped back and forth on the issue of IQ tests.  They get used in school admissions, job applications, criminal trials, military recruitment and performance management cycles but there seems to be great disagreement over whether they're actually telling us anything.  The problem is that so much has been written about them (for and against) that the waters of debate are pretty muddy and it's not easy to know what the deal is. 
    Some books condemn the whole concept and argue that IQ tests only tell us how well people do on IQ tests (The Mismeasure of Man by Stephen J Gould, IQ: A Smart History of a Failed Idea by Stephen Murdoch, The Sun Shines Bright by Isaac Asimov) while others argue that IQ tests are useful but get nervously dismissed because they raise uncomfortable results people would rather ignore (What Is Intelligence? by James Flynn, Looking Down on Human Intelligence by Ian J Deary, The Neuroscience of Intelligence by Richard J Haier). 
    A while back, I decided I wanted to write a book of my own on the topic and present readers with a non-biased summary of what IQ research has actually uncovered.  I approached my publisher with an idea for a book called The Science of Clever and although they were interested, it soon became obvious to me that it was going to end up a mess.  The consensus on IQ testing is not cut and dried and you can't spend 50,000 words shrugging your shoulders and saying "Ummm...I dunno".
    The study of intelligence research is called "psychometrics" (which sounds like a Transformers villain) and although I couldn't quite turn the whole field into a single light-hearted book, I still ended up with a bunch of interesting research, not to mention a half-finished manuscript on my hard-drive.  So, I figured I might as well summarise the surprising highlights in my first blog of 2020.  I doubt The Science of Clever will happen, but if it does, you're getting the first couple of chapters for free.  Merry Christmas guys!

"Ah, mon capitaine...if you're looking for something to say, might I suggest...'Hi, Q'?"

What Is An IQ Score?
    First thing's first: there is no such thing as a standard or internationally recognised IQ score.  IQ tests are actually products sold by private companies and there are dozens of brands available, none of which line up.  For instance an IQ of 130 on The Culture Fair IQ Test is equivalent to a score of 150 on The Cattell III-B IQ Test and so on.  If someone claims to have an IQ of whatever that doesn't actually mean anything unless they specify which company's test they scored it on. 
    The two most commonly used versions are the Wechsler Adult Intelligence Scale Version 4 (WAIS-IV) sold by Pearson Assessment for $980, and the Raven's Progressive Matrices Test sold by Hogrefe Ltd for $200, although you can't just purchase the tests from amazon as a private citizen.  They are only sold to professional psychologists who have to agree to keep the materials under lock and key and never publish the contents.
    From a cynical point of view, keeping the tests secret allows these businesses to continue making profit because a psychologist can't just download them from the internet.  From a scientific point of view, however, keeping the contents secret means the results are more useful because subjects can't cheat by looking up questions.
    The WAIS-IV takes about two hours and is administered individually (in private) with a psychologist holding a stop-watch, timing you on questions they ask.  The Raven's test is a booklet of pattern-recognition questions you fill in against the clock like any exam.  Both tests arrange questions in order of increasing difficulty and the further you get, the more value the questions have.
    Your score is then ranked against other people and expressed as a comparison of where you come in the overall population.  This is actually quite important because it means an IQ score is not an absolute number - it's a "quotient" i.e. a ranking of where you fit in a group.  A good analogy is to think of it like a race in which you compete against other runners and measure your position rather than your raw speed. 
    How the group gets selected is very important for the results to be reliable.  For instance, if you come third in a race of 100 professional sprinters that's impressive.  If you come third in a race of only three people and the other two were on crutches, that's less so.  The trick to getting a reliable quotient is therefore to make sure you're grouped with people of a similar type.  IQ tests do this by age.  Nothing else.
    You probably assume, like I did, that psychologists have a) a good reason for grouping people by age and b) that the questions on the test are carefully arranged to distinguish easy from difficult.  But you'd be wrong.  IQ tests are unfortunately the result of historical convenience which tends to form the crux of anti-IQ arguments.  They aren't based on any theory whatsoever.

"But I was the original 'Hi, Q!'"

The Worrying Origins of IQ
    The first intelligence test was composed by the French psychologists Alfred Binet and Theodore Simon who asked local schools to provide them with a sample of "normal" boys to see what a child should be able to do.  This is a problem because the baseline of normal was decided by teacher opinion rather than actual data...but that's all Binet and Simon needed; their test was a tool to help schools decide which boys were developmentally behind others.  The problem came when other psychologists took the test and stretched it beyond what it was designed for.
    The American psychologist Henry Goddard got hold of the Binet-Simon test and began administering it to thousands of American children (male and female) without checking to see what they could already do.  Thus, if a child couldn't do what a small group of suburban French boys could, they were deemed below average. 
    After Goddard's test, a physician named Howard Knox wrote an adult version of the test to assess the intelligence of immigrants coming to America via Ellis island.  Knox seems to have composed his questions out of thin air assuming that a "normal" adult should be able to do whatever he personally felt they should. 
    Following on from Knox, the Stanford psychologist Lewis Terman published his own supposedly more sophisticated version in 1916, again making-up questions based on his own personal feelings about what the average person should be able to do.  Unfortunately, Terman was a noted eugenecist and racist so his test questions were based on information common to white middle-class America that black immigrants couldn't possibly be expected to know.  Then, in some astonishing and offensive circular reasoning, Terman pointed to the fact that black immigrants didn't perform well on his test as proof that they worked. 
    Along with his student, Arthur Otis, Terman also introduced the idea of expressing results as a quotient based on the assumption that results should fit a bell-curve for a given age group.  But here's the wacky thing...Terman never actually tested that assumption and nobody has since.  The whole basis of calculating IQ score comes from an idea some guy basically made up!
    The problem we now face is that it's unclear whether IQ test companies have actually rectified the problem because they can't publish data on how the tests are written.  For example, if they published data that showed a certain type of question was harder, people could easily practice that kind of question, score higher on the test and artificially distort the results.  The only way to assess whether the assumptions of IQ tests are correct would be to study them openly...which would immediately invalidate the tests themselves.  A catch-22 of epic proportion!
    So, does this mean IQ tests are useless ?  Well, not quite.  Critics of IQ tests have every right to point out that there are a wealth unfounded assumptions which go into them but, and this is important, it doesn't matter how a theory is arrived at or how sloppy the original scientists were.  If a theory works, it works. 
    Fritz Haber was practically a war criminal but we still use his ammonia reaction to make fertiliser, Alexander Fleming discovered pencilin because he was a clumsy lab chemist who sneezed into a petri dish, but we still prescribe it for infections.  Terman and the rest may have been irresponsible, even downright racist in their methodology, but we shouldn't evaluate IQ tests based on how awful the inventors were.  We have to look at the data.  And, against all the odds, IQ tests actually do sort of work.  A bit.  Sometimes.  Vaguely.

"Hi, cue!"

g-Shock
    By the mid 1930s, the main debate over intelligence was whether there really was such a thing.  According to some psychologists, there was a single characteristic of the brain called g for general intelligence which fed down into all the different types of skill - memory, puzzle solving, pattern-recognition etc.  According to others there was no such thing and people were simply good at different stuff. 
    One of the biggest surprises in the history of psychology came when a young student named David Wechsler settled the debate in another gob-smackingly lazy bit of science which accidentally revealed a valid result.
    Wechsler had studied under many different psychologists who all had their own view of how  to measure intelligence.  Wechsler decided, in 1939, that the best thing to do would be to stick all the tests together and create one super-test.  If g really did exist and intelligence was one thing, people who did well on one test would do well on all the others.  The result was the aforementioned WAIS; a battery of thirteen sub-tests as follows... 

1) Vocabulary and definitions (e.g. what does vulnerable mean?)
2) Similarity spotting (e.g. what is the link between a fork and a knife?)
3) General Knowledge
4) Comprehension (e.g. why do people put food in a fridge?)
5) Spot the missing shape in a series
6) Pattern recreation (you are given coloured blocks and have to recreate images)
7) Picture arrangement (you are given a series of pictures and have to put them in logical order)
8) Matrix puzzles (you are given a grid of shapes which follow a pattern and you need to spot it)
9) Arithmetic questions
10) Repeating sequences of numbers from memory
11) Putting strings of letters and numbers in a given order
12) Code breaking (you have to decipher a message using a table of values)
13) Finding given symbols in a grid of random ones

    Wechsler himself  believed there was no such thing as general intelligence and people simply had talents and preferences.  The idea of a single number to reflect some underlying factor was ridiculous because how could one number reflect your whole ability?  But, to his surprise, it turned out that there really is something there.  People who do well on one WAIS test end up doing well on all of them...by similar amounts.
    The largest survey to investigate the existence of g was carried out in 1997 by The Psychological Corporation who looked at breakdowns of the WAIS-III test, analysing 2450 Americans from 28 different cities ranging in age from 16 to 89.  Given the 13 components in the WAIS-III there are a possible 78 relationships between sub-tests and every single one of them has a positive correlation.  Which shouldn't have happened.  Wechsler's test was cobbled together like Frankenstein's monster and yet somehow yields the inescapable result that there really is some unifying brain-feature which translates to a variety of mental skills. 
    The strength of a correlation can be expressed as a percentage where 0% means "no relationship" and 100% means "perfect relationship".  A value of something like 20% means there is a weak relationship between the two variables while a value of 80% means there is a strong one.  For the WAIS-III the average correlation across the 78 comparisons was a remarkable 50%.  The strongest was 80% (between vocabulary and general knowledge) while the weakest was 30% (between number-sequence tasks and missing shape tasks). 
    A very important caveat is that g does not mean the same as the everyday word "intelligence".  It refers to a currently unidentified cognitive feature and when we talk about intelligence there are other factors such as creativity and wisdom to consider...but g is a part of it.  My personal feeling is that a correlation of 50% is still too vague for a score to be expressed as a single number, so I think an "IQ range" would be more useful, but the fact is g apparently exists and the WAIS IQ Test goes some way toward measuring it. 

"Hi, queue..."

What Does It Tell Us?
    So far all we've shown is that a single IQ score is a moderately reasonable way of measuring something inside the brain.  But does it actually translate to the real world or is it just a number which tells you how well you're likely to do on other tests?  Well, there are a few characteristics which an IQ score does seem to predict.  I've picked a few example studies to show the most agreed-upon findings and although different studies disagree on the strengths of the correlation, the ones I've picked are fairly representative.  It would appear that IQ can partly predict...

School Performance - 2015 study by Frank Spinath analysing over 105,000 students comparing IQ score with school performance and grades.  They found a correlation of 54%.  That's quite an important finding for me as a teacher.  It means that while IQ plays a factor in how well a student does, it's only half the story.  Optimistically, this means if you have a low IQ you are not destined to do badly in school although conversely if you have a high IQ you are not guaranteed straight A's either.  Other factors like work-ethic, effort, teaching quality and probably a bit of luck will play into it as well.  IQ is half the story of how well people typically do in school. 

Income as an Adult - 2007 study by Tarmo Strenze analysed over 29,000 people comparing IQ score with income.  Strenze found a 23% correlation which means there is a loose relationship but it's not everything.  IQ can contribute a little bit to how much you'll earn as an adult but it's one of many factors. 

Crime and Violence - 2013 study by Predesco of crime and IQ statistics of a dozen European countries found correlations of violence 50%, homicide 37%, motor-theft 39%, burglary 30% and  nothing for robbery (6%).  People convicted of crimes, especially violent ones, are more likely to have low IQs.

The P300 - 1998 study by Diaz found a correlation of 44% between IQ score and what's called the P300 value.  This one is tantalising because your P300 value is a measure of neuron speed.  It measures how quickly after a stimulus your brain waves change and on average, human neuron response-time is 300 milliseconds (hence the number) but people with higher IQs tend to be a few milliseconds faster.  People with higher IQ scores are, on average, slightly faster in neuron response speed.  Important note: This study sample was a lot smaller than the others but its result is too intriguing to not mention.

Weirder Stuff: IQ correlations have been shown between a few other unexpected personal characteristics.  I won't go into much detail but the studies are out there if you want to go looking.  IQ correlates positively with...

a) Being a fan of dark humour
b) Enjoying puns
c) Having been breastfed as a child

Yup.

Ummm...What???
    Because IQ is a slightly fuzzy measurement of a slightly fuzzy concept, it's possible to pull all sorts of correlations out of the data, some of which are subtle or misleading.  It's widely accepted for instance that the tests are very dependent on culture and upbringing (after all, the first few sub-tests on the WAIS-IV are about things like general knowledge and vocabulary which are very environmentally shaped). 
    Sometimes this can be geographic e.g. children raised in Asian cultures perform, on average, slightly better than children raised in American cultures.  This isn't really surprising given that the two cultures are vastly different, but it leads to the unfortunate misconception that "race" affects IQ.  In fact, when you correct for environment, culture and upbringing, these differences vanish and people of all races do just as well on IQ tests.
    But there are a few harder to explain quirks which come out.  For instance, IQ scores negatively correlate with SAT scores i.e. people who do well on IQ tests are likely to do worse on their SATs...which is the exact opposite of what you'd expect.  Another really weird correlation is that left-handed people tend to perform a few IQ points higher than right handed.  Maybe because they live in a world which requires a bit of extra focus and puzzle-solving to do basic tasks (all the tools and equipment of the world are geared for right-handers) so perhaps their brains are just a teensy bit sharper and more practised? 
    Another interesting finding is that people's IQ tends to decrease during adolescence but then increase with old age i.e. a child with an IQ of 140 at the age of 9 might drop to an IQ of 110 by the time they're 22, but then go back up to 140 by their 40s.  Contrary to popular myth, older people actually have higher IQs than younger people.  Nobody has a clue why that's happening. 
    More baffling, and probably one of the most heated debates in the psychological community, is the "Flynn effect" discovered by James Flynn: the undeniable fact that the global average IQ scores are going up and not by a small amount.  Flynn, who is not anti-IQ testing as has been reported, discovered that every generation performs significantly better on IQ tests than their parents did at the same age.  So although your parents might have a higher IQ than you, you're probably smarter than they were at your age.  But why is this happening???  Is modern culture preparing people to do better on the tests?  Is it a change in diet?  Are smarter people breeding more?  We don't have a clue but the fact is there: IQ scores are going up.
    There are also fascinating correlations between IQ and voting habits, religious practices and this is part of what makes IQ such a hotly debated topic.  There currently isn't an accepted theory of what IQ even is, so it's up for grabs and nobody's exactly sure what's baby and what's bath-water. 

The Young Science
   Psychology is a new science and it's finding its feet.  IQ tests are one of its first major contributions and, naturally, they're going to be a bit clunky.  But, I ask you fellow scientists, were physics, biology and chemistry so different once? 
   Mendel's theory of genetics was a simplistic way to explain variation but it has a few nuggets of truth in it and pointed us in the right direction.  Bohr's early model of the atom was unrefined and misleading but it gave us a starting point and, again, pointed us in the right direction.  Lavoisier's first periodic table muddled up compounds with elements but, once again, it pointed us in the right direction.  Science doesn't usually jump straight to the right answer in one go, it makes casual stabs first and I motion that IQ testing is of a similar nature. 
    The tests are problematic, vague and unreliable I agree but the fact that they do predict a few things with accuracy means they are a step in the right direction.  There is something to be learned from them, we just don't know what it is yet.  Yes, we have to be cautious and recognise that it is not the final picutre or even close to it, but it's an approximate sketch that vaguely resembles what we're ultimately looking for.  Trusting or disregarding IQ in its entirely seems unwise to me.  Recognising it as the first feeble attempt at something which might eventually work...that seems more like the "intelligent" thing to do.

Happy New Year

Image Credits
Bond and Q: Clearly
Picard and Q: Nerdist
Bond and Q2: DenofGeek
Pool cue: Let's play billiards
Join the queue: queuezone
Haiku: amazon