A write-up in The (Irish) Sun yesterday (posted on Twitter by Simon Dymond) has brought a site called Raise Your IQ to my attention. It’s a brain-training site that makes a startling claim; that their SMART (Strengthening Mental Abilities with Relational Training) program can raise IQ, by an average of 23 points. Those are some pretty big words, right there.
The business is a spin-out company from the National University of Ireland, and was started by Bryan Roche and Sarah Cassidy, of the psychology department at NUI. Dr Roche is an expert in a fairly niche area of psychological inquiry known as Relational Frame Theory. RFT is a theoretical framework that seeks to extend the radical behaviourist ideas of BF Skinner to encompass and adequately explain complex cognitive processes (language, abstract thought, etc.). Skinner himself was firmly of the opinion that cognitive processes are in fact aggregations of lower-level behavioural responses, and that things like our sense of free-will, and our awareness of our own phenomenal consciousness are entirely illusory (views he expressed in his classic book Beyond Freedom and Dignity, as well as in many more academic works). RFT is essentially a neo-Skinnerian approach which takes into account the accumulated evidence for complex (i.e. nominally non-associative) cognition.
As a theoretical approach RFT has some interest then, however Dr Roche has managed to add a practical twist and come up with something marketable, dubbed ‘relational training’. Essentially it’s a set of questions and tests that are based on relationships between different objects, words or categories. Some examples are below (pasted from The Sun article):
LEVEL 1: If a broad bean is a type of bean, is a bean a type of broad bean?
LEVEL 2: If an Alsatian is a type of dog, and a dog is a type of mammal, is an Alsatian a type of mammal? Is a mammal a type of Alsatian?
LEVEL 3: If yesterday I felt sad and today I feel happy, and if now was then and then was now, how did I feel then?
LEVEL 4: “Car” is the English word for the Swedish word “Bil”. “Car” is also the English word for the Italian word “auto”. “Coche” is the Spanish word for the Italian word “auto”. What is a Coche in Swedish?
Readers who were fortunate enough to have endured a Classical education will immediately recognise the broad form of these as a kind of logical puzzle known as a syllogism, the formal description of which dates back to at least Aristotle; plus ça change. Syllogistic reasoning is also a mainstay in various IQ-type tests.
So, what about these pretty radical claims of raising your IQ by 20 or even 30 points? Does it work? What’s the evidence? As usual I’m afraid, the answer is ‘scant, to non-existent’. A prominent page on the Raise Your IQ site is titled ‘Scientific Evidence’ and discusses in some detail this paper by Cassidy et al (2011; PDF), which does indeed show some impressive effects in two separate experiments. However, experiment one had only four participants in each group (training vs. control), and experiment two had only eight participants (with no control group). These are tiny samples and (as has been extensively discussed, very recently) small samples can lead to the ‘winner’s curse’; an inflation of the apparent effect size. The effects reported in this paper are very large indeed, but honestly, I’m surprised the reviewers didn’t flag up the fact that you just can’t do ANOVAs with four data points! It’s a massive violation of the assumptions of the central limit theorem. In addition, the control group in experiment one was an ‘inactive’ control i.e. one that received no training at all, rather than some kind of placebo training.
The rest of the papers listed on the Scientific Evidence page of the site appear to be fairly theoretical, or only tangentially related to the SMART program. The available evidence from the Cassidy et al. (2011) paper is wafer-thin, compared to the startlingly bold claims being made. One other thought that occurred to me is that because the form of the training is so similar to the form of some questions used on many IQ test, it might represent a form of training-to-the-test; practicing IQ tests makes you better at performing IQ tests – well, duh.
I find myself unable to completely condemn these guys though. Maybe I’m going soft, or maybe I’m just a sucker for some good old-school radical-behaviourism-based interventions. The program does have the (somewhat unusual) virtue of being based on a fairly coherent (if not widely-accepted) theoretical foundation. This doesn’t change the fact that the big claims it makes are wildly out of sync with the available evidence, and in this sense, it’s precisely the same as all the other brain-training neurobollocks-merchants out there fighting for a slice of the gullible consumers cash.