The health marketplace has a life of its own, mostly separated from science and evidence. Generally the marketplace gets a hold of an idea and runs with it, before the science is carefully worked out. Since most new ideas in science turn out to be wrong, that means most products will eventually be found to be worthless.

One such idea is that “brain training” can improve overall cognitive function – so of course now there is an industry of products which claim to train your brain. Lumosity (just to pick a random example served up by Google) claims on their website:

Brain Train
SCIENTIFICALLY DESIGNED

* Improve memory and attention
* Shown to improve cognitive function
* Neuroscience based brain training
* Train your brain today

I always enjoy the phrase “scientifically designed” or “scientifically formulated” – they are wonderful marketing phrases that invoke “science” without making any specific claims.

The notion of improving brain function by practicing certain tasks or playing cognitively demanding games is an attractive one. I certainly would like this to be true: get smarter while playing video games – I’ll buy that for a dollar.

It also may or may not be plausible, depending upon how you look at it. What is uncontroversial is that the brain learns, and practice does improve function – no question. But, the evidence also suggests that genetics is a dominant determining factor of overall cognitive function – we all eventually seek our maximum potential. (This does not apply to knowledge or skills, but raw brain power in specific areas.)

Prior studies, both observational and experimental, have shown a correlation with playing certain kinds of games and cognitive ability. For example, one study showed that playing Half Life (a first-person shooter featuring physicist Dr. Gordon Freeman) significantly improved surgical skill performing virtual reality endoscopy.

Another study showed improvement in “executive control functions, such as task switching, working memory, visual short-term memory, and reasoning,” from strategy video game playing. And yet another study showed improvements in visual processing from playing action video games.

But questions remain. For example, how transferable are the skills learned from video games? Do subjects improve only in their ability to play the specific video game and closely related tasks, or does brain function improve in more abstract ways not directly tied to the video game?

Also, how much game playing is necessary? Does benefit come only from playing hours a week, and over months and years, or can more modest “training” be beneficial? And further, is self-selection bias primarily responsible for some of the positive results? In other words, are people who are already superior at certain tasks playing video games because they are good at them and therefore enjoy them more?

With this in mind, Adrian Owen et al. set out to conduct a large study of specific brain training products (not video games optimized for fun) on four standard measures of cognitive function: grammatical reasoning, verbal short-term memory, spatial working memory, and paired associates learning. They randomized 4,678 subjects to three arms – one group was trained on programs designed to enhance reasoning, planning, and problem solving. The second group was trained on memory, attention, visuospatial processing, and mathematical calculations. The third group, the control, was given five obscure knowledge questions and asked to use the internet to find the answers.

Each group was tested at baseline and after six weeks, regardless of how many training sessions they completed, but on average the first “treatment” group completed 28.39 training sessions, compared with 23.86 in experimental group 2 and 18.66 in the control group. At the end of the study there were no statistically significant differences among the three groups – all improved slightly in all four measures.

The mild improvement is almost certainly due to the training effect – this is seen generally in studies of cognitive function or studies that use performance on a task as an outcome – subjects get better just from experience. That is why control groups are always needed.

The study showed that the subjects improved in the tasks on which they were training, but that these improvements did not generalize or translate to the benchmarks of cognitive function. This is a fairly solid study with a clearly negative outcome.

Conclusion

This one study, of course, is not definitive. It is possible that more training is needed before significant benefits are seen. Perhaps video games are more effective because they are more engaging and players will spend more time playing.What this study shows, however, is that products sold as brain training games had no documented benefits after six weeks of use.

Putting this study into the context of the overall research, it does make us more cautious about concluding that there are general cognitive benefits to brain training games or entertainment video games. Benefits are likely to be closely related to the specific tasks involved in training, and not transfer to unrelated tasks.

But there is already enough published evidence showing visual tracking, multitasking, and executive function benefits from action and strategic video games respectively that this study will not be the final word. When there is conflicting research, more study is needed.

This study is most applicable to brain training products, and shows that the marketing claims for these products are not justified. There is very unlikely to be any benefit, or any specific advantage, to “scientifically designed” brain training applications. For now, you are better off just playing a video game.

Long live Gordon Freeman.