Moar Science Plz
Physical activity program leads to better behavior for children with ADHD
This is kind of headline I’m used to seeing when people talk about press about studies of children with ADHD. There is so much populist pushback against the idea of medicating of children with drugs like Ritalin (or in my case, dexedrine) that it seems you can’t swing a dead cat without coming across the idea that there is some easier non-pharmacological way to improve outcomes. And then when I look closer, I find that the glowing ideas are somewhat dubious when you look at the actual procedures behind the headlines. This one is no exception.
When I came across this study, the person promoting it trumpeted it as “no surprise” since everybody knows that the whole problem with ADHD kids is that they can’t sit still and focus, right? You know what, fuck you. You’re part of the problem that led to years and years of my misery going undiagnosed (ignorance, confusion and stigma) and untreated (ZOMG, don’t give kids drugs that clearly help them). But before we get to that, let’s look at the particulars.
- The study did not include children without ADHD to provide a contrast of neurotypical students (Because this will undoubtedly lead to replacement of other treatment modalities, rather than addition in most cases, you must contrast results of ADHD children with neurotypical children to get some idea of effectiveness as a coping strategy.)
- They studied exactly ten children and eleven as a control, quite a small sample size (Do I even need to point out why a tiny sample size is problematic in scientific research?)
- The control group is not particularly well controlled for environmental variables as they were recruited from numerous other schools while the experimental students all attended the same school (Because the results being measured are of a fuzzy social/behavioral aspect, controlling for variables in teaching and environment is key)
- Girls with ADHD were not well represented in this study, 1 per group (Women are underresearched in medicine to begin with, but I can tell you that as a girl with ADHD, I was basically invisible. Girls are often underdiagnosed in ADHD, although I hope it’s improved from my childhood, and ignoring their response to treatment modalities as unimportant during behavioral research will continue to produce studies that may not be useful for treating female children and adolescents.)
- The control group were all taking medication while only 30% of the experimental population was on medication to manage their symptoms (I took dextroamphetamines to treat my ADHD and while it was almost miraculously helpful, it is not without physical effects. I was terribly underweight at times and remember having to force myself to eat when I wasn’t hungry. I also had to take medication that aided in sleep since I was taking powerful stimulants. Given that this treatment is physical and measures changes in fitness/strength/motor skills, medical side effects that could include reduced appetite and weight loss should not be ignored. Moreover, this is one of many ways in which variables in the study were ignored rather than controlled; comparison between these two groups can’t discount medication as a difference.)
- Two of the three types of ADHD were represented in the study (Hyperactive-impulsive, and combined); those with inattentive type were described as “not included in the theoretical model of ADHD” (I have a couple of problems with this. For one, it is another variable that’s not well controlled for in the experiment. For another, this is simply excluding a whole population of those who meet the diagnosis criteria for ADHD as unimportant while attempting to make generalized statements about improvements in children with ADHD without qualification.)
- Initial evaluations prior to the physical activity program were done with children explicitly not taking their medication, although they were allowed to be on medication during the experiment (Testing conditions prior to the experiment should be the same as those used during and afterward. Duh.)
- Behavioral results were tracked by parents and teachers using Achenbach’s Child Behavior Checklist; no self-evaluation or independent evaluation was used (Even during the ages referenced, I was a fairly secretive child and tended to hide depressive symptoms and problems with my homework/other tasks out of frustration and shame. Without any input from the children themselves, we have no way of being sure of many internal behavioral issues that are being evaluated by parents and teachers. Moreover, much like the sugar/hyperactivity studies in children, we have a huge problem of confirmation bias where teachers and parents expecting to see change will believe they observe change.)
- No way to blind the observers involved for control/experimental groups (Again, confirmation bias)
- Previous evaluation showed the control group had a higher incidence of withdrawn/depressed behavior prior to program and evaluation (Still more methodological problems in getting comparable data between research groups.)
- [Personal Note: No attempts are made regarding statistical analysis of task completion such as homework. It would not have been difficult to track any change in ability to finish and turn in homework as a hard statistical evaluator of task-related behavior improvement.]
- Study itself acknowledges flaws within both design and recruitment render the positive results reported more or less useless (They try to spin it positively, claiming that the pointless results are “exploratory” and really means that more study should be done on the subject, hopefully more well designed. I’m not against this kind of research per se, but I do think that it any positive results need to be cautiously described lest they lead more children to go without treatment like I did for much of my childhood. No one should have to suffer through that frustration, shame and pain.)
So the rosy headline that physical activity improves outcomes for children with ADHD is basically a lie. The researchers very clearly state that their data collection and other structural flaws cast doubt on the results. It’s not entirely the fault of the researchers as medical science reporting is notoriously bad. Not only can reporters and bloggers often not tell good science from bad, they overstate and exaggerate conclusions until they are sometimes unrecognizable.
So I want more science and I want good science and I want good science reporting. Is that so much to ask?