Why the study authors made this a pilot study to see if they should do another study is confusing to me, although I think I can figure out why. But I’m getting ahead of myself and will address that later in this post. They ostensibly designed and carried a full fledged study which could have tested the hypotheses they put forth, but instead they framed the study as a dress rehearsal for the “real” study. This was based upon studies some of this paper’s authors had previously done which suggested they should do more studies.
By their own admission in the paper, CAM has offered very little in terms of successful management of TMD. They cite references evaluating Traditional Chinese Medicine (TCM), acupuncture, application of a topical herbal ointment, osteopathic manual therapy, a myofascial form of chiropractic therapy, and prior pilot studies by some of the authors using AMCT (one a case report; the other a prospective case series of eight patients). The authors’ conclusion regarding this literature review was:
“Although investigators in these studies reported some degree of reduction
of TMD symptoms, the improvements were modest, with no definitive
conclusion reached about which approach was best.”
Translation: All are equally ineffective and probably reach the level of no treatment at all (TMDs tend to be self limiting, after all) or placebo. This should be a huge scientific neon sign screaming “NO FURTHER STUDIES ARE NEEDED! The verdict is in- these CAM modalities don’t work. Let’s pick another rock to look under.” But nope, a few people saying “Yeah, I think I’m feeling better” is “proof” that they are onto something and that they need to keep studying whether poking here, rubbing there, cracking this, or channeling energy on that will work. Not only are they beating a dead horse; they’re beating the greasy spot where the dead horse’s corpse used to be.
Study Design and Methodology
Eighty participants (out of 721 candidates) who met the inclusion criteria were randomized into four groups. They were:
Self Care. All participants in all groups received instructions on a self care TMD program. Conservative, do-it-yourself strategies such as stretching, relaxation, thermal packs, OTC pain medication, etc. This group served as a control, as no clinical intervention was performed.
RIST. An acronym for Reversible Interocclusal Splint Therapy. Also known as a biteguard or nightguard, a splint is a clear fitted acrylic appliance worn to mitigate the effects of tooth clenching and grinding. Participants in this category were instructed to wear their appliance every night and for two hours each day. This was also used as a control to compare the results with AMCT.
AMCT. As stated above, this group received the Activator Method treatment of chiropractic care. This involves adjustments using a small spring loaded device designed to apply rapid force onto the selected area. The authors did not state which areas were adjusted with the Activator.
Sham AMCT. This group received a bogus treatment in which the Activator was sheathed so that the device made the audible “clicking” sounds while not delivering an actual thrust of force. The patient was blinded and was not aware that it was a sham treatment. The operator was not blinded. Also used as a control.
Out of the 721 initial screening respondents, 360 did not meet the study criteria while 137 (wisely) declined to participate. Eventually the number of enrolled participants dwindled to 80, and these were randomized into the four treatment groups. Only 52 participants completed the study. The Self Care group, RIST group, and Sham AMCT group served as controls against the AMCT treatment, although in my opinion there should have been a “No Treatment” group to accurately gauge the effectiveness between the study samples.
The groups were evaluated at baseline, at two months, and at six months, at which time they scored their progress in pain relief, quality of life index, a “bothersome index” (impact of TMD pain on their daily living), and a satisfaction of care rating.
The authors stated that they “chose the sample size to determine feasibility and, therefore, the study was not powered to detect differences between groups.”
Translation: “We had an embarrassingly small inclusion rate (11%) and an even embarrassinglier smaller completion rate (7%), and the differences in results were so small as to be insignificant (Oops! I gave away the ending! Forget I said that.). That doesn’t make sexy science, and yet another research paper about the ineffectiveness of CAM therapies will not get us very many speaking gigs at the Mercola conference next year.” Also, I must point out that they did not choose the sample size; the sample size chose them. At 11% inclusion, an African American woman had a better chance of being accepted into the Augusta National Golf Club in the 1950s.
In general, all groups experienced improvements in all areas. All groups experienced greater improvement at Month Six than Month Two except for the Sham AMCT group, which remained approximately the same. None of the treatments, including the Self Care control, showed any statistically significant advantages over any other group. At the six month evaluation, the satisfaction of care index was roughly equal, with the RIST group being significantly higher than the other three.
There are many problems with the scope and methods of this paper. First, as was noted before, the number of participants dwindled through exclusion and dropouts from 721 to 80 to 52. This resulted in sample sizes too small to have any statistical or clinical clout.
“But” you might say, “this was a pilot study to determine the feasibility of another study just like it; the small sample size doesn’t matter because the authors weren’t focusing on treatment outcomes.”
Whatever lets you sleep at night.
And therein lies the rub.
I question whether the original intent of this research project was really to be a pilot study necessary to prepare for a larger study. After all, they went through the entire research protocol anyway and gathered (and published) the data. No, I suspect (but could never prove) that the focus of the study was changed once they realized that their pet treatment (AMCT) was ineffective. Or, perhaps they called it a pilot study to begin with, but had the AMCT been proven effective, they would have had a ready made paper, ripe for publishing. I think they hedged their bets and gave themselves an out, an out they ultimately needed to take when things didn’t go their way. This is pure speculation, of course, but I think I smell a rat.
Looking at it from another angle, perhaps the authors realized that they wouldn’t be able to get a paper involving a CAM modality past the esteemed and astute JADA scientific editors, especially if said modality was no more effective than do-it-yourself treatment, sham treatment, or conventional dental management (RIST). However, designing a study about a study can be thought of as scientific, as long as they played by the rules of research, which they did. It was this tack that allowed them to sneak by the editorial censors and wind up in JADA.
Either way, in my opinion, the authors moved the goalposts.
Another issue I have is that their “control groups” were accptable control groups, but a group with no treatment or information at all would have made a better control. As stated earlier, many if not most TMD issues are self-limiting and, like many chronic pain conditions, tend to ebb and flow and often resolve on their own with no intervention. A No Treatment group would have allowed the researchers to not only tease out differences between the groups, but compare them with patients who did nothing.
A minor point, but one worth mentioning, is the use of the term “Chronic Myofascial Temporomandibular Disorder.” There is some controversy as to whether Myofascial Pain is an actual diagnosis or if it is merely descriptive, or even if it’s a valid designation at all. It is the dental equivalent of fibromyalgia in that it is ill-defined, there are no clinical or laboratory tests that are diagnostic for it, and that its diagnosis is not standardized or agreed upon by clinicians. When a diagnosis is vague and elusive, it makes it difficult to assess whether a specific treatment is efficacious or not.
In the AMCT and Sham AMCT groups, the participant was blinded, but the operator was not. I’m not sure that blinding is possible in this situation, so I’m not criticizing that per se. However, there was the potential of introducing operator bias into the treatment protocol when the operator favors one treatment over the other. Perhaps it is in the way he/she interacted with the patient, the confidence or gentleness with which he/she operated the Activator, and so on. This bias is unavoidable, no matter how conscientious the researcher might be. To their credit, the authors stated in the discussion that in “future studies”, they would not recommend using the Sham AMCT group in the project because it would be “difficult to achieve truly consistent clinician-patient interactions between the active and sham AMCT groups.” Of course, another reason to leave out sham treatment would be because it is just as (in)effective as their “real” treatment. Awkward.
The really humorous aspect of this article is that the authors acted as though they already knew their CAM therapies wouldn’t work. It seemed like they were making pre-emptive excuses as they wrote the paper. Phrases such as “we targeted 20 participants per group to assess their willingness to be assigned randomly to one of the four treatment groups…” were conveniently inserted in order to minimize the disappointing study size and treatment outcomes. When discussing what must have been disheartening results, the authors phrased it thus: “As expected at the onset of the trial, none of the differences was statistically significant at the .05 level” (emphasis mine). Yeah, they knew there would be no difference in any of the study metrics, but they recommend designing another study just like it, only larger and more expensive, just to make sure. Makes sense to me.
Why the JADA Should Have Passed On This One
First and foremost, this research paper did not significantly add to the body of knowledge in the treatment of TMD. A literature search by anyone with a cursory knowledge of science would reveal that there are no effective CAM treatments for TMD beyond placebo, and to compare an unproven, unscientific chiropractic treatment modality to other forms of treatment diminishes the scientific status of a prestigious journal such as JADA. As an ADA member, I am angry and embarrassed that they would even consider publishing such nonsense, as any science based clinician could have predicted the outcome before it started. As the saying goes: “What do you call alternative medicine that works? Medicine.” Why is the ADA wasting its time and money on voodoo?
It diminishes the dental profession when its Association embraces (or at least turns a blind eye to) CAM. Proponents of CAM, including chiropractors, naturopaths, acupuncturists, and others, have been fighting for legitimacy for decades. Universities will not allow schools of chiropractic to be associated with them because of
conspiracy by physicians the lack of scientific and biologic plausibility, unscientific magical “innate intelligence” beliefs, and unproven claims and techniques. By cooperating on a study with University of Iowa College of Dentistry dentists, the chiropractors have hitched their woowagon onto a legitimizing horse, and being blessed by the ADA adds undeserved scientific street cred to pseudo-scientific treatments.
The JADA editors should have rejected this article out of hand.
Why should I care if a bunch of people who think tapping on a person’s bones will move them and make them all better do a study about TMD? Why don’t I leave well enough alone? Why aren’t I more open-minded?
Not only was this a colossal waste of time- by the authors themselves, the participants, the JADA reviewers and editors, the readership of the JADA who had to read through this drivel, and by people like me who waste time stewing and writing about it – but it was a waste of financial resources. As stated at the first of this post, this study was funded by the NCCAM, the woo factory of the NIH that has poured billions of dollars of taxpayer money over the past ten years or so to fund CAM research. An accounting of the NCCAM has shown that since its founding there have been NO significant medical findings to come out of it. Zip. Zilch. They throw money at worthless research projects, money that could be better used funding legitimate research.
It is a common tactic in alternative medicine research to pull the “More Research is Needed” Gambit when their pet treatments are shown to be ineffective. This is no different. Earlier research (here and here) by some of the same authors going back to 2003 also stated that “more research is needed.” So this paper is more research, and what did they conclude? That is was a good way to do research, so now they’re ready to do more research. I call this fractal research, and in this case, Benoit B. Mandelbrot would indeed be proud.
When CAM research is published in respected scientific journals- The Lancet, JAMA, JADA, etc.- the studies are exposed to a wide audience and are archived for future reference. In this paper, even though it was presented as a proof of concept paper, it will be used by future woopeople as evidence for the effectiveness of AMCT. The language used in the title, as well as the wording in the article itself, was ambiguous enough as to throw up a smokescreen to its “real” intent. When down the road some alternative practitioner wants to shore up the credibility of his/her treatment or theory, he/she could likely cite this article as a reference, which on its face (if one only reads the title or abstract) appears to legitimize and confirm Activator therapy. Unless one dives into the primary research (which is almost never), no one would be the wiser.
Shame on the JADA. Shame on them for publishing such scientific pabulum. Shame on them for condoning unproven, unscientific treatments. And shame on them for contaminating the already confusing TMD literature with confusing and irrelevant material. I expect better from them.