Commentary

Introduction

Acupuncture clinical research can be split into efficacy trials and effectiveness trials. Efficacy trials have a sham/ placebo control group. It is assumed that patients will feel better by going through the process of having a consultation and the ‘ritual’ of acupuncture. This is often described as the placebo effect. The objective of efficacy trials is to understand the clinical benefits of acupuncture beyond these psychological (placebo) effects. Effectiveness trials compare acupuncture to another form of treatment, for example medication. The results of clinical trials can be brought together in a systematic review (See Acupuncture Research).

The objective of a systematic review is to gather all the available clinical evidence and, hopefully, draw a conclusion about the effectiveness of therapy or medication. Clinical trials are often not exactly the same: they are designed by different people, in different places, and sometimes try to answer slightly different questions. Therefore, when compiling a systematic review, researchers are often faced with a problem known as clinical diversity. This refers to variability in the participants, the interventions or the outcome measures. For example, one clinical trial may only include people over 50 whereas in another trial the criteria is over 40. Therefore, there is variability in the participants. Slightly younger people might show greater improvement. Whilst this is a small difference, sometimes there is greater variability and the researchers must decide whether it is appropriate to combine the results.

No-acupuncture controls

A description of ‘no-acupuncture’ group might give the impression that the participants didn’t get any form of therapy. However, it refers to a number of different things: another therapy or medications or waiting list.  For example, in one trial, acupuncture was compared to ‘waiting list’. The patients could take oral NSAID, if required, but not corticosteriods [3]. In another trial, acupuncture was compared to ‘conventional therapy’. This meant the patients had 10 sessions with a doctor or physiotherapist who administered physiotherapy or exercise [4].  The key thing to remember is that ‘no-acupuncture’ does not mean the patients received no treatment.

Differences in design can sometimes be the reason why clinical trials report different results. As stated on the summary page, the overall result for acupuncture vs. no acupuncture showed, for headaches, acupuncture led to a greater reduction in pain SMD 0.44 (95% CI 0.39, 0.48). This is the result for all seven trials combined. But if we look at individual trials, we can see there is variation in the results. At one end Ferro et al (2012) indicated that acupuncture was superior to the no-acupuncture control by 1.5 SMD (95% CI 0.96, 2.05)[5]. This is a big difference. At the other end, the result of Diener et al (2006) was 0.09 SMD (95% CI -0.06, 0.24) [6]. Although this study does show a superiority of acupuncture it is a small difference that cannot be differentiated from chance. The largest study Jena et al (2008) SMD 0.46 (95% CI 0.41, 0.52). This is evidence of moderate clinical benefit.

If we examine the details of each of these three trials, we can get an idea of some of the factors that may lie behind these different results.

Table 1 shows 3 aspects of the clinical trials that may lie behind the differences in the results. These are the comparison group, the number of acupuncture sessions, and the severity of the headaches.

Table 1: Headache clinical trials

Comparison group

Naturally the results will depend on the effectiveness of the therapy to which acupuncture is being compared. In Ferro et al (2012) the comparison group took Tanacetum parthenium (TAN), popularly called Feverfew. Whereas, in Diener et al (2006) the comparison group followed the guidelines of the German Migraine and Headache Society which was beta-blockers first choice then flunarizine and then valproic acid, as the third choice. Therefore, the reason why Ferro et al (2012) indicates a large benefit from acupuncture and Diener et al (2006) indicates a small benefit could be that beta-blockers etc are more effective than TAN.

Beta-blockers, flunarizine and valproic acid are prophylactic medications for migraine. The patients in Diener et al (2006) were not taking these prior to enrolling on the study. So, the medication was a new course of therapy for these patients. The primary outcome data was measured using data from an endpoint of 26 weeks after treatment had commenced. At 26 weeks the acupuncture group had not had acupuncture for 20 weeks. On the other hand, those in the comparison group were still taking the prophylactic medication. Research suggests that in practice many people choose to discontinue taking prophylactic medication after six months [7]. The reasons for discontinuation are unclear but may be attributable to the side effects  [7].

Number of treatments

In evaluating the evidence base for acupuncture, it is important to assess whether the acupuncture given within clinical trials is similar to that given in practice. If there are differences, this may impact on the conclusions that can be drawn. This is called external validity. An important aspect of external validity is the number and frequency of treatments given. All three trials provided more than ten treatment sessions which probably reflects usual practise to some extent. However, in practice after an initial course of 10 sessions, treatments may be spaced out to once a month for a period of time.  The systematic review by Vickers et al (2018) demonstrates that acupuncture is dose dependent for chronic pain condition such as headache: more treatments lead to a greater reduction in pain [1]. Similarly, a recent systematic review shows more treatments lead to a greater reduction in the severity of depression [8]. It is worth noting that Ferro et al (2012) provided 20 sessions and that this may be the reason why this study indicated a particularly large size of effect.

Diener et al (2006) suggests that acupuncture and prophylactic medication are equally effective. However, we don’t know the long-term outcomes for these patients.  Acupuncture may be more effective in the long-term especially if monthly treatments are given after the initial course. In addition, if patients discontinue the use of prophylactic medication because of the side effects, then the reduction in headaches maybe lost. The number of treatments required is an important subject. For those who wish read more: Is acupuncture dose dependent? Ramifications of acupuncture treatment dose within clinical practice and trials by Bauer, McDonald and Saunders (2020) [9] provides a good explanation.

Severity of headaches

Patients in the Ferro et al (2012) trial had a migraine on at least 15 day per month, whereas Diener et al (2006) had 2-6 attacks in four weeks. Therefore, the Ferro et al (2012) patients were more severely affected, and this may be a factor in greater benefit observed.

Study size

Larger studies generally provide more reliable results. The Jena et al (2008) study with 15056 patients was by far the largest. It is also the broadest in terms of entry criteria, including those with both migraine and tension type headache. The study accepted those who were taking prophylactic medication and those who were not. In short, the study examined the benefits of adding acupuncture to the care usually given to those with migraine and tension headaches. The authors concluded:

Acupuncture plus routine care in patients with headache was associated with marked clinical improvements compared with routine care alone.

Efficacy, placebo and controversy

Despite the no acupuncture control evidence of effectiveness, acupuncture remains controversial. Researchers make different conclusions and recommendations. This can be largely attributed to differences of opinion regarding placebo/sham acupuncture. There are two things to consider: statistical significance and clinical significance.

Statistical significance evaluates whether the differences between groups might be due to chance. This is a bit like playing a betting game with someone by tossing a coin. If the person says heads and wins, we put this down to luck. If we play again and it is heads again there’s a one in four chance of that happening, still lucky: then 1:8, 1:16. The question gradually becomes is the person cheating or just extremely lucky? In clinical research the question is at what point do we conclude that one therapy is better or the result is due to chance. The more research that has been done in high-quality clinical trials more confident researchers are about the results. The study by Vickers et al (2018) Acupuncture for Chronic Pain: Update of an Individual Patient Data Meta-Analysis includes data from over 20,000 patients. Therefore, we can be confident in the results.  This study indicates that acupuncture is superior to placebo.

Vickers et al (2018) found the difference between acupuncture and placebo acupuncture for headaches to be SMD 0.16 (95% CI 0.06, 0.25)[1]. Some researchers believe that the size of the effect is too small to be clinically significant. Clinical significance can also be described as minimal important differences (MID). If the benefits of a therapy are small, it might not be worth allocating precious resource to provide that therapy. Naturally, the value of any improvement is a subjective issue. For someone with the condition any improvement may be significant. Clinical guidelines must, however, consider the cost of delivering a therapy compared to its therapeutic gains.

There are two fundamental problems with the assessment that acupuncture does not have clinically significant benefits when compared to sham acupuncture. The first is that sham acupuncture is an active procedure, it is not an inert placebo. An active control reduces the size of effect. In other words, the size of effect is most likely larger than the SMD 0.16 estimated by Vickers et al (2018). Secondly, sham-placebo acupuncture is not given in practice. The resource is required to provide acupuncture and sham acupuncture are the same. There is no additional cost to provide acupuncture.

For further reading on sham placebo acupuncture see:

• Appleyard, I., T. Lundeberg, and N. Robinson, Should systematic reviews assess the risk of bias from sham–placebo acupuncture control procedures? European Journal of Integrative Medicine, 2014. 6(2): p. 234-243 [15]
• Factsheet – Back pain 2022
• Factsheet – Osteoarthritis: knee 2021

References

References

1.         Vickers, A.J., et al., Acupuncture for Chronic Pain: Update of an Individual Patient Data Meta-Analysis. The journal of pain : official journal of the American Pain Society, 2018. 19(5): p. 455-474.

2.         NICE, Headaches in over 12s: diagnosis and management, in Clinical guideline CG150. 2021, National Institute for Health and Care Excellence.

3.         Brinkhaus, B., et al., Acupuncture in patients with chronic low back pain: a randomized controlled trial. Arch Intern Med, 2006. 166(4): p. 450-7.

4.         Haake, M., et al., German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Archives Of Internal Medicine, 2007. 167(17): p. 1892-1898.

5.         Ferro, E.C., et al., The Combined Effect of Acupuncture and Tanacetum parthenium on quality of life in women with headache: Randomised study. Acupuncture in Medicine, 2012. 30(4): p. 252-257.

6.         Diener, H.C., et al., Efficacy of acupuncture for the prophylaxis of migraine: a multicentre randomised controlled clinical trial. Lancet Neurol, 2006. 5(4): p. 310-6.

7.         Hepp, Z., et al., Persistence and switching patterns of oral migraine prophylactic medications among patients with chronic migraine: A retrospective claims analysis. Cephalalgia, 2016. 37(5): p. 470-485.

8.         Armour, M.A.-O., et al., Acupuncture for Depression: A Systematic Review and Meta-Analysis. . Journal of Clinical Medicine, 2019. 8(2077-0383 (Print)).

9.         Bauer, M., J.L. McDonald, and N. Saunders, Is acupuncture dose dependent? Ramifications of acupuncture treatment dose within clinical practice and trials. Integrative Medicine Research, 2020. 9(1): p. 21-27.

10.       Mu, J., et al., Acupuncture for chronic nonspecific low back pain. Cochrane Database of Systematic Reviews, 2020(12).

11.       NICE, Low Back Pain and Sciatica in Over 16s: Assessment and Management: , in Clinical Guidelines NG59. 2016, National Institute for Health and Care Excellence

12.       Witt, C.M., et al., Pragmatic Randomized Trial Evaluating the Clinical and Economic Effectiveness of Acupuncture for Chronic Low Back Pain. American journal of epidemiology, 2006. 164(5): p. 487-496.

13.       Cherkin, D.C., et al., A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Archives Of Internal Medicine, 2009. 169(9): p. 858-866.

14.       Hinrichs, T.J. and L.L. Barnes, Chinese medicine and healing: an illustrated history. 2013, Cambridge, Mass.: Belknap Press of Harvard University Press. 464.

15.       Appleyard, I., T. Lundeberg, and N. Robinson, Should systematic reviews assess the risk of bias from sham–placebo acupuncture control procedures? European Journal of Integrative Medicine, 2014. 6(2): p. 234-243.