Acupuncture and mental health – the problem of external validity

There are a number of challenges in acupuncture research. Firstly, it is not possible to develop an inert sham/procedure [1]. Secondly, the challenge of ensuring the acupuncture is similar to acupuncture delivered in practice, known as external validity. You can find more information about these challenges in general and the placebo problem here acupuncture research. So let’s talk about the problem of external validity.

Acupuncture is often described as holistic. For BAcC registered acupuncturists, who have trained in one of the traditional based styles of acupuncture, this means treatment should be tailored to the individual. Therefore, two people who have both been diagnosed with depression will most likely receive different treatments: the acupuncture points will be different, moxibustion may or may not be used, different lifestyle advice maybe given. Of course, there will be similarities and certain acupuncture points will be commonly used.

In clinical research, the standard approach is to provide the same treatment. This means that often the same acupuncture points are used for all of those in the acupuncture treatment group. These are known as fixed point protocols. Some clinical trials allow the acupuncturist to choose from a pool of pre-selected acupuncture points. These are semi-fixed protocols. Finally, some clinical trials allow the acupuncturist to select points as they would in normal practice. This is known as ‘individualised’ acupuncture. Therefore, clinical trials that use fixed point protocols are less like practice than those that use ‘individualised’ acupuncture; with semi-fixed protocols are somewhere in the middle. This means, potentially, the acupuncture delivered in a clinical trial is not as effective as the acupuncture received in practice. There may be an underestimation of the benefit of acupuncture.

A systematic review explored the potential mechanisms that may underlay the improvements in depression and schizophrenia with acupuncture [2]. The reviewers noted there was a significant improvement in sleep quality as well as in depression. For traditional acupuncturists, the depression and insomnia are not separate conditions. They are part of an overall symptomatic picture, along with many other potential signs and symptoms (e.g. indigestion, pain, headache etc), that is used to make an individualised diagnosis.

Insomnia provides a good example of how treatment may be refined and individualised. Specific points can be used to promote sleep. So, these would be used for someone with depression who also found it difficult to sleep but may not be used if insomnia was not a problem. Insomnia itself can be further differentiated.  Different points are chosen if there is difficulty in falling asleep as opposed to waking up in the night. Similarly, if the insomnia is accompanied by night sweats different points may be selected.

Surprisingly, little attention has been paid to the quality of acupuncture provided in clinical trials, even though poor quality acupuncture may lead to misleading results. Fortunately, things are beginning to change. Relatively recently, a means to evaluate the quality of acupuncture has been developed: the NICMAN scale (National Institute for Complementary Medicine Acupuncture Network)[3]. The recent systematic review on depression, Armour et al 2019, used the NICMAN scale [4]. Evaluating the quality of acupuncture is a complex problem and much more work will need to be done. Nevertheless, the Armour et al 2019 represents an important step in in the scientific evaluation of acupuncture.

NICE Guideline NG222 Depression in adults: treatment and management
The Guidelines state (p100)9:
There was some evidence of effectiveness and cost effectiveness for the combination of acupuncture and antidepressants but the committee were aware this evidence was based on Chinese acupuncture, which is different to Western acupuncture and so these results may not be applicable to the UK population, …

Trials conducted in China report larger effects when compared to those conducted outside of China [4]. During the consultation process, the guideline committee highlighted the concern that this may be due to acupuncture being more commonplace in China which might lead to higher expectations [5]. Cultural differences may play a role. However, perhaps more likely, the difference is because clinical trials in China provide more treatment sessions. A greater total number of treatments is related to a greater reduction in the severity of depression [3]. Clinical trials in China often provide 30 to 40 treatments whereas outside China, 12 treatments or less is common. The failure to provide an adequate number of treatments is a common problem with clinical research conducted in the West for all conditions, not just depression.

During the Guideline consultation the committee raised the concern that there was a lack of appropriately trained and competent people to deliver acupuncture for depression [5]. British Acupuncture Council members are fully trained and competent. BAcC members can deliver acupuncture similar to that provided in China. The acupuncture in MacPherson et al (2013), the large high-quality trial conducted in UK, was provided by BAcC members. Of course, it would not be possible to establish all the necessary clinical services overnight, however, without inclusion in the guidelines there is no impetus to develop the services. If the committee had taken a more pragmatic approach, then acupuncture services could be developed.

What’s next?

In 2022 our Research & Policy Manager, Ian Appleyard, introduced his vision for ‘Integrated Projects’ – an umbrella term which describes any project where BAcC acupuncturists work with other healthcare professionals and organisations. Two years later the Inner Gloucester Primary Care Network (PCN) now has an acupuncture service delivered by three BAcC Members. The service is designed to delivering ear acupuncture for anxiety in an NHS setting. The feedback so far has been excellent. The PCN is pleased with the service and the acupuncturists involved are finding it very rewarding. At the UK Conference of Chinese Medicine in June, you’ll be able to hear from Jennie Heckford MBAcC, Wendy Williams MBAcC and Fleur Clackson MBAcC who created the service and will be sharing tips on how to deliver future projects.


[1] Appleyard I, Lundeberg T, Robinson N. Should systematic reviews assess the risk of bias from sham–placebo acupuncture control procedures? European Journal of Integrative Medicine 2014;6(2):234-43. doi: 10.1016/j.eujim.2014.03.004
[2] Bosch P, van den Noort M, Staudte H, et al. Schizophrenia and Depression: A systematic Review of the Effectiveness and the Working Mechanisms Behind Acupuncture. Explore (New York, NY) 2015;11(4):281-91. doi: 10.1016/j.explore.2015.04.004
[3] Smith CA, Zaslawski CJ, Cochrane S, et al. Reliability of the NICMAN Scale: An Instrument to Assess the Quality of Acupuncture Administered in Clinical Trials. Evidence-Based Complementary and Alternative Medicine 2017;2017:5694083. doi: 10.1155/2017/5694083
[4] Armour MA-O, Smith CA, Wang LQ, et al. Acupuncture for Depression: A Systematic Review and Meta-Analysis. . Journal of Clinical Medicine 2019;8(2077-0383 (Print)) doi: 10.3390/jcm8081140
[5]. NICE. Depression in adults: treatment and management: History. NICE Guidelines National Institute for Health and Care Excellence, 2022.
[6] MacPherson H, Richmond S, Bland M, et al. Acupuncture and Counselling for Depression in Primary Care: A Randomised Controlled Trial. PLOS Medicine 2013;10(9):e1001518. doi: 10.1371/journal.pmed.1001518