3.3.3 Problems with the Trial Methodology
There are a number of limitations associated with the methodology of the trials selected for inclusion in this review. As noted, all six of the studies were randomised controlled trials, as these produce the highest quality of primary evidence to inform clinical practice (Oxford Centre for Evidence-Based Medicine, 2009). However, the use of randomised controlled trials for research on acupuncture has been strongly criticised (McDonald, 2019). This is because randomised controlled trials attempt to measure a single variable whilst controlling for all others – however, and as shown throughout the previous section of this review, acupuncture is associated with a variety of variables which may not be easily controlled (McDonald, 2019). Further complicating this is the fact that acupuncture’s mechanism of action in relation to hypertension is still largely unclear (Li et al., 2019). Thus, some authors go so far as to claim that randomised controlled trials for acupuncture are “inappropriate” (McDonald, 2019: p.47).
There are also problems associated with the use of sham acupuncture as a control in the trials selected for inclusion in this review. Most of the studies (n=5, 83.3%) used sham acupuncture where needles were inserted at acupoints not related to hypertension (Macklin et al., 2006; Flachskampf et al., 2007; Yin et al., 2007; Zheng et al., 2016; Zheng et al., 2018). The alternative is to place needles at actual acupoints, but at a superficial depth (Lin et al., 2012). However, both approaches to sham acupuncture have been strongly criticised because there is no evidence that either is inert, and so both may influence patient outcomes (McDonald, 2019).
It has also been argued that sham acupuncture delivers a placebo effect (Xiang et al., 2018). McDonald (2019) goes so far as to suggests that the placebo effect may explain up to 30.0% of all treatment effects seen in randomised controlled trials of acupuncture interventions where sham acupuncture is used. If this was the case in the studies selected for inclusion in this review, the positive effects of acupuncture on hypertension would be significantly exaggerated. This was suggested in a recent meta-analysis on the use of acupuncture for pain, where trials involving sham acupuncture were far more likely to report positive findings (Chen et al., 2016).
It is also noteworthy that none of the studies selected for inclusion in this review involved blinding. ‘Blinding’ occurs when clinicians and/or patients are made unaware of whether an intervention or a control treatment is being delivered, thereby reducing the risk of bias (Polit & Beck, 2010). The blinding of both clinicians and patients can be achieved in randomised controlled trials on acupuncture using placebo needles and/or needle tubes, though patients who are experienced with acupuncture may guess a placebo needle is being used because it produces a different sensation to a real needle (Lin et al., 2012). Again, there is also no evidence that placebo needles or needle tubes are inert, and no evidence about which type of sham device is more effective (Zhang et al., 2015). Nevertheless, blinding of clinicians and patients is an important consideration for future trials testing acupuncture for hypertension.
A final limitation of the studies selected for inclusion is that many included small cohorts. As noted, the studies selected for inclusion involved an average of 147 participants, ranging from 30 patients (Zheng et al., 2016) to 428 patients (Zheng et al., 2018). Three of the trials included <50 participants (Yin et al., 2007: 40 patients; Kim et al., 2012: 33 patients; Zheng et al., 2016: 30 patients). When a cohort is small, this increases the likelihood that it is not representative of the broader population (Moule & Goodman, 2009) –
