Shang reviews of homeopathy: Difference between revisions
imported>Ro Thorpe mNo edit summary |
imported>Gareth Leng No edit summary |
||
(15 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
{{subpages}} | {{subpages}} | ||
In | In 2005, ''The [[Lancet]]'' published the results of an innovative meta-analysis that became the single most cited study of homeopathy, arousing considerable media attention and a storm of protest from homeopaths.<ref>Shang A ''et al.'' (2005) Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. ''Lancet'' 366:726–32</ref> | ||
In 1999, the Government of [[Switzerland]], for a trial period of 5 years, allowed health costs for treatment with homeopathy and four other CAM modalities to be reimbursed under the country’s compulsory health insurance scheme, and set up a programme to evaluate the cost-effectiveness of these treatments (the Complementary Medicine Evaluation Programme (Programm Evaluation Komplementärmedizin, PEK). | |||
<ref>[http://biblio.parlament.ch/e-docs/139404.pdf The official PEK report] (In German, with summaries in German, French, Italian and English) "in the view of the authors of the meta-analyses, the available placebo-controlled studies on homeopathy do not demonstrate any clear effect over and above placebo" - from the English summary</ref><ref>Wallach H ''et al.''(2006) Summary consensus statement of the Review Board of the Swiss Complementary Medicine Evaluation Programme ''Homeopathy'' 95:28-30</ref> As part of this evaluation, a multi-disciplinary team, including a homeopath, set out to evaluate the evidence for the efficacy of homeopathy. The study, published by Shang ''et al.'', took a novel approach; whereas traditional [[meta-analysis|meta-analyses]] combine all studies of a single, given condition, this was a "global" meta-analysis testing the [[Scientific method|hypothesis]] that ''all'' effects of homeopathy are [[placebo]] effects. If so, the authors reasoned, then the predominance of positive homeopathy trial reports reflects [[publication bias]], and hence the magnitude of effects should diminish with sample size and study quality. They analyzed 110 placebo-controlled homoeopathy trials and 110 matched conventional-medicine trials. In both, effect size declined with improved study quality; however, some effect was still present in the largest and best conventional medicine trials, but not in the largest and best homeopathy trials. In the Shang ''et al.'' review, 21 homeopathic trials were judged of (relatively) “high quality”; these, overall, showed a benefit of homeopathic treatment. In the final analysis,<ref> "When the analysis was restricted to the larger trials of higher reported methodological quality, the odds ratio from random-effects meta-analysis was 0·88 (0·65–1·19) based on eight trials of homoeopathy and 0·58 (0·39–0·85) based on six trials of conventional medicine. Similarly, for prediction of treatment effects in trials as large as the largest trials, the odds ratio was 0·96 (0·73–1·25) for homoeopathy and 0·67 (0·48–0·91) for conventional medicine." Last paragraph of results section</ref> the researchers included only the eight largest studies; these showed that homeopathic treatment was comparable with a placebo, while six similarly large conventional trials were not compatible with a placebo effect. | |||
Thus the study exposed a problem with the reliability of small trials generally, but particularly small trials with methodological weaknesses; it's a problem that affects both conventional medical trials and trials of homeopathy equally in that trials that appear to produce a positive outcome are more likely to be published than trials which are inconclusive or produce a negative outcome. This problem, called "[[publication bias]]", reflects both the perceived unwillingness of academic journals to publish unexciting results, and the unwillingness of authors to write up 'uninteresting' results. It is compounded when weaknesses in trial design mean that statistical evaluations of outcomes are of doubtful validity. | |||
The authors concluded that homeopathy was no better than placebo, and suggested that no further research on homeopathy is necessary. The article was accompanied by an unsigned editorial titled “The end of homeopathy"<ref> Editorial (2005) The end of homeopathy ''Lancet'' 366:690 </ref> and another, signed, editorial.<ref>Vandenbroucke JP (2005) Homoeopathy and ‘the growth of truth’ ''Lancet'' 366:691–2</ref> | |||
==Criticism from supporters of homeopathy== | |||
The ''Lancet'' subsequently published critical correspondence, and received an open letter from the Swiss Association of Homoeopathic Physicians (SVHA).<ref> [http://www.homeopathy.org/research/editorials/Rutten.pdf Open letter] to the Editor of The Lancet from the Swiss Association of Homoeopathic Physicians (SVHA)</ref> which declared that none of the published trials of homeopathy had any value anyway: | |||
<blockquote>"The meta-analysis may be statistically correct. But its validity and practical significance can be seen at a glance: not one single qualified homoeopath would ever treat one single patient in clinical practice as presented in any of the 110 analysed trials! The study cannot give the slightest evidence against homoeopathy because it does not measure real individual (classical) homoeopathy. It confounds real homoeopathic practice with distorted study forms violating even basic homeopathic rules.”</blockquote> | |||
Other critics of the Shang ''et al.'' analysis noted that it involved subjective judgements of study quality. Several studies defined as "high quality" by Linde ''et al.'' (1997) were not defined by high quality by Shang ''et al.'' most of which showed a positive effect of homeopathic treatment.The Shang ''et al.'' analysis also excluded a relatively large study of chronic polyarthritis (N=176) because no matching trial could be found. The authors of an article in the ''Journal of Clinical Epidemiology''<ref>Ludtke R, Rutten ALB (2008) The conclusions of the effectiveness of homeopathy highly depend on the set of analyzed trials. ''J Clin Epidemiol'' doi: 10.1016/j.jclinepi.2008.06.015 and Rutten ALB & Stolper CF (2008) The 2005 meta-analysis of homeopathy: the importance of post-publication data. ''Homeopathy'' doi: 10.1016/j.homp.2008.09.008; see also Wilson P (2009) Analysis of a re-analysis of a meta-analysis: in defence of Shang et al. ''Homeopathy'' 98:127-8 for a refutation of the Rutten critique</ref> say "This result can be interpreted differently. Following Shang's perspective it can be explained by small study bias (which includes publication bias). In contrast, one may hypothesize that Shang's result is falsely negative." The authors noted that four of the 21 best trials selected by Shang ''et al.'' dealt with preventing or treating muscle soreness—these consistently found no benefits to homeopathy, so if it is accepted that homeopathy is not useful in this condition, the remaining 17 trials show an overall significant effect, mainly determined by two trials on influenza-like diseases. Thus they argue that it is possible that homeopathy might be effective for some conditions and not others. | |||
==References== | ==References== | ||
{{reflist|2}} | {{reflist|2}} |
Latest revision as of 08:03, 1 April 2011
![](http://s9.addthis.com/button1-share.gif)
In 2005, The Lancet published the results of an innovative meta-analysis that became the single most cited study of homeopathy, arousing considerable media attention and a storm of protest from homeopaths.[1]
In 1999, the Government of Switzerland, for a trial period of 5 years, allowed health costs for treatment with homeopathy and four other CAM modalities to be reimbursed under the country’s compulsory health insurance scheme, and set up a programme to evaluate the cost-effectiveness of these treatments (the Complementary Medicine Evaluation Programme (Programm Evaluation Komplementärmedizin, PEK). [2][3] As part of this evaluation, a multi-disciplinary team, including a homeopath, set out to evaluate the evidence for the efficacy of homeopathy. The study, published by Shang et al., took a novel approach; whereas traditional meta-analyses combine all studies of a single, given condition, this was a "global" meta-analysis testing the hypothesis that all effects of homeopathy are placebo effects. If so, the authors reasoned, then the predominance of positive homeopathy trial reports reflects publication bias, and hence the magnitude of effects should diminish with sample size and study quality. They analyzed 110 placebo-controlled homoeopathy trials and 110 matched conventional-medicine trials. In both, effect size declined with improved study quality; however, some effect was still present in the largest and best conventional medicine trials, but not in the largest and best homeopathy trials. In the Shang et al. review, 21 homeopathic trials were judged of (relatively) “high quality”; these, overall, showed a benefit of homeopathic treatment. In the final analysis,[4] the researchers included only the eight largest studies; these showed that homeopathic treatment was comparable with a placebo, while six similarly large conventional trials were not compatible with a placebo effect.
Thus the study exposed a problem with the reliability of small trials generally, but particularly small trials with methodological weaknesses; it's a problem that affects both conventional medical trials and trials of homeopathy equally in that trials that appear to produce a positive outcome are more likely to be published than trials which are inconclusive or produce a negative outcome. This problem, called "publication bias", reflects both the perceived unwillingness of academic journals to publish unexciting results, and the unwillingness of authors to write up 'uninteresting' results. It is compounded when weaknesses in trial design mean that statistical evaluations of outcomes are of doubtful validity.
The authors concluded that homeopathy was no better than placebo, and suggested that no further research on homeopathy is necessary. The article was accompanied by an unsigned editorial titled “The end of homeopathy"[5] and another, signed, editorial.[6]
Criticism from supporters of homeopathy
The Lancet subsequently published critical correspondence, and received an open letter from the Swiss Association of Homoeopathic Physicians (SVHA).[7] which declared that none of the published trials of homeopathy had any value anyway:
"The meta-analysis may be statistically correct. But its validity and practical significance can be seen at a glance: not one single qualified homoeopath would ever treat one single patient in clinical practice as presented in any of the 110 analysed trials! The study cannot give the slightest evidence against homoeopathy because it does not measure real individual (classical) homoeopathy. It confounds real homoeopathic practice with distorted study forms violating even basic homeopathic rules.”
Other critics of the Shang et al. analysis noted that it involved subjective judgements of study quality. Several studies defined as "high quality" by Linde et al. (1997) were not defined by high quality by Shang et al. most of which showed a positive effect of homeopathic treatment.The Shang et al. analysis also excluded a relatively large study of chronic polyarthritis (N=176) because no matching trial could be found. The authors of an article in the Journal of Clinical Epidemiology[8] say "This result can be interpreted differently. Following Shang's perspective it can be explained by small study bias (which includes publication bias). In contrast, one may hypothesize that Shang's result is falsely negative." The authors noted that four of the 21 best trials selected by Shang et al. dealt with preventing or treating muscle soreness—these consistently found no benefits to homeopathy, so if it is accepted that homeopathy is not useful in this condition, the remaining 17 trials show an overall significant effect, mainly determined by two trials on influenza-like diseases. Thus they argue that it is possible that homeopathy might be effective for some conditions and not others.
References
- ↑ Shang A et al. (2005) Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 366:726–32
- ↑ The official PEK report (In German, with summaries in German, French, Italian and English) "in the view of the authors of the meta-analyses, the available placebo-controlled studies on homeopathy do not demonstrate any clear effect over and above placebo" - from the English summary
- ↑ Wallach H et al.(2006) Summary consensus statement of the Review Board of the Swiss Complementary Medicine Evaluation Programme Homeopathy 95:28-30
- ↑ "When the analysis was restricted to the larger trials of higher reported methodological quality, the odds ratio from random-effects meta-analysis was 0·88 (0·65–1·19) based on eight trials of homoeopathy and 0·58 (0·39–0·85) based on six trials of conventional medicine. Similarly, for prediction of treatment effects in trials as large as the largest trials, the odds ratio was 0·96 (0·73–1·25) for homoeopathy and 0·67 (0·48–0·91) for conventional medicine." Last paragraph of results section
- ↑ Editorial (2005) The end of homeopathy Lancet 366:690
- ↑ Vandenbroucke JP (2005) Homoeopathy and ‘the growth of truth’ Lancet 366:691–2
- ↑ Open letter to the Editor of The Lancet from the Swiss Association of Homoeopathic Physicians (SVHA)
- ↑ Ludtke R, Rutten ALB (2008) The conclusions of the effectiveness of homeopathy highly depend on the set of analyzed trials. J Clin Epidemiol doi: 10.1016/j.jclinepi.2008.06.015 and Rutten ALB & Stolper CF (2008) The 2005 meta-analysis of homeopathy: the importance of post-publication data. Homeopathy doi: 10.1016/j.homp.2008.09.008; see also Wilson P (2009) Analysis of a re-analysis of a meta-analysis: in defence of Shang et al. Homeopathy 98:127-8 for a refutation of the Rutten critique