Originally published on Skeptic North 23 August 2010 (wayback link), this article was in response to another article (wayback link) I had written calling out Bryce Wylde on his claims about the usefulness of homeopathy in first aid kits. He responded to the first article with a Gish gallop of 21 articles that I’m sure he never expected anyone to look into. The below is a follow-up article I wrote calling this bluff:
___________________________________________________________________________
A few weeks ago, I wrote an article critical of Bryce Wylde’s appearance on Canada AM where he indicated that homeopathic treatments were of benefit for cuts, bruises, burns, and bug bites. Mr. Wylde responded in the comments section of that post (leading to further discussion here) where he provided a list of his “favourite scientific documents” for my review.
As Mr. Wylde took the time to respond to criticism with a list of citations that are his favourite, I must assume that he intends this list to be persuasive supporting data for homeopathy, if not the best data available. Given that he prides himself on his evidence-based practice and discusses homeopathy in regular media appearances, I expect that if there’s good data to support homeopathy, he would have it. I also expect that Mr. Wylde, in using scientific papers to validate his position, values the scientific method and recognizes that science is not inherently biased against homeopathy or ineffective in evaluating its effects.
Before reviewing the papers, let’s consider some background on evidence to ensure that we are all clear on what that word means in the context of science.
Evidence-Based Practice
There’s an apparent philosophical distinction between what homeopaths and what science advocates consider evidence-based (or science-based) practice. Perhaps this is why, after reviewing the same research, scientists and homeopaths have come to different conclusions regarding the efficacy of homeopathy. This point that was made after the British Homeopathic Association faced strong criticism for their interpretation of data in the Evidence Check just conducted in the United Kingdom. The house of commons committee noted:
73. We regret that advocates of homeopathy, including in their submissions to our inquiry, choose to rely on, and promulgate, selective approaches to the treatment of the evidence base as this risks confusing or misleading the public, the media and policy-makers.
In the view of homeopathy supporters, it seems that any literature or personal testimonial supporting the use of homeopathy is evidence of efficacy. However, the scientific view is that there are several factors that need be taken into account. Even if we set aside prior plausibility, the treatment has to be demonstrably and repeatedly effective in objective contexts with high quality research. A high-quality, objective study reduces as much bias as possible by employing certain standard methodologies and appropriate statistical analyses. It is important to sort out objective change from personal perception, because feeling better is not the same as affecting the course of an illness and the patient’s condition could worsen while they subjectively feel better.
In evidence-based practice, research (supporting and non-supporting) must be evaluated for both content and quality. Not to consider literature in context is colloquially called “cherry picking” and is undesirable because it gives a skewed representation of the data — any trial could be an outlier. A high quality approach to review a lot of data in a short time is to examine systematic reviews, such as Cochrane Reviews, that comprehensively summarize research on a particular topic.
With these principles in mind, each citation is reviewed below.
Literature Review
For the sake of brevity, the following critical appraisals are not exhaustive. The most obvious or problematic factors with the paper are noted, with links to further discussion where applicable. As Mr. Wylde provided no contextual analysis, it’s not clear why these papers in particular are his favourite. I invite him to provide this context in the comments, if he so desires.
Summary
None of the 21 provided citations had any direct relevance to the topic of first aid (the topic on Canada AM). Most of the studies’ conclusions were not representative of the literature, had inadequate statistical analysis/power, and/or had significant methodological flaws. Even the most remote positive results were reported enthusiastically by the authors, whereas negative results were downplayed or said to call for “further research” — despite reviews demonstrating negative overall results that are more pronounced with improved study quality. This pattern is not necessarily due to devious attempts at misrepresenting data; rather this can arise from unintentional investigator biases, hence the value of peer review and independent replication. For more information, see:
- Edzard Ernst’s analysis of homeopathy-related Cochrane reviews (cached here) and systematic reviews (here).
- The NHS Homeopathy Evidence Check (pdf) from the UK.
- A 2003 critical overview of homeopathy, concluding: “There is a lack of conclusive evidence on the effectiveness of homeopathy for most conditions. Homeopathy deserves an open-minded opportunity to demonstrate its value by using evidence-based principles, but it should not be substituted for proven therapies.”
Readers who are in a hurry can skip to the conclusion.
Clinical trials
Jacobs et al. (2003). Homeopathy for childhood diarrhea: combined results and meta analysis from three randomized, controlled clinical trials. Pediatric Infectious Disease Journal, 22: 229-234.
- Unfortunately homeopathy was not directly compared to oral rehydration therapy (standard treatment), only to placebo. So this study does not show that homeopathy has clinical usefulness for childhood diarrhea. Later investigation by the same authors in Honduras showed negative results. This systematic review of similar research shows a general lack of support for homeopathic treatments in children and adolescents.
Vickers et al. (2006). Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like syndromes (Cochrane review). In: The Cochrane Library. Chichester, UK: John Wiley & Sons, Ltd. CD001957.
- From that review: “Trials do not show that homoeopathic Oscillococcinum can prevent influenza. However, taking homoeopathic Oscillococcinum once you have influenza might shorten the illness, but more research is needed.” This is not exactly a glowing recommendation, particularly given that the average time shortened was about 6 hours. This paper is discussed in more detail here and here. Notably, a 2009 update to this review has been withdrawn.
Taylor et al. (2000). Randomised controlled trials of homoeopathy versus placebo in perennial allergic rhinitis with overview of four trial series. British Medical Journal, 321: 471-476.
- Public full text here. One of the main outcomes (visual analogue scale scores) of this study showed no significance. Criticisms of this study, among others, can be found here in a PubMed topic review of homeopathy for allergic rhinitis. For example, the statistical power used in the analysis was not appropriate for the number of participants. No conclusion can be drawn from this study, due to the lack of statistical power.
Frass et al. (2005). Adjunctive homeopathic treatment in patients with severe sepsis: a randomized, double-blind, placebo-controlled trial in an intensive care unit. Homeopathy, 94: 75–80.
- Only 35 participants were in each group (homeopathy and placebo). After 30 days, there was no significant difference. After 180 days there was a barely significant difference. Due to the small numbers, even 1-2 deaths would have significantly affected these results. Dr. Mark Crislip, an infectious disease specialist, reviews the paper here and questions the 6 month endpoint.
Oberbaum et al. (2001). A randomized, controlled clinical trial of the homeopathic medication Traumeel S in the treatment of chemotherapy-induced stomatitis in children undergoing stem cell transplantation. Cancer, 92: 684-690.
- Public full text here. Though the results of this single trial were significant, there were only 15 participants in each treatment group: 10 people in the Traumeel group developed stomatitis, whereas 14 people in the control group did. A 2009 Cochrane Review on the general topic of the adverse effects of cancer treatments, that included this paper, states that these trials require replication as “the risk of bias was unclear, and four further studies reported negative results.” The authors of the review conclude: “There is no convincing evidence for the efficacy of homeopathic medicines for other adverse effects of cancer treatments.” Homeopathic treatments for cancer are also discussed here.
Frei et al. (2005). Homeopathic treatment of children with attention deficit hyperactivity disorder: a randomised, double blind, placebo controlled crossover trial. European Journal of Pediatrics, 164: 758-767.
- Bias is suggested in the abstract, where the authors state that they aimed to “obtain scientific evidence of the effectiveness of homeopathy in ADHD”. Objective researchers should test the hypothesis of whether homeopathy is effective for ADHD. Perhaps unsurprisingly, this study showed significant results. However a 2009 Cochrane review on the topic of homepathy and ADHD, concluded: “There is currently little evidence for the efficacy of homeopathy for the treatment of ADHD.”
Brinkhaus et al. (2006). Homeopathic arnica therapy in patients receiving knee surgery: Results of three randomised double-blind trials. Complementary Therapies in Medicine, 14: 237-246.
- Of the three trials in this study of oral Arnica treatment, only one was significant and it had only 57 participants. There is not enough statistical power to draw firm conclusions and previous reviews of better-designed studies than this one have shown that homeopathic Arnica is not a promising avenue for acute treatment. This 1998 review states that Arnica is not supported beyond placebo effects; this 2001 review of more robust studies concurs.
Adler et al. (2009). Homeopathic Individualized Q-potencies versus Fluoxetine for moderate to severe depression: double-blind, randomized non-inferiority trial. Evidence-based Complementary and Alternative Medicine: eCAM. doi:10.1093/ecam/nep114
- This study had negative results, concluding only that the feasibility of such research was demonstrated. One would think that feasibility had already been sufficiently determined, given there were enough studies to populate these 2005 and 2007 reviews of similar research. Both conclude that homeopathy is not effective for treating depression, with one citing the low quality of available research.
Cost effectiveness
Cost, while relevant to economic impact, is not relevant to the efficacy of a particular treatment. Given the lack of objective established efficacy for homeopathy beyond the placebo effect, one wonders how any associated cost (beyond base ingredients) is ethically justified.
Rossi et al. (2009). Cost–benefit evaluation of homeopathic versus conventional therapy in respiratory diseases. Homeopathy, 98: 2-10.
- This was a non-blinded, non-random study of the treatments costs for patients of a homeopathic clinic compared to retrospective matched controls receiving conventional therapy. Insufficient evidence is provided to demonstrate that the groups were properly matched (e.g., diagnosis was not verified, which could bias the results if the homeopathy group was less ill on average than the conventional therapy group). The drug tracking methodology is unclear and homeopathy costs were not tracked. Therefore, the title (“versus”) is misleading and conclusions that costs were “reduced” in the homeopathy group are inappropriate. As there was no intervention, the only fair observation would be that costs “differed” between the two groups, which could be due to several factors. There is no evaluation of the appropriateness of treatments given, nor of the efficacy of homeopathy for respiratory illnesses (for that, see this review and this Cochrane review).
Witt et al. (2005). Outcome and costs of homeopathic and conventional treatment strategies: a comparative cohort study in patients with chronic disorders. Complementary Therapies in Medicine, 13: 79-86.
- Public full text here (pdf). “Health economic data were obtained for a subgroup of 38% of the patients.” – there was no explanation as to why 62% of the participants were excluded, even though these data are the primary endpoint. In any case, the study concluded: “In the present study, there were no significant differences between the overall costs incurred by patients according to the homoeopathic or conventional treatment strategies.” And the study “does not provide firm data on the comparative efficacy of conventional and homoeopathic treatments.”
Kneis et al. (2009). Economic evaluation of Sinfrontal® in the treatment of acute maxillary sinusitis in adults. Applied Health Economics and Health Policy, 7: 181-191.
- This paper is based on a previous trial that investigated Sinfrontal with apparent bias — the primary objective was to “demonstrate the efficacy of [Sinfrontal]”, as opposed to testing the hypothesis of efficacy. The methodology was questionable, as diagnosis of acute sinusitis is not reliably determined by x-ray and participants were allowed to use “saline inhalations, paracetamol, and over-the-counter medications” throughout the study. Either of these factors could potentially bias the results. The economic evaluation is also questionable, as antibiotics are not typically indicated for acute sinusitis. No solid conclusions can be reliably drawn from these two studies. Some criticisms of the rationale behind the methodology of the underlying trial can be found here.
Outcomes
Witt et al. (2005). Homeopathic medical practice: long-term results of a cohort study with 3,981 patients. BMC Public Health, 5: 115.
- Public full text here. The results were based on self- and homeopath-reported outcomes (1-10 scale) and quality of life measures at baseline and intervals. There were “major” improvements in quality of life for adults with severe disease and young children. The authors conclude that homeopathy may play a beneficial role in the long-term care of chronic patients. However, the study had no control group, so there’s no way to know whether these improvements were due to the homeopathic intervention, a placebo effect, or other variables.
Spence et al. (2005). Homeopathic treatment for chronic disease: a 6-year university-hospital based outpatient observational study. Journal of Alternative and Complementary Medicine, 5: 793-798.
- Patients self-reported (on a Likert scale) feeling better after homeopathic treatment. The authors conclude from these data that “Homeopathic intervention offered positive health changes to a substantial proportion of a large cohort of patients with a wide range of chronic diseases.” Unfortunately, they did not measure health changes. They measured subjective patient perception of health. Though interesting, this is not the same as an actual change in health outcome, as is implied by the conclusion. This study had no control group, so these improvements are indistinguishable from placebo effects.
Biological models
Belon et al. (2004). Histamine dilutions modulate basophil activation. Inflammation Research, 53: 181-188.
- This study is an attempt at replicating a previous study (that was later discredited) by some of the same authors. Two other independent attempts at replicating these findings were negative. This area of research has a controversial history, but the pattern appears to be that the effects seen by homeopathic researchers disappear with independent investigation.
Aguejouf et al. (2008). Prothrombotic and Hemorrhagic Effects of Aspirin. Clinical and Applied Thrombosis/Hemostasis, doi:10.1177/1076029608319945.
- Table 3 and Figure 3 from this animal study (rats) demonstrate quite clearly that the dilutions of Aspirin were no better than saline or salicylate. Also shown is that ASA, at a non-homeopathic dose of 100 mg/kg, significantly reduced the number and duration of emboli.
Witt et al. (2007). The in vitro evidence for an effect of high homeopathic potencies – A systematic review of the literature. Complementary Therapies in Medicine, 15: 128-138.
- The authors state that 75% of studies produced positive results, but from the conclusion: “No positive result was stable enough to be reproduced by all investigators. A general adoption of succussed controls, randomization and blinding would strengthen the evidence of future experiments.” In other words, though there were many positive studies found, they were not of high enough quality to be consistently replicated in order to draw solid conclusions from the data.
Endler et al. (2010). Repetitions of fundamental research models for homeopathically prepared dilutions beyond 10-23: a bibliometric study. Homeopathy, 99: 25-36.
- This paper was a literature search of studies about “high homeopathic potencies that have been subjected to laboratory-internal, multicenter or independent repetition trials”. Of the studies they included, almost a third did not support previous research and this went up to over half if the studies were independent replications. So what this paper might actually show is that there is bias in homeopathy research, highlighting the importance of independent replication. Also, repeatability is an irrelevant measure if both the original research and the replication are of poor quality (which wasn’t assessed).
Physics
The following three papers come from the same special issue of the journal Homeopathy, investigating the concept of “water memory”. The blog Bad Science discussed the entire series in an online journal club and copies of each paper can be found there.
Rey (2007). Can low temperature thermoluminescence cast light on the nature of ultra-high dilutions? Homeopathy, 96: 170-174.
- This study aims “to demonstrate that the high dilutions are physically different from the diluent and have, indeed, an ‘individual personality’.” There were no controls and the methods do not describe how many samples were analyzed. Graphs are presented with no accompanying statistical analysis. No justification is provided for the methods which do not resemble traditional homeopathic preparation processes. New Scientist discussed the article here.
Elia et al. (2007). The “memory of water”: an almost deciphered enigma. Dissipative structures in extremely dilute aqueous solutions. Homeopathy, 96: 163-169.
- The methods and materials used aren’t disclosed and the results contain no statistics, nor even a description of the number of trials run — for all we know, there was only one trial, making the results indistinguishable from chance. Consider this graph from the paper presented without units, error bars, or proper labeling. The authors state “we cannot derive reproducible information concerning the influence of the different degrees of homeopathic dilution or the nature of the active principle (solute) on the measured physicochemical parameters.” With such poor detail in their methods and results, lack of reproducibility is not surprising.
Chaplin (2007). The memory of water: an overview. Homeopathy, 96: 143-150.
- From the introduction: “whether homeopathy works or not is a mostly separate issue from the content of this paper … It follows that simply proving that water does have a memory does not prove that homeopathic medicines work.” This paper discusses how impurities might affect water’s structure, but this is not clearly linked to memory or increased potency after dilution, such as in homeopathic preparations. No rationale is given for why every dilute water solution doesn’t have potent effects due to memory of past solutes. More in depth discussion can be found here.
Other
Teixeira (2006). Evidence of the principle of similitude in modern fatal iatrogenic events. Homeopathy, 95: 229-236.
- This meta-analysis discusses the withdrawal effects and side-effects of pharmaceutical drugs. These are used to justify homeopathic treatment, as homeopathy is so dilute that the remedies produce no undesirable effects. However, though it is an interesting summary of the research on several pharmaceutical drugs, this paper does not establish homeopathic efficacy — undesirable drug effects provide no justification for the clinical use of homeopathy.
Conclusion
A review of this literature in broader scientific context demonstrates that the efficacy of homeopathy does not match that of available therapeutic interventions and it does not appear to be effective beyond the placebo effect. Positive effects are generally found in studies of poor quality that suffer from multiple methodological and analytical issues and these effects do not persist in higher quality studies. No evidence has been provided, nor does any appear to exist, to suggest that homeopathy is an appropriate or necessary intervention for either first-line or co- treatment among self-limiting, acute, or chronic conditions.
Mr. Wylde’s list of citations reinforces, rather than addresses, concerns about homeopathy. As a self-proclaimed evidence-based practitioner, he is presumably familiar with the principles of evaluating research and is open to peer review, an important aspect of scientific discourse. Yet he has chosen these papers apparently without an appreciation of the numerous limitations identified in each. In addition, he apparently failed to consider a number of negative well-designed studies and systematic reviews that are far more persuasive in their findings due to their adherence to objective scientific standards. One wonders why this research is apparently inapplicable or non-valuable to discussions about homeopathy.
There are many medical treatments that do not pan out in the long run, however few of them have the marketing and subsequent social support that homeopathy has. Therapies with an even less abysmal research history than homeopathy (for example, bloodletting and balancing the four humours, high dose chemotherapy for breast cancer, antiarrhythmic agents following a heart attack, etc) have been discarded for more promising avenues of treatment. Yet homeopathy remains due to persistent misunderstanding of the placebo effect, confusion between subjective assessment of illness and objective health outcome, and almost religious devotion in the face of copious non-supporting evidence.
Given appropriate evidence, I would of course re-evaluate my position. But because of the negative history of homeopathic research, this evidence would have to be relatively extraordinary. Until then, I remain skeptical and so should conscientious health consumers.
*Papers were gathered and reviewed by Kim Hebert and Scott Gavura. Special thanks to @hanna_louise, @psweetman, @xtaldave, @ethicsblogger, and @coxar for their assistance in the literature retrieval.
__________________________________________________________________________
Wylde responded on Twitter:
In the words of my colleague, Erik Davis:
So let me get this straight. Kim called Bryce out for touting unsubstantiated treatments on TV. Bryce responded to say, “They’re not unsubstantiated, here’s 21 citations that prove homeopathy works.” Kim, despite already knowing that the broader body of literature disagrees, suspended her disbelief and went painstakingly through ALL 21 PAPERS — only to find that they didn’t prove much of anything because they’re either off-point or so riddled with methodological errors as to have no evidentiary value. But rather than respond substantively, as she did, he just taunts her and says there are yet more citations.
After 21 papers — the ones he told you were his “favorite scientific documents” — I think he’s had a more than fair shot, and there’s no reason to believe that if you went through another 30 or 40 the picture would change. Nor would his advocacy, because in the end, he’s just proven that he really doesn’t care about evidence as long as the Vaughan Medical Centre thrives.
I started reading this thread doubting his science, and ended it doubting his motives. Those papers aren’t evidence, they’re shiny objects designed to mesmerize his patients — not meant to be read, but admired.
Science-Based Therapy 
