By Henry RM
Misleading Research on the Safety of Electroconvulsive Therapy
Electroconvulsive therapy (ECT) is a long-standing medical practice that has been controversial for decades. While ECT practitioners insist the procedure is safe and effective, much of the research into ECT is of dubious quality, failing to record adverse events associated with the procedure and establishing efficacy based on very small randomized trials that do not meet modern standards of evidence-based medicine.
A recent example of questionable ECT research is a systematic review published in 2019 entitled “Major Adverse Cardiac Events and Mortality Associated with Electroconvulsive Therapy: A Systematic Review and Meta-analysis”. The study claims, as per its title, to document major cardiac adverse events and all-cause mortality. While it is difficult to summarize the flaws in this review briefly, a short list (solely related to mortality) would include:
1) claiming to capture all-cause mortality when the study clearly fails to do that
2) Finding 1 in 40 patients experience major cardiac events and for unexplained reasons misrepresenting it as 1 in 50 in the abstract and conclusion
3) Finding heart failure occurs in 2.4% of patients as an adverse event and then reporting an all-cause mortality rate that is not consistent with this finding, even were it the only adverse event to occur (heart failure is fatal in about 10% of patients 30 days after diagnosis)*
4) Citing voluntary hospital surveys and then assuming this will accurately capture all-cause mortality
5) Finding an all-cause mortality rate that is a genuine statistical impossibility given the age distribution of the patient population included
This latter claim of the study is the one I will be focusing on here.
The authors note in the abstract that they find “All-cause mortality was 0.42 [0.11 – 1.52] deaths per 1,000 patients”. Before accepting this statistic, it’s important to take a couple issues into consideration. Number one: that the mortality rate in any given year for most industrialized countries with an aged population hovers just below 1%, and two: the average age of patients who receive ECT is consistently in the late 50s / early 60s, a population which – all else being equal – tends to die at a higher rate than the population as a whole.
Keeping these two general statistics in mind, we can check how credible the reported numbers are in this meta-analysis by taking the number of deaths reported in the study and the total number of follow-up days for every patient and converting them to a mortality rate per year, which allows for an easy comparison to population mortality rates reported by the CDC and other organizations. Doing this, we find a mortality rate for the entire study of 804.2 per 100,000 per year, which seems quite low for a population with many elderly, medically ill patients having a last line medical procedure. A recent population-based study from Sweden found a mortality rate in 30 days after ECT that was the equivalent of 12,960 per 100,000 per year, or 16 times higher than the rate cited by the authors of this study. So how did they arrive at such a low estimate?
Looking at each individual study, we see that four of the largest ones, which account for 83% of the sample (61,447 patients), report a total of only 5 deaths, for a mortality rate of 99 deaths per 100,000 people per year. According to the CDC, this is a mortality rate you would expect to see in a population of mostly healthy 15-24 year-olds, not a group of very ill patients with an average age of 60. It seems neither the authors nor the journal noticed that this number is a statistical impossibility given the large sample of patients and follow-up time. The authors themselves note, correctly, that:
“For mortality estimates, pooled sample sizes included more than 75,000 patients and more than 680,000 treatments. Sample size of that magnitude provide robust estimates that approximate population-level incidence rates.”
Thus, given the large sample of mostly older adults and the incongruously low death rate, we can pretty safely conclude that this study did not capture all-cause mortality, and that the mortality rate reported here should not be used to communicate risks to patients or doctors.
It is a bit difficult to reconcile the authors’ use of advanced statistical methods in this study with the seeming lack of basic numeracy when evaluating the credibility of those findings. It’s clear from reading some of the larger studies in this meta-analysis that they make little effort to document any deaths that might occur in the weeks following ECT, and checking the results against official population mortality statistics confirms that skepticism of the result is warranted. Unfortunately, this study appears to contribute little to the evidence base about ECT safety, and serves only to mislead doctors and patients by giving an unrealistic impression of ECT safety.
*Remember, the authors claimed all-cause mortality associated with ECT was 0.42 per 1000. Yet even if heart failure was the only adverse event and occurred 2.4% of the time, as reported, expected mortality from heart failure alone would be around 2.4 per 1000 patients after 30 days of followup. See: https://www.nejm.org/doi/full/10.1056/nejmoa020265
- https://www.cambridge.org/core/journals/bjpsych-advances/article/electroconvulsive-therapy-for-depression-a-review-of-the-quality-of-ect-versus-sham-ect-trials-and-metaanalyses/8B8A6FBE2A609D43DFC77ED778F8F935 “Electroconvulsive therapy for depression: a review of the quality of ECT versus sham ECT trials and meta-analyses”
- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6300062/ “ Major Adverse Cardiac Events and Mortality Associated with Electroconvulsive Therapy: A Systematic Review and Meta-analysis”
- https://www.nejm.org/doi/full/10.1056/nejmoa020265 “Long term trends in the Incidence of and Survival with Heart Failure”
- https://worldpopulationreview.com/country-rankings/death-rate-by-country
- https://www.cdc.gov/nchs/data/statab/gm290-98.pdf
- Calculations: MACE ECT review mortality rate
Number of patients – 74,128
Deaths – 49
Years of follow-up – (74,128×30)/365 = 2,223,840/365 = 6092.7
Deaths per year = 6092.7/49 = 1 death per 124.34 years
Or 804.2 per 100,000 per year
- https://www.sciencedirect.com/science/article/pii/S0165032722011910 “Risk factors for mortality from medical causes within 30 days of electroconvulsive therapy”
****
Life After ECT offers a platform where people can express their different perspectives on electroconvulsive therapy and related topics. The opinions shared on this platform may not necessarily align with the views of Life After ECT Inc.
