For decades, randomized controlled trials (RCTs) have been considered the gold standard in medicine. The robustness of RCTs comes from the i) random allocation of volunteers to the treatment or control arm and ii) strict adherence to pre-determined endpoints and statistical plan. These two characteristics, combined with the large size of phase 3 RCTs, considerably reduce the risk of bias and provide the best estimate of a vaccine’s or medicine’s performance. Observational studies, where case and matched controls are selected a posteriori or based on known facts, also offer a view on a product performance but with lower confidence than an RCT. The danger of concluding on the benefit of a product based on observational trials was exemplified during the Covid-19 pandemic, when initial observational studies suggested that ivermectin and hydroxychloroquine were effective in preventing Covid-19 until rigorous RCTs proved unequivocally that this was not the case.
The danger of concluding on the benefit of a product based on observational trials was exemplified during the Covid-19 pandemic, when initial observational studies suggested that ivermectin and hydroxychloroquine were effective in preventing Covid-19 until rigorous RCTs proved unequivocally that this was not the case.
In our example of a recent review on the efficacy of newer influenza vaccines, the ECDC considered that an opaquely reported 512 participant, observational study was of an equivalent strength of evidence as a transparently reported, 31,989 participant, gold standard randomized controlled trial (RCT) using the data of two flu seasons.1 The former 512 participant trial was based on a subset of a larger study and utilized a modified, statistical analysis that favored the treatment group over control. This resulted in lifting the negative unadjusted relative vaccine efficacy of -92% to an adjusted relative vaccine efficacy of +59%.2,3 This naturally raises questions about the validity of the study, and how the ECDC could equate this study to an RCT.
We are not the only ones questioning the rigor and validity of this report, which sparked controversy within the scientific community. Physicians and scientific experts are questioning the quality and transparency of the evidence the ECDC used to underpin its recommendation.
Public health and investment in medical innovation are at stake
Flu does more damage than most people realize. It can lead to severe complications, increasing the risk of heart attack and pneumonia. Flu also puts a considerable strain on already stretched healthcare systems across Europe during the winter season. We believe that the ECDC equating the validity of observational and RCT trial findings at a time when we need to build trust in vaccines is a retrograde step that may place vulnerable people at greater risk of hospitalization.
Moreover, there is a possibility of a domino effect that destabilizes future investment in therapeutic innovations backed by the highest standards of scientific rigor and transparency. What incentive is there for the life sciences industry to keep investing in expensive RCTs if health agency recommendations signal that the gold standard is no longer necessary?
This post was originally published on here
