Approximately 96 COVID-19 vaccines are at various stages of clinical development.
1 At present, we have the interim results of four studies published in scientific journals (on the Pfizer–BioNTech BNT162b2 mRNA vaccine,2 the Moderna–US National Institutes of Health [NIH] mRNA-1273 vaccine,3 the AstraZeneca–Oxford ChAdOx1 nCov-19 vaccine,4 and the Gamaleya GamCovidVac [Sputnik V] vaccine)5 and three studies through the US Food and Drug Administration (FDA) briefing documents (on the Pfizer–BioNTech,6 Moderna–NIH,7 and Johnson & Johnson [J&J] Ad26.COV2.S vaccines).8 Furthermore, excerpts of these results have been widely communicated and debated through press releases and media, sometimes in misleading ways.9 Although attention has focused on vaccine efficacy and comparing the reduction of the number of symptomatic cases, fully understanding the efficacy and effectiveness of vaccines is less straightforward than it might seem. Depending on how the effect size is expressed, a quite different picture might emerge (figure; appendix).
ARR is also used to derive an estimate of vaccine effectiveness, which is the number needed to vaccinate (NNV) to prevent one more case of COVID-19 as 1/ARR. NNVs bring a different perspective: 76 for the Moderna–NIH, 78 for the AstraZeneca–Oxford, 80 for the Gamaleya, 84 for the J&J, and 117 for the Pfizer–BioNTech vaccines. The explanation lies in the combination of vaccine efficacy and different background risks of COVID-19 across studies: 0·9% for the Pfizer–BioNTech, 1% for the Gamaleya, 1·4% for the Moderna–NIH, 1·8% for the J&J, and 1·9% for the AstraZeneca–Oxford vaccines.
ARR (and NNV) are sensitive to background risk—the higher the risk, the higher the effectiveness—as exemplified by the analyses of the J&J’s vaccine on centrally confirmed cases compared with all cases:
both the numerator and denominator change, RRR does not change (66–67%), but the one-third increase in attack rates in the unvaccinated group (from 1·8% to 2·4%) translates in a one-fourth decrease in NNV (from 84 to 64).
There are many lessons to learn from the way studies are conducted and results are presented. With the use of only RRRs, and omitting ARRs, reporting bias is introduced, which affects the interpretation of vaccine efficacy.
When communicating about vaccine efficacy, especially for public health decisions such as choosing the type of vaccines to purchase and deploy, having a full picture of what the data actually show is important, and ensuring comparisons are based on the combined evidence that puts vaccine trial results in context and not just looking at one summary measure, is also important. Such decisions should be properly informed by detailed understanding of study results, requiring access to full datasets and independent scrutiny and analyses.
Unfortunately, comparing vaccines on the basis of currently available trial (interim) data is made even more difficult by disparate study protocols, including primary endpoints (such as what is considered a COVID-19 case, and when is this assessed), types of placebo, study populations, background risks of COVID-19 during the study, duration of exposure, and different definitions of populations for analyses both within and between studies, as well as definitions of endpoints and statistical methods for efficacy. Importantly, we are left with the unanswered question as to whether a vaccine with a given efficacy in the study population will have the same efficacy in another population with different levels of background risk of COVID-19. This is not a trivial question because transmission intensity varies between countries, affected by factors such as public health interventions and virus variants. The only reported indication of vaccine effectiveness is the Israeli mass vaccination campaign using the Pfizer–BioNTech product. Although the design and methodology are radically different from the randomised trial,
report an RRR of 94%, which is essentially the same as the RRR of the phase 3 trial (95%) but with an ARR of 0·46%, which translates into an NNV of 217 (when the ARR was 0·84% and the NNV was 119 in the phase 3 trial). This means in a real-life setting, 1·8 times more subjects might need to be vaccinated to prevent one more case of COVID-19 than predicted in the corresponding clinical trial.
Uncoordinated phase 3 trials do not satisfy public health requirements; platform trials designed to address public health relevant questions with a common protocol will allow decisions to be made, informed by common criteria and uniform assessment. These considerations on efficacy and effectiveness are based on studies measuring prevention of mild to moderate COVID-19 infection; they were not designed to conclude on prevention of hospitalisation, severe disease, or death, or on prevention of infection and transmission potential. Assessing the suitability of vaccines must consider all indicators, and involve safety, deployability, availability, and costs.
Some of the posts we share are controversial and we do not necessarily agree with them in the whole extend. Sometimes we agree with the content or part of it but we do not agree with the narration or language. Nevertheless we find them somehow interesting, valuable and/or informative or we share them, because we strongly believe in freedom of speech, free press and journalism. We strongly encourage you to have a critical approach to all the content, do your own research and analysis to build your own opinion.