This study assesses the efficacy of the new BNT162b2 mRNA vaccine1 against Covid-19 in a nationwide mass vaccination scenario. The estimated vaccine efficacy during the follow-up period starting 7 days after the second dose was 92% for documented infection, 94% for symptomatic Covid-19, 87% for hospitalization and 92% for severe Covid-19. The estimated effectiveness during days 14 to 20 (after a dose) and days 21 to 27 (gradual change between the first and second doses of the vaccine) was 46% and 60% for documented infection, 57% and 66% for Covid -19, 74% symptomatic and 78% for hospitalization, 62% and 80% for severe Covid-19 and 72% and 84% for Covid-19-related death, respectively.
The first primary outcome assessed in the randomized trial of the BNT162b2 vaccine was symptomatic Covid-19. In both the randomized trial and our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and unvaccinated groups began to diverge around day 12 after the first dose.1 The estimated efficacy of the symptomatic Covid-19 vaccine from day 7 after the second dose was 95% in the randomized trial, compared with 94% in our study. The estimated effectiveness between the first and the second dose was 52% in the trial, compared to 29% in our study. This difference may reflect the high level of transmission in Israel during the study period,14 which affected both vaccinated people and controls equally during the first 12 days after the first dose. To eliminate this distortion, we estimated the effectiveness of the first dose of the Covid-19 vaccine for the period from days 14 to 20; the estimated effectiveness was 57%.
The estimated efficacy for documented infection during days 14 to 20 was 46% in our study. A relatively similar effectiveness of 51% has been reported by Chodick et al.,15 who evaluated a cohort from another health organization in Israel and used a different study design that compared infection between people vaccinated on days 13 to 24 after the first dose against infection during days 0 to 12.
In the randomized trial, the estimated vaccine efficacy for severe Covid-19 (89% over the entire study period) was based on only 10 cases. Our study recorded 229 severe cases of Covid-19 – 55 in the vaccinated group and 174 in the unvaccinated group – resulting in an estimated effectiveness of 62% for days 14 to 20 after the first dose, 80% for days 21 to 27, and 92% for 7 or more days after the second dose.
The large sample size in our study also allowed us to estimate the vaccine’s efficacy for specific subpopulations that the randomized trial was not powerful enough to assess. In the trial, the estimated efficacy for Covid-19 among people up to 55 years of age, over 55 and 65 years or more 7 days after the second dose was 94 to 96%. We were able to study more granular age groups and we estimated that the vaccine’s efficacy was similar for adults 70 years of age or older and for younger age groups in the same period.
The randomized trial estimated the vaccine’s efficacy for patients with one or more coexisting diseases according to the Charlson comorbidity index16 and specifically for patients with obesity or hypertension. These measures do not provide clarity as to efficacy in patients with several coexisting diseases. We estimated the effectiveness of the vaccine in relation to several numbers of coexisting diseases and found indications that the efficacy may be slightly lower among people with a greater number of coexisting diseases.
Two factors make the present study uniquely suited to assess the effectiveness of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical data, Covid-19 PCR test results (for the documented infection result) and follow-up of the patient data in the community (for the Covid-19 symptomatic result) and in the hospital environment (for all other results) – CHS maintained this integrated data repository for more than half of the Israeli population, and updated it daily, for more than two decades; and, second, the rapid pace and high acceptance of the Covid-19 vaccine in Israel and the high disease rates during the vaccination campaign. On the other hand, the fast pace of the vaccination campaign contributed to the frequent censorship of data for paired unvaccinated controls, especially among those over 60 (often only a few days after correspondence) and the corresponding reduction in the mean of follow-up. -up study period.
Concerns have been raised about the possible resistance of the SARS-CoV-2 variants to Covid-19 vaccines17.18 and neutralizing antibodies.19.20 During the study period, an increasing share of SARS-CoV-2 isolates in Israel – up to 80% in the days before data extraction – were variant B.1.1.7.21 Thus, this study estimates an average vaccine efficacy on several strains. Although we cannot provide a specific efficacy estimate for variant B.1.1.7, the plateau observed during later periods on the cumulative incidence curve for vaccinated persons suggests that the BNT162b2 vaccine is also effective for this variant, an observation consistent with reports that showed preserved neutralizing antibody titers.22 Variant B.1.351 was estimated to be rare in Israel at the time of data extraction.23
As with any observational study, our study may have been affected by residual confusion due to differences between vaccinated people and unvaccinated controls, especially in terms of health-seeking behavior. Therefore, we performed a rigorous comparison on a wide range of factors that can confuse the causal effect of the vaccine on the various results. After the matching process, we found a consistent pattern of similarity between the groups on the days immediately preceding day 12 after the first dose (the expected start of the vaccine’s effect), which therefore serves as a “negative control”24 time course (Figure 2, Fig. S6 and Table S7). This similarity occurred despite a temporary increase in events among unvaccinated controls during the first few days after the first dose of the vaccine, probably due to the fact that people who choose to be vaccinated on a specific day are feeling good at the time of vaccination. The similarity of the study groups in coexisting conditions and known risk factors for severe Covid-19 (table 1 and Fig. S2) provides further evidence of interchangeability (ie, no confusion). However, this rigorous matching process came at the cost of not including in the final cohort approximately 34% of the vaccinated people who would otherwise meet the study’s eligibility criteria. Correspondence limited by age and sex would have been insufficient to eliminate the initial confusion (Fig. S6).
We also exclude population groups with high internal variability in the likelihood of vaccination or outcome, such as health professionals, people confined to their homes for medical reasons and residents of nursing homes, to avoid residual confusion. Although the randomized trial is also less likely to include people who were not healthy enough to meet scheduled visits and the vaccination plan, it did not exclude health professionals.
To assess a possible selection bias that could result from informative censorship, whereby vaccinated controls feel good at the time of vaccination, we performed a sensitivity analysis in which they were kept in the unvaccinated group for a period of time that was defined differently for each result (Fig. S7 and Table S5). This analysis showed results similar to those of the main analysis, which suggests that any such bias was small in our analysis.
Finally, the symptom onset date was not available for analysis. Instead, for infection results, the date was defined as the smear collection date for the first positive PCR test. Given that there was probably a time lag between symptom onset and smear collection, the divergence observed in the cumulative incidence graphs for infection results between vaccinated people and unvaccinated controls may be slightly delayed. In parallel, there may be an underestimation of the vaccine’s effectiveness in each window of time, since the estimate actually reflects a narrower window for the vaccine to be active. Since the SARS-CoV-2 PCR test is highly accessible in Israel and can be done without reference in a matter of hours, we estimate this potential time interval and therefore the underestimation of the vaccine’s effectiveness is small. When interpreting the efficacy estimates for more severe outcomes, longer median intervals should be kept in mind (Fig. S3): 1 day for hospitalization, 5 days for severe Covid-19 and 11 days for Covid-19’s death.
This study estimates a high efficacy of the BNT162b2 vaccine in preventing symptomatic Covid-19 in an uncontrolled environment, similar to the vaccine efficacy reported in the randomized trial. Our study also suggests that the effectiveness is high for the most serious results: hospitalization, serious illness and death. In addition, the estimated benefit increases in magnitude over time. These results reinforce the expectation that newly approved vaccines can help mitigate the profound global effects of the Covid-19 pandemic.