What this is
- This research estimates the effectiveness of SARS-CoV-2 mRNA vaccines in preventing infections within the Borriana COVID-19 cohort in Spain.
- The study spans from January 2021 to August 2023, focusing on both symptomatic and asymptomatic cases.
- It employs two analytical approaches to assess vaccine effectiveness (VE) based on the type of infections reported.
Essence
- SARS-CoV-2 mRNA vaccines provide modest protection against infection and symptomatic COVID-19, with an effectiveness of 37% for infection and 50% for symptomatic cases after three doses.
Key takeaways
- Three doses of mRNA vaccines show a 37% effectiveness in preventing SARS-CoV-2 infection and 50% for symptomatic infection. The study indicates that while vaccination rates are high, the incidence of infection remains substantial.
- The study reports that 94.6% of participants received at least one vaccine dose, yet 75.1% of the cohort experienced infection post-vaccination, underscoring the ongoing challenge of viral transmission despite high vaccination coverage.
Caveats
- The study's sample size is relatively small, limiting the generalizability of the findings. Additionally, the absence of variant detection may affect the accuracy of the VE estimates.
- Potential biases may arise from retrospective exposure assessments and the observational nature of the study, which can complicate causal interpretations.
Simplified
1. Introduction
SARS-CoV-2 vaccines have been evaluated extensively for their effects on COVID-19 severity, hospitalizations, deaths, and long COVID incidence, highlighting their protective results [1,2,3,4,5]. However, SARS-CoV-2 infection has not been evaluated as extensively, considering the new virus variants and the waning immunity [6,7].
Quantifying SARS-CoV-2 vaccine effectiveness (VE) has been considered essential from a public health perspective to tackle the pandemic situation [8]. However, an estimation of VE presents some difficulties, requiring considering the demographic, clinical, and socio-economic context, follow-up of the vaccinated population, measuring SARS-CoV-2 exposures, and taking into account the high proportion of asymptomatic infections, the prevalence of infected populations before vaccination, SARS-CoV-2 variants, and the different SARS-CoV-2 vaccines and doses received. This complex situation could lead to bias in the VE estimation due to selection and confounding biases, among other reasons [9]. In addition, different effectiveness estimation methods have been employed [10].
The objective of this study was to estimate the VE of SARS-CoV-2 messenger RNA (mRNA) vaccines to prevent SARS-CoV-2 infections in the Borriana COVID-19 cohort during the period from January 2021 to August 2023.
2. Materials and Methods
2.1. Study Design
We carried out a prospective population-based cohort study of individuals who were infected during the Borriana COVID-19 Fallas festival in March 2020 in Borriana, a city in the Castellon province within the Valencia Community (Spain), where a large outbreak of COVID-19 took place, with an attack rate of 46% [11]. Several studies on this cohort with follow-up have been published from 2021 to 2025, considering the persistence of SARS-CoV-2 antibodies, complications, risk factors, long COVID, and cellular immunity [12,13].
In June 2022, a third study was performed on this cohort, including participants with laboratory-confirmed COVID-19; 722 of 1132 participants (63.8%) were included in the study. This study conducted serologic and questionnaire surveys to determine the SARS-CoV-2 infection and vaccination status. In addition, a follow-up of the cohort was implemented from June 2020 to August 2023 by examining their healthcare records. The present research covered the period from January 2021, when the SARS-CoV-2 vaccination campaign, to August 2023, when a survey of the primary healthcare record of each participant was conducted.
The inclusion criteria for this study were that participants had a negative laboratory-confirmed test of COVID-19 in previous surveys of this cohort (11–12) and that they did not suffer from the SARS-CoV-2 infection before the start of the SARS-CoV-2 vaccination campaign in January 2021. Asymptomatic SARS-CoV-2 cases were included if laboratory confirmation tests were performed after the start of the vaccination campaign in January 2021. Only participants with a first SARS-CoV-2 infection were considered, and reinfections were studied. Participants who suffered COVID-19 infection before the vaccination campaign were excluded, based on a follow-up of the cohort from 2020 onwards [11,12]. In addition, participants without laboratory-confirmed infection were excluded.
Two approaches were performed to estimate the VE of SARS-CoV-2 mRNA vaccines among the participants included in this study: all SARS-CoV-2 infections, including both symptomatic and asymptomatic cases (first approach), and only SARS-CoV-2 symptomatic infections (second approach). Symptomatic infections were defined by the presence of at least one illness symptom, including fever, coryza, cough, headache, myalgia, sore throat, weakness, loss of smell and/or taste, dyspnea, diarrhea, vomiting, and laboratory-confirmed tests. Asymptomatic infections were defined as the absence of any symptoms of SARS-CoV-2 infections, confirmed through laboratory tests. No SARS-CoV-2 infections were defined as the absence of any symptoms of SARS-CoV-2 infections and negative laboratory SARS-CoV-2 tests. SARS-CoV-2 cases were participants who suffered asymptomatic or symptomatic SARS-CoV-2 infections, and non-cases were participants who did not suffer SARS-CoV-2 infections. Figure 1 shows the flow diagram of this study.
To estimate VE, it is important to distinguish between asymptomatic and symptomatic cases, since it is unknown when asymptomatic cases suffered the infection, and there is a high proportion of asymptomatic cases from SARS-CoV-2 infections [14]. Therefore, whether the vaccination was before or after the infection cannot be determined. The first approach included all SARS-CoV-2 infections, and the second approach estimated VE based on symptomatic infections only. The use of these two approaches could estimate VE with higher precision by considering how viral transmission was passed over a more extended period of the pandemic, allowing us to gain better insight into the pandemic [15].
Considering the initial date of the vaccination campaign in January 2021 and the final date of the study in August 2023, the follow-up time of this cohort was 36 months. In this period, the predominant SARS-CoV-2 variants in the study area were Alpha from January 2021 to July 2021, Delta from August 2021 to December 2021, and finally Omicron from January 2021 to August 2023 [16].
2.2. SARS-CoV-2 Vaccination
The onset of the SARS-CoV-2 vaccination began in January 2021, and the last vaccinations took place in February 2022. Then, a participant was considered vaccinated 14 days after receiving one, two, or three vaccine doses [17]. To estimate the number of unvaccinated participants, medical and vaccination records were consulted.
Regarding the type of SARS-CoV-2 vaccines, most participants in the cohort received (228 participants) mRNA vaccines, including Pfizer-BioNTech (BNT162b2) (Pfizer, New York, NY, USA) (Comirnarty®) for the first and second doses, and Moderna (mRNA-1273) (Moderna, Cambrige, MA, USA) (Spikevax®) for the third dose. In addition, the AstraZeneca (AZD1222) (AstraZeneca, Cambridge, UK) (Vaxzevria® Covishiel®) vaccine was administered to 54 participants for the first and second doses. A first dose of the Janssen® SARS-CoV-2 vaccine (Ad26.CoV2.S) (Johnson and Johnson Innovative Medicine, Titusville, NJ, USA) (Jcovden®) was administered only to 3 participants. A third dose of the SARS-CoV-2 mRNA vaccine was administered to all participants. Information about vaccine brand and vaccination dates was obtained by consulting the population-based Valencia Region Vaccine Information System.
2.3. Laboratory Confirmation Tests for SARS-CoV-2 Infections
Laboratory confirmation tests for SARS-CoV-2 infections for the cohort began in May 2020 [11] with the determination of antibodies against SARS-CoV-2 nucleocapsid protein N via electrochemiluminescence immunoassay [18] (Roche Diagnostics, Mannheim, Germany). In the serologic survey conducted in June 2022, anti–SARS-CoV-2 spike IgG antibodies and IgG and IgM anti-nucleocapsid antibodies were detected via chemiluminescence microparticle immunoassay (CMIA AlinityI series, Abbot, Chicago, IL, USA) [19]. During the follow-up of this cohort, for the detection of the SARS-CoV-2 virus, reverse transcription–polymerase chain reaction (RT-PCR) was employed on different platforms from various brands: Genexpert, Roche Diagnostics, and Simplexa (Barcelona, Spain), as well as rapid antigen tests (RATs). Serology tests and the RT-PCR were performed by the Laboratory of Microbiology Service of the University Hospital de la Plana (Vila-real, Spain), and RATs by the participants at home.
The laboratory tests presented the following sensitivity and specificity values: CMIA, 96–98% sensitivity and 100% specificity [18]; RT-PCR, 99–100% sensitivity and 99–100% specificity [20]; and RATs (according to different brands frequently used in Spain), 80.2–96% sensitivity and 98.9–99.6% specificity [21]. SARS-CoV-2 infection status was determined for positive cases using the following laboratory tests: CMIA, 136 cases (60.2%); RT-PCR, 17 cases (7.5%); and RATs, 73 cases (32.3%). The status for non-cases was determined using CMIA, confirming 75 cases (100%).
2.4. Research Surveys
Three surveys were performed from June 2020 to June 2022 [22]. Information was obtained through specific questionnaires about socio-demographic characteristics; cohabitants at home; chronic disease; body mass index (BMI) (kg/m2); lifestyle, including smoking habits, alcohol consumption, and physical exercise; history of SARS-CoV-2 infections, including date of onset; and risk factors for SARS-CoV-2 exposure, including family member COVID-19 cases, exposure to other people at work, restaurant/bar attendance, and face mask wearing. Face-to-face interviews and telephone surveys were conducted by health staff from the Primary Care Health Center of Borriana and Vila-real, the Emergency Service of University Hospital de la Plana, and the Public Health Center of Castellon.
In addition, the primary healthcare record of each participant was reviewed and revised accordingly, and the computerized application outpatient care of the Valencia Health Agency (ABUCASIS) was queried from January 2020 to August 2023 to detect new COVID-19 cases, reinfections, and disease evolution.
2.5. Statistical Methods
Statistical methods for the two approaches were similar. Means, standard deviations, and percentages were used to describe the characteristics of the studied population. The cumulative incidence rate was estimated by dividing the SARS-CoV-2 cases by the total participants, and the 95% confidence interval (CI) was estimated according to a binomial distribution. Relative risk (RR) was defined as the ratio between the incidence rate of SARS-CoV-2 infection in exposed participants and the incidence rate of SARS-CoV-2 in unexposed participants, considering the different variables. Crude and adjusted relative risk (aRR) were estimated, with 95% CI.
SARS-CoV-2 infection and SARS-CoV-2 mRNA vaccination were the dependent and predictor variables, respectively, with several independent variables adjusted in the analysis using robust Poisson regression models [23]. These models exhibited robust standard errors that could correct for the high variability in the data. All models fitted well based on the Pearson and chi2 tests. Computer outputs are found in Supplementary S1.
Vaccine effectiveness (VE) was calculated as (1-RR) × 100 [24] from the aRR. To control for potential confounding factors, Directed Acyclic Graph (DAG) methodology was conducted [25,26] using the DAGitty version 3.1 program [27] (Figure 2). Confounding factors included age, sex, cohabitants at home, social class (considering upper and middle class (I–II) versus lower class (III–VI)), lifestyles (smoking, obesity as BMI ≥ 30 kg/m2, alcohol consumption, physical exercise), chronic diseases, and SARS-CoV-2 exposures (family member with COVID-19, exposure to other people at work, visiting restaurants/bars, face mask wearing) [28]. A crude RR of each of these factors in relation to SARS-CoV-2 infection was determined. In addition, adjusted VE was estimated for stratification, considering sex, age 50 years and above, and the presence of chronic diseases. The Stata® version 14 program(StataCorp., College Station, TX, USA) was used for all calculations.
2.6. Ethical Issue
This study was submitted to and approved by the Ethics Committee of University Hospital de la Plana (14 October 2021; registry number: 2961). All participants, or their parents in the case of minors, provided informed written consent to be included in the study.
3. Results
3.1. Follow-Up of the Borriana COVID-19 Cohort from March 2020 to August 2023
Participants of the initial Borriana COVID-19 cohort with laboratory-confirmed COVID-19 (n = 1132) were invited to participate in the June 2022 survey, and 722 were enrolled, yielding a participation rate of 63.8% (722/1132). From March 2020 to August 2023, 644 participants were infected with SARS-CoV-2, as confirmed by a laboratory test. Three participants had suspected infection but without laboratory confirmation, and 75 participants did not suffer an infection, yielding a cumulative incidence rate of 89.2% (95% CI: 86.7–91.4%) (644/722).
Participants who suffered a SARS-CoV-2 infection before the vaccination campaign in January 2021 and participants without a laboratory-confirmed test were excluded. From January 2021 to August 2023, out of 301 participants, 226 participants suffered a SARS-CoV-2 infection and 75 participants did not. Among the cases, 73 were asymptomatic and 153 were symptomatic. The cumulative incidence rate was 75.1% (95% CI: 69.8–79.9%) (226/301).
Clinical characteristics of the symptomatic cases included fever (59.4%), coryza (46.2%), cough (45.8%), headache (43.4%), myalgia (38.5%), sore throat (34.3%), weakness (33.6%), loss of smell and/or taste (14.2%), dyspnea (11.9%), diarrhea (9.2%), and vomiting (6.3%). The average disease duration was 5.7 ± 4.2 days, with a median of 5.0 days (rank 1–30). Long COVID sequelae were reported in 28 cases (12.4%). A total of 102 cases sought medical assistance (66.7%), and 6 cases were hospitalized due to COVID-19 (3.9%). In the study period, one 72-year-old woman died two months after the onset of SARS-CoV-2 infection, but the cause was not associated with the infection.
The temporal distribution of the 153 cases by symptom onset is shown in Figure 3. The onset of the first COVID-19 case occurred in May 2021, with an increase in cases from July to December 2021 (18.2%), a peak of cases in January 2022 (39.6%), followed by a slow decline up to August 2022 (37.7%), and the last case occurred in August 2023. Most cases were probably caused by Omicron variants.
Crude RRs were estimated to compare SARS-CoV-2 cases and non-cases (Table 1). Cases were significantly younger than non-cases, with the proportion of subjects aged 50 and over being lower. Males presented a lower risk compared with females (RR = 0.87; 0.95% CI: 0.75–0.84). Social class, obesity, and chronic disease were not risk factors for SARS-CoV-2 incidence. Alcohol consumption and physical exercise also had no effect on SARS-CoV-2 incidence. In contrast, smoking status had a protective effect (RR = 0.76; 95% CI: 0.60–0.97). A high number of cohabitants at home (RR = 1.08; 95% CI: 1.02–1.16) and exposure to other people at work (RR = 1.20; 95% CI: 1.00–1.43) were risk factors for infection. Even though having a family member with COVID-19 and visiting restaurants and bars presented an increased risk of SARS-CoV-2 infection, their effect was not significant. Face mask wearing was a protective factor, but it was not significant.
3.2. The First Approach Included Participants Vaccinated Before Suffering from a SARS-CoV-2 Infection
This study included 301 participants, with 226 SARS-CoV-2 cases and 75 non-cases. Among the cases, 73 were asymptomatic and 153 were symptomatic. In the Borriana COVID-19 cohort, SARS-CoV-2 vaccination began in January 2021, and 57.9% (418/722) of the cohort had already suffered from COVID-19 disease. Table 2 presents the results of SARS-CoV-2 vaccination in cases and non-cases. Of the 301 participants, 285 (94.7%) were vaccinated against COVID-19 disease with at least one dose, 275 (91.4%) with two or three doses, and 165 (54.8%) with three doses. Only 16 participants were not vaccinated. SARS-CoV-2 mRNA vaccines were exclusively administered to 228 participants (80%), and 57 participants (19%) received mRNA vaccines and other SARS-CoV-2 vaccines (AstraZeneca and Janssen). This highlights the high proportion of vaccinated individuals (92.9%) among SARS-CoV-2 cases. However, vaccination was a significant protective factor against SARS-CoV-2 infection, specifically with at least one dose (RR = 0.74 95% CI 0.69–0.79), with a complete vaccination (two to three doses) (RR = 0.72; 95% CI: 0.68–0.78), and with a booster vaccination (three doses) (RR = 0.68; 95% CI: 0.59–0.77). Comparing the number of vaccine doses administered, two and three doses of vaccines were protective against SARS-CoV-2 infection, with (RR = 0.89; 95% CI: 0.83–0.95) and (RR = 0.62; 95% CI: 0.55–0.70), respectively. No significant differences were found when comparing types of vaccines.
The adjusted relative risks (aRR) of SARS-CoV-2 vaccine doses and VE are presented in Table 3. The ARRs showed that vaccination was significantly protective against SARS-CoV-2 infections, considering vaccinated versus unvaccinated (aRR = 0.78; 95% CI: 0.63–0.96), vaccinated 2–3 doses versus 0–1 dose (aRR = 0.82; 95% CI: 0.70–0.95), and vaccinated with 3 doses versus 0–1–2 doses (aRR = 0.71; 95% CI: 0.61–0.82). Considering the number of vaccine doses, three doses were significantly protective (RR = 0.63; 95% CI: 0.61–0.82), whereas two doses were not significantly protective. The VE was 22% in vaccinated versus unvaccinated individuals (95% CI: 4–37); 18% in individuals vaccinated with 2–3 doses (95% CI: 5–30); and29% in individuals vaccinated with 3 doses (95% CI: 18–39). Regarding the number of vaccine doses, three doses had a VE of 37% (95% CI: 22–49), and two doses had a VE of 11% (−3–24).
The results of SARS-CoV-2 vaccine effectiveness stratified by sex, chronic disease, and age of 50 years and over are presented in Table 4. Males had a lower aRR for suffering from a SARS-CoV-2 infection than women (0.62 versus 0.74), and their VE was higher, at 38% versus 26%. Participants with a chronic disease had a lower aRR than participants without a chronic disease (0.60 versus 0.74), and their VE was higher, at 40% versus 26%. Participants aged 50 years and over had a lower aRR than participants aged less than 50 years (0.51 versus 0.77) and a higher VE of 49% versus 23%.
3.3. The Second Approach Excluded Participants with Asymptomatic SARS-CoV-2 Infections
Among the 226 SARS-CoV-2 cases in the first approach, 153 participants suffered from a symptomatic SARS-CoV-2 infection. In 73 participants, the infections were asymptomatic, and thus, these participants were excluded from this analysis. Asymptomatic cases represented 32.3% of cases (73/226), and among them, 66 participants were vaccinated (90.4%). In the second approach, there were 228 participants, including 153 cases and 75 non-cases.
The non-cases were significantly older than the cases, and males had a lower risk of SARS-CoV-2 infection (Table 5). Social class, chronic disease, alcohol intake, obesity, and physical exercise were not risk factors for infection. However, current smoking was a significant protective factor (RR = 0.76; 95% CI: 0.60–0.97). The number of cohabitants at home (RR = 1.14; 95% CI: 1.04–1.25) was a risk factor, as well as exposure to other people at work (RR = 1.30; 95% CI: 1.01–1.68). Having a family member with COVID-19 and visiting restaurants/bars increased the risk of infection, though their effect was not significant, and face mask wearing was not a significant protective factor.
Of the 228 participants, 219 were vaccinated, and 153 were cases while 75 were non-cases (Table 6). Only nine cases were unvaccinated. Vaccination was significantly protective against SARS-CoV-2 symptomatic infections, specifically with at least one dose (RR = 0.66; 95% CI: 0.60–0.72), with two or three doses (RR = 0.64; 95% CI: 0.58–0.71), and with three doses (RR = 0.51; 95% CI: 0.42–0.64). Considering the number of doses, three doses yielded higher protection (RR = 0.46; 95% CI: 0.38–0.56) than two doses (RR = 0.87; 95% CI: 0.81–0.94). No differences were observed between receiving an mRNA vaccine alone versus receiving mRNA and other vaccines.
The adjusted RR and VE of SARS-CoV-2 vaccines for the symptomatic cases are presented in Table 7. Receiving one dose of SARS-CoV-2 vaccine did not protect against symptomatic SARS-CoV-2 infection (aRR = 0.81; 95% CI: 0.53–1.22); the same was observed for vaccination schedules with two and three doses (aRR = 0.82; 95% CI: 0.67–1.01), but three-dose vaccination was a significant protective factor (aRR = 0.54; 95% CI: 0.44–0.68), with a VE of 46% (95% CI: 32–56). Considering the number of doses, three doses were significantly protective against symptomatic SARS-CoV-2 infection (aRR = 0.50; 95% CI: 0.37–0.67), with a VE of 50% (95% CI: 33–63). Two doses of vaccine were not significantly protective against symptomatic SARS-CoV-2 infection.
The results of SARS-CoV-2 vaccine effectiveness for symptomatic cases stratified by sex, chronic disease, and age of 50 years and over are presented in Table 8. Males had a lower aRR than women (0.38 versus 0.61) and a higher VE (62% versus 39%). Participants with a chronic disease had a lower aRR than participants without a chronic disease (0.48 versus 0.55) with a higher VE (52% versus 45%). Participants aged 50 years and over had a lower aRR than participants under 50 years old (0.40 versus 0.61), and their VE was higher (60% versus 39%).
3.4. Comparisons of aRR and VE Between the First and Second Approaches
Comparisons of aRR and VE between the first and second approaches are presented in Table 9. In both approaches, vaccination with three doses significantly reduced the risk of SARS-CoV-2 infections, with an aRR of 0.63 (first approach) and 0.50 (second approach), and the VE was 0.37%, and 0.50%, respectively. In the two approaches, vaccination with only one or two doses was not protective. After stratification, the second approach reduced the aRR compared with the first approach, especially in male participants and participants aged 50 years and over. The VE increased with the second approach.
4. Discussion
Three years after the beginning of the COVID-19 pandemic in 2020, 89.2% of the cohort suffered a SARS-CoV-2 infection, and 75.1% after the vaccination campaign started in January 2021, showing the magnitude of the pandemic. Despite a high vaccination rate with mRNA vaccines in most participants, with 94.6% receiving at least one dose and 54.8% receiving three doses, the incidence of SARS-CoV-2 infection was high. However, in both approaches, vaccination gave significant protection against SARS-CoV-2 infection when three doses were administered. In the first and second approaches, the VE was 37% for infection and 50% for symptomatic infection, respectively. These two VE values are modest, but in line with the increase in VE in cases with severe illness, hospitalizations, and deaths reported in cohort studies; however, in the Omicron wave, severe outcomes decreased [29,30,31,32]. Moreover, a reduction in the VE of mRNA vaccines related to the number of doses during the Omicron wave was found in previous cohort and meta-analysis studies [33,34]; three doses significantly decreased the risk of infection and symptomatic infection, but two doses were not sufficient to reduce the risk.
The VE values estimated by the two approaches may reflect the spectrum in which VE moves. The VE of the first approach should be close to the true value of the VE that occurred in the community since it included all cases with an infection. In the second approach, the VE included cases with symptomatic infections only. In this regard, Williams and coauthors [35] mentioned that the inclusion of asymptomatic cases reduces the VE in vaccines that prevent disease but not infection. In test-negative design studies of VE, the inclusion of asymptomatic cases could bias the VE [36].
Regarding three vaccine doses, results of the two approaches with stratification highlight a higher VE in males, participants with chronic diseases, and participants 50 years old and above. This could correspond with a higher incidence of Omicron variant infections in young people [37]. As a consequence, the VE is higher in older participants and participants with chronic conditions, although the vaccine response decreases with age and chronic conditions [38]. A higher VE in males versus females has also been found in other cohort studies, but the reasons are not well understood [39], since the immune response is lower in males [40].
To understand our results, it is important to consider the long duration of the follow-up and the SARS-CoV-2 variants during this period, with the Omicron variant being predominant for most of the study period. Two variables, time from vaccination and virus variants, are crucial for SARS-CoV-2 risk infection quantification and VE. In initial clinical trials of SARS-CoV-2, the VE during the first 6 months was very high, reaching 96%, and declined progressively afterwards. The protection against infection declined 20–30 percentage points [2,41] 6 months after vaccination. In addition, new variants of SARS-CoV-2 led to an increase in breakthrough infections in vaccinated populations [42,43]. Other factors could also contribute, including a decrease in non-pharmacological measures against infection 2 years after the onset the COVID-19 pandemic and vaccine reluctance [44]. Very high SARS-CoV-2 vaccination rates might be necessary to stop SARS-CoV-2 transmission, considering reproductive numbers between 5.8 and 6.1 [45]. In addition, vaccine breakthrough infections allowed for continued transmission, given that vaccinated cases had viral loads similar to unvaccinated cases [46], although infection duration could be shorter in vaccinated subjects [47].
In this context, comparing our results with those of other cohort studies reveals that the effectiveness of mRNA vaccines in preventing SARS-CoV-2 infection after the first epidemic period from 2020 to 2021 decreased against the Omicron variant. Regarding symptomatic SARS-CoV-2 cases, the VE was 47.9% (95% CI: 41.85–53.75) in a review study [48]. Among healthcare workers in a clinic in Cleveland, the decline in VE was from 49% to 19% across the periods considered [49]. In Spain, Monge and co-authors found VE values of 52·5% (95% CI: 51.3–53.7%) for an mRNA-1273 booster and 46.2% (95% CI: 43.5–48.7%) for a BNT162b2 booster in a national cohort [50]. In Hong Kong, the BNT162b2 vaccine was protective against asymptomatic and symptomatic SARS-CoV-2 Omicron infections with a VE of 41.4% (95% CI 23.2–55.2%); the VE of BNT162b2 boosters was 50.9% (95% CI 31%·65%) for symptomatic Omicron infections [51]. In Japan, a cohort study on a general population of 16- to 64-year-olds found that the mRNA vaccine had a VE of 71.8% (95% CI: 60.1–80.1%) for a third dose against symptomatic infection during the Omicron period [52]. In Canada, a cohort study on healthcare workers found a VE of 43% (95% CI: 29–54%) for three doses, and 56% (95% CI: 42–67%) for four doses, after adjusting for previous infection and other covariates [53].
In our study, we controlled for potential risk factors, including demographics, lifestyles, virus exposure, and prevention measures, as recommended for VE studies [54,55]. However, some authors reported that exposure had little effect on VE [17]. Regarding related factors of SARS-CoV-2 infections, our study found that older and male participants had a lower risk of infection. On the contrary, the number of cohabitants at home and exposure to other people at work increased the risk of infection, while current smoking was a protective factor. Other lifestyle factors and preventive measures against virus transmission had no effect on the risk of infection. Some of these factors have been reported in studies on SARS-CoV-2 epidemiology, such as a higher risk of infection for female sex and larger household size [56], young age [57], exposure to other people at work as a proxy for working outside the home [58], and certain education and social care occupations [59]. Current smoking was a protective factor, in line with other studies [60]. Wearing a face mask was not protective, which is in line with a prospective cohort study in Germany [61], but this factor was found to be protective in other studies [62]. Moreover, obesity was not a risk factor, in contrast to what was found in England [63].
To explain the low VE in preventing SARS-CoV-2 infection, some studies have indicated that the intramuscular injection of mRNA vaccines could be the cause of low IgG antibody levels in the upper respiratory tract, in contrast with higher antibody concentrations in the circulation [64]; other factors may be the high frequency of SARS-CoV-2 variants, including the Omicron variant with a predilection for the upper respiratory tract, and the rapid decrease in neutralizing antibodies against this variant with a need for vaccine boosters to increase these antibodies [65,66]. Regarding the reduction in severe events, its biological mechanism is not completely understood. In addition to neutralizing antibodies with immune memory T cells and B cells to control the disease, a hypothesis has been proposed that mRNA vaccines reduce the activation of alveolar macrophages, decreasing the release of cytokines, with some side effects [67].
The study of VE is critical for the implementation of strategies to control and prevent COVID-19 epidemics, but it is subject to a high likelihood of bias, considering the design, performer, and results. Several reviews have mentioned these issues, including case–control, test-negative, and cohort studies [68,69,70]. In addition, the estimation of the VE duration still requires accurate analysis [36]. Our study has strengths and limitations. Regarding strengths, this study employed a prospective, population-based cohort design with long follow-up, considering laboratory-confirmed cases; it used an official vaccine register, controlled for potential confounding factors, and employed two approaches to measure the VE of SARS-CoV-2 vaccines, one including all incident cases and one including only symptomatic cases. Regarding the laboratory tests used to determine the SARS-CoV-2 infection status, CMIA was used in 70.1% of participants, RT-PCR in 5.6%, and RATs in 24.3, with the last test showing inferior sensitivity. However, the sensitivity and specificity of all tests conformed to the recommendations of the World Health Organization, such as sensitivity ≥80 and specificity ≥97–100% [71]. With respect to the adjusted VE of three doses for the two approaches, we calculated the E-values [72] to be 2.55 (upper limit of 1.89) and 3.41 (upper limit of 2.35) for the first and second approach, respectively, indicating a low magnitude of unmeasured confounding, which is in line with other studies on VE [73].
This study's limitations are as follows: First, the number of participants in the cohort is small. Second, SARS-CoV-2 variants were not detected. Third, exposure measures were determined through a questionnaire retrospectively in some infected cases. Fourth, non-cases might have suffered a SARS-CoV-2 infection after the June 2022 survey, but the healthcare records of each participant were revised up to August 2023, and the survey of a random sample of 225 participants to study the cellular immunity of SARS-CoV-2 [13] could reduce this quantity. Fifth, there was a reduction in statistical power in the second approach. Sixth, the analysis of potential risk factors was not adjusted. Seventh, a portion of participants received two types of vaccines against SARS-CoV-2. Eighth, some other residual confounding factors could remain. Ninth, the decline in IgG and IgM anti-nucleocapsid antibodies over time in SARS-CoV-2 cases could increase the VE, since some asymptomatic cases could be non-cases. Tenth, given their lower sensitivity, RATs might not have detected all infected cases. Eleventh, COVID-19 is a new disease, and some issues might not have been considered in this study. Finally, the design of our study is observational and has inherent limitations compared to clinical trial designs [9].
Regarding the interpretation of our results, which show modest but significant protection against SARS-CoV-2 infection after three doses of vaccination, following Austin Bradford Hill's causality criteria [74], a causal relationship, rather than association, could be suggested between mRNA vaccine and SARS-CoV-2 infection, considering the prospective design, consistency with other studies, biological plausibility, coherency, dose–response, and specific immune response; in addition, confounding factors were controlled. However, it is necessary to account for the fact that our study was observational with possibility of bias.
The administration of mRNA vaccines against SARS-CoV-2 infection contributed to a substantial reduction in the impact of the pandemic, decreasing the incidence of severe events, but viral transmission could not be completely prevented. Our VE results should be considered in a context where reduced risk of disease via vaccination is evident, but vaccine failure is present and viral transmission continues. An outbreak ends when a high proportion of exposed people are infected, and this situation needs to be considered in vaccination campaigns [75]. In addition, there are controversies regarding the effectiveness of vaccination campaigns and the safety of vaccines [76,77,78]. In this regard, the follow-up of this cohort could provide useful information regarding the evolution of the disease, vaccinations, and side effects.
To address the activity and high-frequency mutation of the virus with a decrease in VE, several measures need to be considered, including surveillance of the incidence of SARS-CoV-2 infections and virus variants, research on the biology of the virus and immune response of vaccines, and vaccination of the at-risk population [79,80]. The WHO recommends a single-dose regimen for primary vaccination against COVID-19, suggesting that the monovalent Omicron XBB vaccines could provide better protection than other vaccines, and revaccination for groups with severe disease and death [81,82]. In Europe, people over 60 years of age, people with immunodeficiency, and pregnant women should be vaccinated but with different administration schedules [83].
In addition, the development of new vaccines against SARS-CoV-2, including a pan-coronavirus vaccine, with long-term protection is a priority for public health strategies to prevent new COVID-19 epidemics [84,85].
5. Conclusions
The results of this study suggest that SARS-CoV-2 mRNA vaccines have modest protection against SARS-CoV-2 infections and symptomatic infections after three doses. One or two doses are insufficient to protect against infection. Thus, we recommend surveilling for SARS-CoV-2 infections and virus variants, vaccinating at-risk populations, and developing new vaccines.