A statistician explains: What does ‘90% efficacy’ for a Covid-19 vaccine mean?

While I was writing a newspaper article on performances of various potential Covid-19 vaccines from different organisations, my wife, who is neither a statistician nor a vaccine expert, asked me the meaning of 90% efficacy of a vaccine. And I suddenly realised that such figures for potential vaccines under trial are never very clear to common people unless their implications are clearly illustrated.

In fact, different pharmaceutical companies are now coming out with the success stories of their potential vaccines, and the efficacy rates of these vaccines are reported to be high – 70%, 90%, 95%, and so on.

Here is how we can understand vaccine efficacy.

Phase III trial

Vaccines are usually approved on the basis of results from three stages of clinical trials. The trials aim to assess short-term safety, ability to generate an immune response, and efficacy. Short-term safety is usually judged in early phases.

Phase III trial, which is often the most elaborate one, investigates the efficacy of the vaccine under trial – usually in comparison to a placebo, which maybe a similar-looking injection having no medical effect. In a phase III trial, often thousands of people are given the vaccine or a placebo, and then these people are monitored over several months to see whether the people receiving the vaccine get infected at a lower rate than people who get the placebo, on average.

Thus, the performance of the vaccinated group is compared to that of the unvaccinated group. Phase III is a full-scale evaluation of efficacy, and it provides an opportunity to monitor for some safety issues which cannot be done in earlier phases. ‘Vaccine efficacy’ is defined to measure whether the vaccine is able to prevent the disease significantly or not, and if so, to what extent.

The mathematical deduction of protective vaccine efficacy was proposed by M Greenwood and GU Yule in a research article published in the Proceedings of the Royal Society of Medicine in 1915, where statistics of antityphoid and anti-cholera inoculations were introduced and interpretated.

Risk ratio

Vaccine efficacy is expressed as a proportionate reduction in disease attack rate, AR, between the unvaccinated, ARU, and vaccinated, ARV, groups under the phase III trial.

ARV is just the proportion of individuals within the vaccinated group who got infected within the study period. Similarly, ARU is the proportion of infected within the unvaccinated group.

The ratio of ARV to ARU is called the risk ratio, RR. A lower value of RR clearly indicates better performance of the potential vaccine. When both the vaccinated and unvaccinated groups have, more or less, equal number of individuals, RR is the simple ratio of the number of infected in the vaccine group to that in the placebo group. And, one minus RR, expressed in percentage, is called the “Vaccine Efficacy”.

Clearly, higher the value of vaccine efficacy, the better the performance of the potential vaccine. Thus, a 90% efficacy (ie 10% RR) means that the proportion of infection in the vaccinated group is about one-tenth of the proportion of infection in the placebo group.

Vaccine efficacy

Well, my wife is satisfied with the answer. And the reported efficacy values of the potential vaccines are also quite high.

The Moderna trial involved 30,000 people, and more than 43,538 people have been enrolled in the Pfizer/BioNTech vaccine trial – approximately half receiving the vaccine, and the remaining a “placebo”, in each trial. Of the first 95 to develop Covid-19 symptoms in the Moderna trial, only five were in the vaccine group, and the remaining 90 were in the placebo group.

Thus, the RR for Moderna vaccine up to this point of time is 5/90, and vaccine efficacy is (1 – 5/90), which is 94.44%. Similarly, of the 170 cases of the Pfizer trial, 162 were in the placebo group, while 8 were in the vaccine group. The vaccine efficacy of the Pfizer vaccine is (1 – 8/162), which is 95.06%.

The Oxford-AstraZeneca vaccine candidate appeared to be 90% and 62% effective for the two dose regimens having 2,741 and 8,895 individuals in the two groups, respectively. The average efficacy was announced to be 70%, the early analysis being based on 131 symptomatic Covid-19 cases that had turned up in study participants. But we still do not know how many cases were found in each group of participants across the vaccined and placebo groups, and also the division among the two dose regimes.

Short-term data

But, we should keep in mind that the vaccine efficacies are based on short-term data only. Both the Modera and Pfizer trials are found to be nearly 95% effective to protect Covid-19 in the short-term, although both trials are still taking place and the final numbers could change.

For example, if among the next 100 infections 98 are from placebo group and 2 from the vaccine group, the efficacy of Moderna vaccine would be enhanced to [1 – (5+2)/(90+95)], i.e. 96.28%. However, if the vaccine effect wanes in course of time and 70 and 30 of the next 100 infections are from the placebo and vaccine groups, respectively, the overall efficacy would become [1 – (5+30)/(90+70)], i.e. 78.13%. However, if the next 100 infections occur equally within the two groups, the resultant efficacy will be only 60.71%.

Thus, we should keep in mind that vaccine efficacy is just a statistical measurement of short-term performance of the vaccine. It cannot guarantee the durability of the vaccine beyond the time frame of experimentation, which is only a few months in these cases.

As per the 2020 ‘Guidance for Industry’ of the US Food and Drug Administration for ‘Development and Licensure of Vaccines to Prevent COVID-19’, all phase III trial participants are expected to be monitored for at least 1 year. The recruitment of phase III for the Moderna vaccine was done in April and May, and that for Pfizer vaccine began in end-July. Thus, these all are short-term efficacy.

There is a widespread speculation that the SARS-CoV-2 antibodies, which would make one immune to Covid-19, may not last beyond a few months. This is supported by different peer-reviewed articles in various journals as well. Thus, it is still not very clear how long a vaccine would provide protection against the disease. Whatever be the case, vaccine efficacy might drastically change if the study individuals are monitored for another few months, at least. Let’s just hope for the best.

Atanu Biswas is Professor of statistics at the Indian Statistical Institute, Kolkata.