Covid-19: Peer-reviewed report shows Oxford vaccine is safe – but how effective is it?

Full interim analysis of the safety and efficacy of the Oxford Covid-19 vaccine – developed in partnership with AstraZeneca – has been published in The Lancet. This paper represents the most comprehensive set of results from any of the phase 3 trials of Covid-19 vaccines to date.

These are still only interim results, meaning they show only how the ongoing phase 3 trial has progressed so far. The final results may differ. But what this analysis reveals is how the Oxford team calculated its vaccine’s efficacy, which it announced in late November.

This interim analysis included 11,636 people, of whom 7,548 were in the UK and 4,088 in Brazil. All participants received two injections, but as has been widely reported, one of the UK groups received a half dose rather than a full dose for the initial injection. This has made analysing the effects of this vaccine more complicated than it probably will be for others in the late stages of testing.

Importantly, the evidence that the vaccine is safe is convincing. There were 175 severe adverse events reported in the trials, and these were actually more common in the control group (which received either saline or a meningococcal vaccine) than with those receiving the Covid-19 vaccine.

Three of these events were judged to be possibly related to either the experimental or control vaccine. They included a case of transverse myelitis, a rare but serious spinal inflammatory disorder, in someone in the vaccine group, a case of haemolytic anaemia in someone in the control group, and a further person who developed a fever of over 40 degrees Celsius. As yet we do not know whether this person received the Covid-19 vaccine or a control.

Across the study, ten patients became sufficiently ill with Covid-19 to require hospitalisation, but all were in the control arm.

Calculating efficacy

The headline figure given for the overall efficacy of this vaccine is 70.4%. This is lower than the equivalent efficacy reported by Pfizer (94%) and Moderna (94.1%) for their vaccines.

Both Pfizer and Moderna have based their estimates on rather bigger trials than the Oxford vaccine and so their estimates are likely to be more precise. But we still need to be able to see the full publication of their results in peer-reviewed journals before we can make valid comparisons between the efficacy and safety of the three vaccines. But a lower efficacy does run the risk of leaving a higher proportion of the vaccinated population unprotected.

But the most difficult issue with the Oxford/AstraZeneca study is that it has pooled analysis of the vaccine’s efficacy from three separate trials. The UK-based arm of the trial was in effect two studies. One study used two standard-dose injections and the other a half dose followed by a standard dose. A third study in Brazil used two standard doses.

These different dosing strategies produced very different efficacy readings. Averaging efficacy across everyone involved in all three studies combined produces the 70.4% figure – but this does not truly reflect the efficacy of the vaccine in either dosing scenario.

Both standard-dose study arms gave similar estimated efficacies – 60.3% in the UK and 64.2% in Brazil. Combined, the efficacy of these two standard-dose trials was 62.1%. By contrast, the low-dose regimen indicated an efficacy of 90%. This finding would place the Oxford vaccine close to the reported efficacies of the Pfizer and Moderna vaccines.

But how reliable is this estimate of what is, in effect, a subgroup within the study? Breaking analysis down in this way to look at outcomes in specific groups can be appropriate in randomised controlled trials – but it increases the risk of what looks like positive findings simply being down to chance.

This sort of practice is “particularly unreliable” and “can produce spurious results”, experts have noted. I would add that one needs to be even more cautious when subgroup analyses produce unexpected results – which is exactly what has happened here, as the low-dose regimen stemmed from an accident. Additional testing of the low-dose regimen is needed.

A further issue is that the study arm that yielded the 90% estimate was only conducted with people under the age of 55 years and almost exclusively with people of white ethnicity. By contrast, the burden of serious disease and death from Covid-19 tends to fall on older people and those from Black and Asian ethnic groups and the priority populations for immunisation in the UK are largely over 55 years old.

Given their increased vulnerability, knowing whether older people respond well to this vaccine is important. Problematically, we know that older people often do not respond as well as younger people to vaccination. On the other hand, in another recent Lancet paper from the Oxford Vaccine Group, groups of participants aged 18 years to 55 years, 56 years to 69 years and 70+ years were all found to have similar levels of antibodies after receiving the standard-dose regimen, suggesting that it will be similarly effective across all ages.

But we do not know yet what the effect of the seemingly more efficacious low-dose regimen is in over-55s. So as well as needing further trials of the low-dose regimen, they also need to include older people and Black and Asian people. Only then can we be sure that it is effective across different groups.

We also do not yet know for certain how effective any of the Covid-19 vaccines will be at preventing asymptomatic infection or transmission of the virus. But the Oxford vaccine’s phase 3 results do suggest that immunisation with the vaccine will generate some downward pressure on transmission.

Getting the development and testing of the Oxford vaccine this far is an incredible achievement – and the vaccine will undoubtedly become a major tool in our ability to combat the current pandemic. The vaccine is safe and effective, albeit maybe not as effective as the two other leading candidates from Pfizer and Moderna. The low-dose regimen may have an efficacy closer to these other vaccines, but we need more data to be sure.

Paul Hunter is a Professor of Medicine at the University of East Anglia.

This article first appeared on The Conversation.