PHS reporting of cases, hospitalisations and deaths from COVID-19 by vaccine status – Interpreting the data

Dr Diane Stockton, Consultant at Public Health Scotland, explains in this blog why simple comparisons of COVID-19 rates in those who are vaccinated and unvaccinated should not be used to assess how effective a vaccine is in preventing serious health outcomes.

Vaccines work. This is the opener of a recent UKHSA blog on data from England on this same topic. Based on our current evidence from Scotland, I agree wholeheartedly with this, and this is the key message to take away from this blog.

There are two key points that we have learnt from the vaccine effectiveness analysis of omicron so far:

• Vaccine effectiveness wanes over time for all doses, so it is important to get your next dose as soon as possible after it becomes due, particularly if you are elderly and vulnerable, as your risk of a severe outcome will increase when the vaccine protection wanes.
  
  
• You can still get infected even if you are vaccinated - the biggest benefit of the vaccine is protection against severe disease. Therefore, even when you have been boosted you should be vigilant for symptoms and test yourself regularly with LFDs – especially before visiting those who are vulnerable.

Alongside the robust analyses we undertake in Scotland on vaccine effectiveness, we also publish simple statistics showing cases, hospitalisations and deaths from COVID-19 by vaccine status. Interpreting these simple statistics can be complex, and these simple comparisons of COVID-19 case rates in those who are vaccinated and unvaccinated should not be used to assess how effective a vaccine is in preventing serious health outcomes.

This blog explains the complexities, including how we under-estimate the rates of COVID-19 in unvaccinated groups, that testing behaviours are likely to vary between vaccinated and unvaccinated groups, and that older people who have had two doses but haven’t had their booster are likely to have a higher risk of severe outcomes.

1: We don’t actually know how many unvaccinated people have moved out of Scotland - so we are underestimating the proportion of unvaccinated people getting COVID-19

Everyone in Scotland who is registered with a GP is assigned a unique CHI number. This number is used to calculate estimates of Scottish residents for vaccine analyses, however it has its limitations when people leave Scotland and do not inform their GP, resulting in an overestimate of Scottish residents.

In order to estimate COVID-19 case rates we need to know exactly how many vaccinated and unvaccinated people there are living in Scotland. We know that most boosted people are living in Scotland as they have interacted with the health service recently. The rates of COVID-19 infection for them will be reasonably accurate.

However, for people who have exceeded the recommended time between doses, we become a little bit less sure. So, someone who has emigrated or been here as a student, may not have de-registered with their GP in Scotland as will be counted as part of population at risk of COVID-19. This could impact COVID-19 rates in all vaccine statuses, however this is particularly problematic in the unvaccinated population, because they have not had recent contact with a vaccination centre and we have to rely on GP records, which are likely to be more out of date than vaccine records.

The estimate of the population for Scotland is 8% higher if we use all people registered with a GP in Scotland compared to the official population estimates for mid-2020 from the National Records for Scotland (NRS). The true population will be somewhere in-between, and the difference is mainly in the <50-year age groups (which fits with more movement in and out of Scotland for younger people).

We believe that the NRS population estimates are probably closer to the reality, particularly for older people, which is why we use these for our vaccine uptake dashboard. However, when we undertake analyses of vaccine breakthrough infections and vaccine effectiveness, we need to use individual patient records and so have to use GP registrations as our population at risk. When we use the NRS populations, we estimate that 8% of individuals aged 12 or over have had no vaccine, but when we use the GP registrations it is closer to 16%. This means we have double the number of people identified as unvaccinated for these vaccine breakthrough and vaccine effectiveness analyses.

With the very high case rates we saw in early January 2022, not knowing who is currently living in Scotland has major implications for our estimates of COVID-19 case rates in the unvaccinated.

This simple example explains why:

Cases	Unvaccinated population (NRS population)	Unvaccinated population (GP registered population)	Case rate in unvaccinated (NRS population)	Care rate in unvaccinated  (GP registered population)
6	50	100	12 per 100	6 per 100
1	50	100	2 per 100	1 per 100

When around 1% of people have COVID in the population the incorrect denominator doesn’t make much difference to the rates (the rate is either 1 or 2 per 100), however, when around 6% of people have COVID-19 (as was estimated for Scotland at the end of December 2021) then the incorrect denominator makes a big difference (the rate is either 12 or 6 per 100). Comparisons of vaccinated and unvaccinated very biased when prevalence of COVID-19 is so high.

The same issue applies when we look at hospitalisations and deaths, but as the numbers are much smaller the effect of the population estimate being wrong is not as marked, and other biases related with small numbers and vulnerability become the issue (see point 3 below).

2: Differences in behaviour between vaccinated and unvaccinated groups

If we could identify all of our unvaccinated population we would find that some have not got the vaccine because they have had previous infection (which will confer them some protection) and some do not want to receive a vaccine. Of this latter group, people who are vaccinated may be more likely to follow other government guidance such as regular testing for COVID-19 and reporting of COVID-19 LFD test results, which would make them more likely to be identified as a case than unvaccinated people, resulting in higher case rates in the vaccinated population.

3: The underlying health of some older people who have not been boosted is different to those that have

A very high proportion of older people have been vaccinated with at least one dose. By 16 January 2022, 95% of people aged 75 and over had received three doses. For the remaining 5% who have not received a booster, reasons for not receiving a booster may be for logistical or personal reasons, having left the country and not updating GP records, or because of their general health at the time.

This 5% is equally split between older people who have had no vaccine (note bias 1 above) and those who have only had one or two doses. Most elderly people who have only had one or two doses, had them over 6 months ago. For these individuals, their vaccine may have waned and they are more likely to be frail and in general ill health due to their older age (hence not getting their third dose). This puts them at higher risk of getting COVID and very high risk of COVID-19 hospitalisation if they get COVID-19.

This is also true for COVID deaths. They are mainly occurring in those over 70 years old with multiple other illnesses. And of those vaccinated with only one or two doses, COVID deaths typically occurred more than six months after vaccination. These people may be more susceptible to a severe outcome and could result in higher cases, hospitalisations and death rates being observed in the first and second dose vaccine groups.

It is also the case that many younger people who are due their booster, but have not yet attended, will have vaccine waning due to the time since they had their second dose.

So, how should we interpret the results?

To overcome these the complexities and biases, we use statistical methods called vaccine effectiveness analyses. These make adjustments to account for the biases as much as possible.

From these analyses we know that Omicron is around one third as severe as Delta for adults (i.e. around one third the number of hospitalisations and deaths per case). This is true whatever your vaccine status – unvaccinated, partially or fully (boosted) vaccinated. For children, severity remains low and is similar to Delta.

Additionally, having had a booster gives a significant protection against severe outcomes for both Delta and Omicron compared to someone not vaccinated (you are at least 4 times as likely to be hospitalised if you haven’t had a booster).

The likelihood of being hospitalised increases with age, but is much reduced at all age if you are fully vaccinated compared to unvaccinated. Of those admitted to hospital in early January 2022, many were elderly and not boosted, and of those who died in early January 2022, they were also mainly elderly and most had not had been vaccinated within the last 6 months.

Vaccination remains the most important way to protect ourselves from COVID-19, and I urge anyone who is due another dose, particularly anyone who is elderly or vulnerable due to ill-health or pregnancy, to get it as soon as possible.

To find out how you can get vaccinated, visit www.NHSInform.scot or phone the vaccination helpline on 0800 030 8013.