Risk stratification of patients with covid-19 in the community

Abstract

Back when covid-19 was emerging and little was known about the disease, there was a monumental effort to understand the evolving data and develop prediction tools that patients, healthcare workers, and policy makers could use to optimise care. The unfortunate result was a tidal wave of poorly conceptualised prediction models, often using small convenience samples, incorporating little or no validation, and including no substantive plan for implementation. As a result, most developed prediction tools were never meaningfully applied in clinical care.

Examples of good practice existed, including two collaborative projects we were fortunate to be involved in – QCOVID (estimating risk of being hospitalised or dying due to catching covid-19) and the ISARIC 4C models (estimating risk of dying or deteriorating after hospital admission with covid-19). However, an obvious gap existed in the assessment of symptomatic patients in the community. As the disease profile has changed and the focus of care shifts to supporting diagnosis, treatment, and monitoring outside hospitals, this has become increasingly important.

In the Lancet Digital Health, we welcome the study by Espinosa-Gonzalez and colleagues on the derivation and validation of two much-needed risk stratification tools for use in a community setting. These pragmatic decision aids support the assessment of patients with covid-19 symptoms, seeking to identify those who will likely require further monitoring (RECAP-GP) and those in whom treatment escalation is warranted (RECAP-O2). They were developed according to a pre-published protocol and utilise linked primary and hospital healthcare records, together with data from the WhatsApp-based patient monitoring platform, Doctaly Assist.

What do these data tell us and how well do the models work? First, it is interesting to reflect on what the models actually capture. These were patients with symptoms of covid-19, but who did not necessarily have covid-19. This is pragmatic and appropriate, as a covid-19 diagnostic test may not be available at the time of assessment. But as covid-19 prevalence drops in the community, how patients are selected to use this tool will impact significantly on its performance.

A second point of reflection goes by the unhelpful term, “incorporation bias”. The researchers here are testing to see if symptoms predict admission, but the same symptoms have likely been used to determine the need for the actual hospital admission. The prediction tool can therefore become a self-fulfilling prophecy, and this circularity can artificially increase sensitivity and specificity. The authors mitigate against this by requiring an admission to be at least one night (and by implication require clinical management rather than simply assessment), but the effects of this bias may persist.

The RECAP-GP model performs well in the first external cohort (NWL), but the discrimination is poorer in the second (CCAS; AUROC 0.66) and similarly for RECAP-O2 (Doctaly-2; AUROC 0.68). These cohorts were from later in the pandemic and differences in population (younger and less comorbid), virus variants, and vaccination status may partly explain this. Calibration (the performance of the model across the range of risk) is important and while good to see included for the development dataset, it would have been useful for the external validation as well. Similarly, while good to see model performance presented by age and sex, it is important to ensure that it performs as well across different ethnic groups.

As presented the models may confuse users. The risk of hospital readmission for those breathless on moderate exertion is lower than for those with mild exertion (RECAP-GP; similar finding in RECAP-O2). For instance, a 45 year-old male with hypertension and a fever complaining of moderate breathlessness will be flagged amber (8.1% risk) while the same patient describing only mild breathlessness will be flagged red (11.5% risk). This is likely explained by the incorporation of non-significant factor levels, but the resulting biological implausibility may reduce face validity.

Applicability in low- and middle-income countries (LMICs) must also be considered. Continued reduced access to vaccination, varied public health policy implementation and higher death rates suggest research should be relevant and generalisable to such settings. The widespread absence of peripheral oxygen monitors means the RECAP-O2 model is currently unlikely have relevance beyond a select few countries. However, RECAP-GP has the potential for global clinical utility and validation in resource-limited settings is an urgent priority.

We declare no competing interests.

Rights

https://creativecommons.org/licenses/by/4.0/

Cite as

Downloadable citations

Identifiers

Repository URI

https://www.research.ed.ac.uk/en/publications/f52d2382-f64f-46ea-9a41-632e7f8c64c4