Data quality

This is a National Statistics publication. National Statistics status means that this publication meets the highest standards of trustworthiness, quality and public value. This publication fully complies with the Code of Practice for Official Statistics (external website) and was awarded National Statistics status (external website) following an assessment in September 2011 (external website) by the UK Statistics Authority.

Strengths and limitations

To determine the quality of the statistics that we publish, Public Health Scotland (PHS) assesses the risk of data quality concerns (PDF, 72.2 KB) (external website) for each publication. Publications are assigned a low, medium or high data quality risk rating. This rating is based on factors such as the number of data suppliers involved, the complexity of the data collection process and the quality assurance checks applied to the data. This publication has been assigned a low data quality risk rating.

The following summarises how the data in this publication can be used and the limitations that users should be aware of:

How can the data in this publication be used?

The data can be used to:

  • compare across Scotland
  • view trends over time
  • compare activity between different specialties, age and sex groups or SIMD deprivation levels
  • assess whether patients were treated within or outside their own health board
  • explore the relationship between hospital activity and living in a deprived area

The data cannot be used to:

  • directly compare with other UK countries (England, Wales and Northern Ireland)
  • combine quarterly data to approximate annual figures
  • compare averages across different time periods
  • identify numbers of patients (only stays or episodes)
  • identify areas of affluence
  • identify how much more deprived one area is than another

Data sources

The data used in this publication are sourced from the Scottish Morbidity Records (SMR) and ISD(S)1 datasets held by PHS.

SMR01 (inpatients and day cases)

The SMR01 dataset comprises episode-based patient records relating to all inpatients and day cases discharged from non-obstetric and non-psychiatric specialties (excluding geriatric long-stay records). Data are updated monthly and include clinical and non-clinical data.

SMR00 (outpatients)

The SMR00 dataset contains patient-based information on appointments at outpatient clinics in all specialties (except A&E and Genito-Urinary Medicine) in NHS Scotland. Data are updated monthly and include first activity, total outpatient activity and attendance status (such as ‘Did Not Attends’ (DNAs)).

ISD(S)1 (beds and return outpatients)

ISD(S)1 is a set of aggregated summary statistics on activity and resources in hospitals in Scotland and is derived from monthly and quarterly returns from the NHS boards. ISD(S)1 does not record information on the age or sex of patients, nor the deprivation of the area in which they live. ISD(S)1 is the only source of bed occupancy and bed availability data and contains summarised data by NHS board of treatment, hospital and specialty. ISD(S)1 is used for return outpatient activity as completeness for historic return outpatients in SMR00 is poor.

Data collection and validation

The data flow diagram below illustrates the process of data collection and validation that is carried out before SMR data are available to analysts in PHS (formerly ISD) databases. NHS boards update their current and historical data every month; therefore, data included within each publication are provisional and subject to change from one publication to another.

Data are subjected to checks by the NHS board where the data were recorded and by PHS once the data are received. Examples of validation carried out include: ‘postcode exists’, ‘consultant worked in the location or specialty at the time of admission’, ‘age and sex at admission are consistent with diagnosis’. Any data errors (missing or invalid information) or queries (information that appears implausible) are sent back to the NHS board for further investigation. Derived items added by PHS include age, continuous inpatient stay markers and geographies such as ‘NHS board of residence’.

Communication with data supply partners

The Data Management Secondary Care Team supports data providers in the submission and quality of national data sets. The team meets regularly with providers to ensure that any issues affecting the data are identified and resolved at an early stage. The team also provides an advisory service to data users, in particular to PHS analytical teams that produce official publications such as this one. PHS analysts are kept up to date with any issues regarding Scottish Morbidity Records (SMRs) via the SMR Information Bulletin.

Summary of data completeness

The data used in this publication were extracted from the SMR national databases in April 2022, and the following table reflects SMR completeness as at 11 April 2022. Full details of SMR data completeness (external website) and SMR data timeliness (external website) can be found on these respective web pages.


NHS board
SMR00
(new outpatients)

October'21-December'21
SMR01
(inpatients and day cases)

October'21-December'21
Ayrshire & Arran 100% 100%
Borders 100% 100%
Golden Jubilee 100% 99%
Fife 100% 97%
Greater Glasgow & Clyde 100% 99%
Highland 100% 96%
Lanarkshire 99% 99%
Grampian 100% 98%
Orkney 100% 100%
Lothian 96% 97%
Tayside 96% 99%
Forth Valley 97% 96%
Western Isles 100% 100%
Dumfries & Galloway 90% 95%
Shetland 100% 69%
All NHS boards 98% 98%

NHS boards aim to submit SMR data to PHS within six weeks following a hospital discharge or transfer, death or a clinic attendance. Completeness is calculated by comparing the number of records submitted to PHS against the total number of records that the board estimates PHS should receive. NHS data providers will know how complete their SMR submissions are and the extent of any backlog. PHS defines ‘backlog’ as data received after six weeks. 

Please note that there are a number of NHS boards where the Data Management Team are unable to accurately measure completeness, as coding reports are not received from these NHS boards. The NHS boards affected are Borders, Greater Glasgow & Clyde, Highland, Orkney and Shetland. Reported completeness figures may change as more information becomes available.

PHS does not produce completeness levels for ISD(S)1. However, the Data Management Team queries any unusual numbers with NHS boards. Note that estimates may be applied by Data Management to any missing data in the ISD(S)1 dataset. For beds, missing data are estimated by using an average of the latest three months of known data. For outpatients, estimation is based on the latest submission received.

Data quality issues

General SMR issues

COVID-19

Please note that this release includes Scotland going into emergency measures due to COVID-19. During this pandemic, NHS boards, hospitals and healthcare providers have been required to change their normal way of working to manage their COVID-19 response. As such, this is directly impacting on the volume of hospital activity and trends observed over the past couple of years. For example, inpatient and day case activity and outpatient activity have reduced by 17% and 13%, respectively, when comparing October to December 2021 to the same quarter of 2019 (pre-pandemic). However, activity levels have generally been recovering from July 2020 onwards but are still not up to pre-pandemic levels.

In addition, PHS anticipates that there might be some changes in terms of our regular statistical production. The current disruption to Scotland and the rest of the UK could affect the quality of some of our statistics, such as lower accuracy, or it could mean that there is less detail available, such as fewer local and regional breakdowns. In some cases, the production of some data series may need to be suspended.

TrakCare Patient Management System (PMS) issues

NHS Lothian was the first board to implement TrakCare PMS, starting in 2005. All NHS boards, apart from NHS Dumfries & Galloway and NHS Western Isles, have now implemented a version of this patient management system. There are known issues relating to SMR data submissions and, unfortunately, these issues are not standard across NHS boards; much depends on which system version is in place and how proactive NHS boards have been in implementing fixes as they become available. Data Management works closely with the NHS boards and Intersystems to investigate issues as they are identified. However, users should be aware that system issues can impact on data quality and often take some time to resolve.

SMR replacement records

It should be noted that PHS identified an issue in June 2018 whereby some SMR replacement records being submitted by TrakCare NHS boards resulted in the original SMR record being amended in error. This caused some data items being overwritten, which may have impacted on some local and national analysis.

From 1 April 2020, SMR replacement records will only be accepted from NHS boards if the clinic/discharge date is within two years of the submission date (excluding SMR04). Data Management is processing replacement records monthly to coincide with the file update.

No action will be taken to restore patient identifiable data items replaced in error prior to the discovery of this issue in June 2018. Therefore, there is a possibility that names or postcodes may have been updated from those submitted originally.

Auto-populated fields

TrakCare PMS includes a facility to ‘auto-populate’ SMR data fields, which was developed to make data entry easier for the NHS boards. The first episode in a Continuous Inpatient Stay (CIS) will be recorded appropriately, but subsequent SMR records in the same CIS could be completed with the same codes as the first episode. Some of the fields affected are ‘Management of Patient’, ‘Admission Type’, ‘Admission Transfer From’, ‘Discharge Type’ and ‘Discharge Transfer To’. Data Management is unable to quantify how much of an impact this will have. Intersystems has noted that it is unable to correct this at the present time.

‘Non-NHS Provider’ data

‘Non-NHS Provider’ figures relate to patients treated in non-NHS locations such as private hospitals, hospices, nursing homes, care homes, etc. Patients who receive treatment paid for by the respective NHS board at a private (independent) hospital should be recorded within the SMR by the relevant NHS board. However, if a patient is treated privately (where treatment is paid for by the patient or a private insurer) and there is no NHS involvement, then this activity will not be recorded within the SMR. The data recording/completeness of non-NHS provider data varies from year to year; therefore, please treat any data provided with caution. It should also be noted that any changes in activity (both increases and decreases) are based on small numbers, which will impact on the percentage change.

‘Unknown Health Board’ recording

‘Unknown Health Board’ refers to locations or residences that cannot be attributed to any of the other NHS boards. Sharp increases have been observed in this category from April 2020 onwards. For example, new outpatient ‘unknowns’ increased tenfold when comparing the period January–March 2020 to April–June 2020. The main reason for these increases is records submitted with unknown location code (‘D299N: Location not otherwise coded’). This results in the health board of treatment not being derived. Instead of being allocated to the actual NHS board, the activity is instead categorised against ‘Unknown Health Board’.

Increased use of the ‘unknown location’ code can be attributed to uncertainty in the provision of outpatient clinics during the COVID-19 pandemic. Clinics are set up as a mixture of face to face, telephone, video link and ‘Near Me’ appointments. However, this code has been used as a workaround during the pandemic when the location of activity was not known. This made the setting-up of clinics simpler in TrakCare. The unknown location code has been used by several boards, predominantly by NHS Tayside and Borders, although use of the code is decreasing in 2021. New outpatient ‘unknowns’ have decreased by nearly three-quarters (74%) comparing October–December 2021 to the same quarter in 2020.

Data Management is looking at identifying codes that boards could use to correctly derive the health board of treatment, and the intention is that records already submitted using this code will be recoded.

General issues – SMR01

Acute Assessment Unit (AAU)/Ambulatory Emergency Care (AEC) activity – definitions

Acute Assessment Unit (AAU)

The AAU is a dedicated facility for the acute clinical care of patients who present to hospital as clinical emergencies or who develop an acute clinical problem while in hospital. The units may also carry out some planned healthcare.

Generally, these units have both trolleyed areas and staffed beds that form part of the hospital’s bed complement. Where trolleys are used in lieu of beds, patients should be counted as inpatients.

Acute Assessment Unit (AAU) is the preferred term for services also known as:

  • medical/surgical assessment unit
  • combined assessment unit
  • clinical assessment unit
  • acute medical (assessment) unit
  • paediatric assessment unit
  • acute receiving ward/unit admission unit

These cases should be recorded under significant facility 40.

Ambulatory Emergency Care (AEC)

An Ambulatory Emergency Care unit is a multidisciplinary, one-stop service. It provides outpatient and day case services only.

These cases should be recorded under significant facility 39.

National recording of AAU & AEC activity

Currently, AAU activity is only being submitted by some NHS boards within SMR01. NHS Greater Glasgow & Clyde AAU activity stopped in 2017. NHS Highland has been submitting AEC cases via SMR01 using criteria agreed by PHS to ensure that they pass validation rules as an interim measure. NHS Greater Glasgow & Clyde has opted to record these cases differently from NHS Highland since it considers a number of these cases to be non-elective day cases that, due to recording rules, cannot be recorded in this way on TrakCare. As such, the board took the decision to record them as Emergency Department activity to enable the patient to be tracked throughout the system. However, from September 2019 onwards, the Royal Alexandra Hospital within NHS Greater Glasgow & Clyde has started to record AAU cases again within SMR01.

NHS Western Isles and NHS Ayrshire & Arran started submitting AEC cases via SMR00 from June 2020 and July 2021, respectively. NHS Dumfries & Galloway started submitting AEC cases via SMR01 from August 2020 onwards. Data Management has been in contact with these NHS boards to discuss whether the records should be updated to reflect a more appropriate significant facility code. NHS Western Isles has since updated the facility code to ‘11 – Other’ and resubmitted these records. NHS Ayrshire & Arran has said that it has changed the Deep Vein Thrombosis (DVT) clinic held at their Clinical Decisions unit to significant facility 39 from July 2021. NHS Dumfries & Galloway has informed that these cases are part of a trial where advanced nurse practitioners see suitable patients. If required, these will be passed to consultants to be seen under significant facility 40 (Acute Assessment Unit/Acute Medical Unit). NHS Lothian has said that it does not plan to update the significant facility codes it uses; therefore, it remains difficult to quantify ambulatory-care activity.

There are ongoing discussions with NHS boards, the Scottish Government and PHS on the most appropriate way for capturing this activity (including AEC cases). It is hoped that national definitions and guidance on how to record this activity can be agreed by all NHS boards.

Transfers

Several NHS boards have experienced changes in internal transfer activity due to the way in which their data have been submitted. However, it should be noted that any changes in activity (both increases and decreases) are based on small numbers, which will impact on the percentage change.

For more information, see the Glossary and Data files sections of this publication.

General issues – SMR00

Recording of SMR00 procedures

The recording of procedures is not consistent across the NHS boards. NHS Borders, NHS Dumfries & Galloway, NHS Grampian, NHS Lanarkshire, NHS Lothian, and the NHS island boards (namely NHS Orkney, NHS Shetland and NHS Western Isles) record procedures on under 10% of records submitted. Other NHS boards record procedures on between 10% and 25% of records.

Recording of return attendances

The submission of all return attendances is mandatory, regardless of whether or not a procedure is performed. However, there is variation in the NHS boards’ submissions of return outpatients in SMR00. For this reason, ISD(S)1 has been used to extract return outpatient attendances.

General issues – ISD(S)1

ISD(S)1 return outpatient and beds NHS board coding

Records have been identified with potentially duplicate information coming from more than one NHS board for the same location. The numbers concerned are very low, and the impact is not significant. Additionally, there are issues with the allocation of NHS hospitals to NHS boards and private hospital activity to the ‘Non-NHS Provider’ code. Data Management has investigated and found that, in many of these cases, this can be attributed to visiting consultant activity and NHS boards sending their patients to private locations.

NHS board-specific issues

Any information provided by the respective NHS board is included in the Trend data section of this publication. It should be noted that many trends observed will be influenced by (and attributable to) the data completeness levels, small numbers and the impact of COVID-19 highlighted above. Therefore, caution should be taken when comparing quarterly information.

Data quality assurance within PHS

Scottish NHS boards have a responsibility to ensure their SMR data are accurate, consistent and comparable across time and between sources. The PHS Data Quality Assurance Team (DQA) audit SMR data at NHS boards to determine whether they have been properly recorded in accordance with national rules and standards. The DQA Team’s assessment web page (external website) contains reports from past audits of inpatient and day case data, including findings on the accuracy of submitted SMR01 data items used in our analysis (specialty, admission type, etc.).

The Quality Indicators Secondary Care Team that produces this publication also carries out quality assurance checks on the data after extraction from the databases. For example, the team compares high-level NHS board figures for the same quarter between the current and previous publications to identify any large changes in the data sources. Additionally, trends are scrutinised within the current publication to identify any unusual patterns. For changes or patterns in the data that cannot be explained by the known completeness estimates, the team contacts Data Management to highlight the issue. In turn, Data Management contacts the relevant NHS board for an explanation. Any information provided by the NHS board is included in the Trend data section of this publication

Last updated: 21 March 2024
Was this page helpful?