Data quality

This is a National Statistics Publication. National Statistics status means that this publication meets the highest standards of trustworthiness, quality and public value. This publication fully complies with the Code of Practice for Official Statistics (external website) and was awarded National Statistics status (external website) following an assessment in September 2011 (external website) by the UK Statistics Authority (external website).

Strengths and Limitations

In order to determine the quality of the statistics that we publish, PHS (Public Health Scotland) assesses the risk of data quality concerns (PDF) for each publication. Publications are assigned a low, medium or high data quality risk rating. This rating is based on factors such as the number of data suppliers involved, the complexity of the data collection process and the quality assurance checks applied to the data. This publication has been assigned a low risk of data quality concerns.

The following summarises how the data in this publication can be used, and the limitations which users should be aware of:

How can the data in this publication be used?

This data can be used to:

  • compare across Scotland;
  • view trends over time;
  • compare activity between different specialties, age and sex groups, or deprivation levels;
  • assess whether patients were treated within or outwith their own health board;
  • explore the relationship between hospital activity and living in a deprived area.

This data cannot be used to:

  • directly compare with other UK countries;
  • combine quarterly data to approximate annual figures;
  • compare averages across different time periods;
  • identify numbers of patients (only stays or episodes);
  • identify areas of affluence;
  • identify how much more deprived an area is from another.

Data Sources

The data used in this publication are sourced from the Scottish Morbidity Records (SMR) and ISD(S)1 datasets held by PHS.

SMR01 (Inpatients and Day Cases)

The SMR01 dataset comprises episode-based patient records relating to all inpatients and day cases discharged from non-obstetric, non-psychiatric specialties and excluding geriatric long stay records. Data are updated on a monthly basis and include clinical and non-clinical data.

SMR00 (Outpatients)

The SMR00 dataset contains patient-based information on appointments at outpatient clinics in all specialties (except A&E and Genito-Urinary Medicine) in NHSScotland. Data are updated on a monthly basis and include first and total outpatient activity and attendance status, for example Did Not Attends (DNAs).

ISD(S)1 (Beds and Return Outpatients)

ISD(S)1 is a set of aggregated summary statistics on activity and resources in hospitals in Scotland and is derived from monthly and quarterly returns from the NHS Boards. ISD(S)1 also does not record information on the age or sex of patients, nor the deprivation of the area in which they live. ISD(S)1 is the only source of bed occupancy and bed availability data and contains summarised data by NHS Board of Treatment, hospital and specialty. ISD(S)1 is also used for return outpatient activity since completeness for historic return outpatients in SMR00 is poor.

Data Collection and Validation

The data flow diagram above illustrates the process of data collection and validation that is carried out before SMR data are available to analysts in PHS (formerly ISD) databases. NHS Boards update their current and historical data every month; therefore, data included within each publication are provisional and subject to change from one publication to another.

Data are subjected to checks by both the NHS Board where the data were recorded and by PHS once the data are received. Examples of validation carried out include: postcode exists; consultant worked in the location or specialty at the time of admission; age and sex at admission are consistent with diagnosis. Any data errors (missing or invalid information) or queries (information which appears infeasible) are sent back to the NHS Board for further investigation. Derived items added by PHS include age, continuous inpatient stay markers and geographies such as NHS board of residence.

Communication with Data Supply Partners

The Secondary Care team in Data Management support data providers in the submission and quality of national data sets. The team meet regularly with providers to ensure any issues affecting the data are identified and resolved at an early stage. The team also provides an advisory service to data users, in particular to PHS analytical teams who produce official publications such as this one. PHS analysts are kept up to date with any issues regarding Scottish Morbidity Records (SMRs) via the SMR Information Bulletin.

Summary of Data Completeness

The data used in this publication were extracted from the SMR national databases in January 2022, and the following table reflects SMR completeness as at 11 January 2022. Full details of SMR data completeness and SMR data timeliness can be found on the SMR Completeness and SMR Timeliness web pages respectively.


NHS Board
SMR00
(new outpatients)

July'21-September'21
SMR01
(inpatients and day cases)

July'21-September'21
Ayrshire & Arran 100% 100%
Borders 100% 100%
Golden Jubilee 100% 99%
Fife 100% 96%
Greater Glasgow & Clyde 100% 99%
Highland 100% 97%
Lanarkshire 100% 99%
Grampian 100% 97%
Orkney 100% 100%
Lothian 94% 98%
Tayside 95% 99%
Forth Valley 96% 97%
Western Isles 73% 100%
Dumfries & Galloway 89% 94%
Shetland 100% 90%
All NHS Boards 98% 98%

NHS Boards aim to submit SMR data to PHS 6 weeks following a hospital discharge or transfer, death or a clinic attendance. Completeness is calculated by comparing the number of records submitted to PHS against the total number of records that the Board estimate we should receive. NHS data providers will know how complete their SMR submissions are and the extent of any backlog. PHS calculates backlog as data which are received after 6 weeks.     

Please note that from March 2020, there are a number of NHS Boards where the Data Management Team are unable to measure accurately the completeness as coding reports were not received from these NHS Boards. The NHS Boards affected are: Borders, Greater Glasgow & Clyde, Highland, Orkney and Shetland. Reported completeness figures may change as more information becomes available.

PHS do not produce completeness levels for ISD(S)1, however, the Data Management team query any unusual numbers with NHS Boards. Note that estimates will be applied by Data Management to any missing data in the ISD(S)1 dataset. For beds, missing data is estimated by using an average of the latest three months of known data. For outpatients, estimation is based on the latest submission received.

Data Quality Issues

General SMR Issues

COVID-19

Please note this release includes Scotland going into emergency measures due to COVID-19. During this pandemic, NHS Boards, hospitals, and healthcare providers have been required to change their normal way of working to manage their COVID-19 response. As such, this is directly impacting on the volume of hospital activity and trends observed over the past couple of years. For example, inpatient and day case activity and outpatient activity has reduced by 15% and 13% respectively, when comparing to the pre-pandemic July to September 2019 quarter. However, activity levels have been recovering from July 2020 onwards, but are still not up to pre-pandemic levels.

In addition, PHS anticipate that there might be some changes in terms of our regular statistical production. The current disruption to Scotland and the rest of the UK could affect the quality of some of our statistics, such as lower accuracy, or it could mean there is less detail available, such as fewer local and regional breakdowns. In some cases, the production of some data series may need to be suspended.

TRAKCare Patient Management System (PMS) Issues

NHS Lothian was the first board to implement TRAKCare PMS, starting in 2005. All NHS boards, apart from NHS Dumfries & Galloway and NHS Western Isles, have now implemented a version of this patient management system. There are known issues relating to SMR data submissions and unfortunately these issues are not standard across NHS Boards; much depends on which system version is in place and how proactive NHS Boards have been in implementing fixes as they become available. Data Management work closely with the NHS Boards and Intersystems (TRAK) to investigate issues as they are identified, however users should be aware that system issues can impact on data quality and often take some time to resolve.

SMR Replacement Records

It should be noted that PHS identified an issue in June 2018 whereby some SMR replacement records being submitted by TrakCare NHS Boards resulted in the original SMR record being amended in error. This caused some data items being overwritten which may have impacted on some local and national analysis.

From 1 April 2020, SMR replacement records will only be accepted from NHS boards if the clinic/discharge date is within 2 years of the submission date (excluding SMR04). Data Management are processing replacement records monthly to coincide with the file update.

No action will be taken to restore patient identifiable data items replaced in error prior to the discovery of this issue in June 2018. Therefore, there is the possibility that names or postcodes may have been updated from those submitted originally.

Auto-populated fields

TRAKCare PMS includes a facility to 'auto-populate' SMR data fields which was developed to make data entry easier for the NHS boards. The first episode in a Continuous Inpatient Stay (CIS) will be recorded appropriately, but subsequent SMR records in the same CIS could be completed with the same codes as the first episode. Some of the fields affected are Management of Patient, Admission Type, Admission Transfer From, Discharge Type and Discharge Transfer To. Data Management are unable to quantify how much of an impact this will have. Intersystems have noted that they are unable to correct this at the present time.

Non-NHS Provider Data

Non-NHS provider figures relate to patients treated in non-NHS locations such as private hospitals, hospices, nursing homes, care homes, etc. Patients who receive treatment at a private (independent) hospital which is paid for by the NHS board should be recorded within the SMR by the relevant NHS board. However, if a patient is treated privately (i.e. treatment paid for by patient or private insurer) and there is no NHS involvement then this activity will not be recorded within the SMR. The data recording / completeness of non-NHS provider data varies from year to year, therefore please treat any data provided with caution. It should also be noted that any changes both increases and decreases in activity are based on small numbers which will impact on the percentage change.

"Unknown" Health Board Recording

'Unknown Health Board' refers to locations or residences that cannot be attributed to any of the other NHS Boards. Sharp increases have been observed in this category from April 2020 onwards. For example, new outpatient unknowns increased ten-fold when comparing January to March 2020 to April to June 2020. The main reason for these increases are records submitted with unknown location code (D299N: Location not otherwise coded). This results in the Health Board of treatment not being derived and the activity is not allocated to the actual NHS Board but instead categorised against ‘Unknown’ Health Board.

Increased use of the unknown location code can be attributed to uncertainty in the provision of outpatient clinics during the COVID-19 pandemic. Clinics are set up as a mixture of face to face, telephone, video link and Near Me appointments. However, this code has been used as a 'work around' during the pandemic, as location of activity was not known. This made the setting up of clinics simpler in Trakcare. The unknown location code has been used by several boards; predominantly by NHS Tayside and Borders, although use of the code is decreasing in 2021. New outpatient unknowns have decreased by over a half (59%) comparing July to September 2021 to the same quarter last year.

Data Management are looking at identifying codes that Boards could use so that the Health Board of treatment can be derived correctly, and the intention is that records already submitted using this code will be recoded.

General Issues – SMR01

Acute Assessment Unit (AAU) / Ambulatory Emergency Care (AEC) activity -  Definitions

Acute Assessment Unit (AAU)

The AAU is a dedicated facility for the acute clinical care of patients that present to hospital as clinical emergencies or who develop an acute clinical problem while in hospital. The units may also carry out some planned healthcare.

Generally, these units have both trolleyed areas and staffed beds which form part of the hospital’s bed complement. Where trolleys are used in lieu of beds, patients should be counted as inpatients.

Acute Assessment Unit (AAU) is the preferred term for services also known as:

  • medical/surgical assessment unit
  • combined assessment units
  • clinical assessment units
  • acute medical (assessment) units
  • paediatric assessment units
  • acute receiving ward/unit admission unit

These cases should be recorded under significant facility 40.

Ambulatory Emergency Care (AEC)

An Ambulatory Emergency Care Unit is a multidisciplinary ‘one stop’ service. It provides Outpatient and Day case services only.

These cases should be recorded under significant facility 39.

National recording of AAU & AEC activity

Currently AAU activity is only being submitted by some NHS Boards within SMR01. NHS Greater Glasgow & Clyde AAU activity stopped in 2017. NHS Highland have been submitting AEC cases via SMR01 using criteria agreed by PHS to ensure that they pass validation rules as an interim measure. NHS Greater Glasgow & Clyde has opted to record these cases differently from NHS Highland since they consider a number of these cases to be non-elective day cases which, due to recording rules, cannot be recorded that way on TrakCare. As such they took the decision to record them as Emergency Department activity to allow them to be able to follow the patient through the system. However, from September 2019 onwards, the Royal Alexandra Hospital within NHS Greater Glasgow & Clyde have started to record AAU cases again within SMR01.

NHS Western Isles and NHS Ayrshire & Arran started submitting AEC cases via SMR00 from June 2020 and July 2021 respectively. NHS Dumfries & Galloway started submitting AEC cases via SMR01 from August 2020 onwards. Data Management have been in contact with these NHS Boards to discuss whether the records should be updated to reflect a more appropriate significant facility code. NHS Western Isles have advised that they are updating the facility code to '11 – Other' and resubmitting. NHS Ayrshire & Arran have said that they have changed the Deep Vein Thrombosis (DVT) clinic held at their Clinical Decisions unit to significant facility 39 from July 2021. NHS Dumfries & Galloway have informed that these cases are part of a trial where Advanced Nurse Practitioners see suitable patients. If required, these will be passed to consultants to see under Significant Facility 40 Acute Medical unit. NHS Lothian have said they are not planning to update the significant facility codes they use; therefore, it remains difficult to quantify Ambulatory Care activity.

There are ongoing discussions with NHS Boards, the Scottish Government and PHS on the most appropriate way for capturing this activity including AEC cases. It is hoped that national definitions and guidance on how to record this activity can be agreed by all NHS Boards.

Transfers

Several NHS Boards have experienced changes in internal transfer activity, due to the way in which their data submissions have been submitted. However, it should be noted that any changes, both increases and decreases, are based on small numbers which will impact on the percentage change.

For more information, see the Glossary and the Data Files sections of this publication.

General Issues – SMR00

Recording of SMR00 procedures

The recording of procedures is not consistent across the NHS Boards. NHS Borders, NHS Dumfries & Galloway, NHS Grampian, NHS Lanarkshire, NHS Lothian and the NHS Island Boards (i.e. NHS Orkney, NHS Shetland, and NHS Western Isles) record procedures on less than 10% of records submitted. Other NHS Boards record procedures on from 10% to under 25% of records.

Recording of Return Attendances

The submission of all return attendances is mandatory regardless of whether or not a procedure is performed. However, there is variation in the NHS Boards submissions of return outpatients in SMR00. For this reason, ISD(S)1 has been used to extract return outpatient attendances.

General Issues – ISD(S)1

ISD(S)1 Return Outpatient and Beds NHS Board coding

Records have been identified with potentially duplicate information coming from more than one NHS Board for the same location. The numbers concerned are very low and the impact is not significant. Additionally, there are issues with the allocation of NHS hospitals to NHS Boards and private hospital activity to the 'non-NHS Provider code'. Data Management have investigated and found that in many of these cases, this can be attributed to visiting consultant activity and NHS Boards sending their patients to private locations.

NHS Board-specific Issues

Any information provided by the NHS Board is included in the Data Trends section of this publication. It should be noted that many trends observed will be influenced by, and attributable to the data completeness levels, small numbers, and the impact of COVID-19 highlighted above. Therefore, caution should be taken when comparing quarterly information.

Data Quality Assurance within PHS

Scottish NHS Boards have a responsibility to ensure their SMR data are accurate, consistent and comparable across time and between sources. The PHS Data Quality Assurance team (DQA) audit SMR data at NHS Boards to determine if they have been properly recorded in accordance with national rules and standards. The DQA team's assessment web page contains reports from past audits of inpatient/day case data, including findings on the accuracy of submitted SMR01 data items used in our analysis (specialty, admission type, etc.).

The Quality Indicators Secondary Care team who produce this publication also carry out quality assurance checks on the data after extraction from the databases. For example, they compare high-level NHS Board figures for the same quarter between the current and previous publications in order to identify any large changes in the data sources. Additionally, they look at trends within the current publication in order to identify any unusual patterns. For changes or patterns in the data which cannot be explained by the known completeness estimates, the team contact Data Management to highlight the issue. The Data Management team then contact the Board for an explanation. Any information provided by the NHS Board is included in the Trends section of this publication.

Last updated: 13 June 2022
Was this page helpful?