Data quality

This is a National Statistics Publication. National Statistics status means that this publication meets the highest standards of trustworthiness, quality and public value. This publication fully complies with the Code of Practice for Official Statistics (external website) and was awarded National Statistics status (external website) following an assessment in September 2011 (external website) by the UK Statistics Authority (external website).

Strengths and Limitations

In order to determine the quality of the statistics that we publish, PHS (Public Health Scotland) assesses the risk of data quality concerns (PDF) for each publication. Publications are assigned a low, medium or high data quality risk rating. This rating is based on factors such as the number of data suppliers involved, the complexity of the data collection process and the quality assurance checks applied to the data. This publication has been assigned a low risk of data quality concerns.

The following summarises how the data in this publication can be used, and the limitations which users should be aware of:

How can the data in this publication be used?

This data can be used to:

This data cannot be used to:

  • Compare areas across Scotland
  • View trends over time
  • Compare activity between different specialties, age and sex groups, or deprivation levels
  • Assess whether patients were treated within or outwith their own Health Board
  • Explore the relationship between hospital activity and living in a deprived area.
  • Directly compare with other UK countries
  • Combine quarterly data to approximate annual figures
  • Compare averages across different time periods
  • Identify numbers of patients (only stays or episodes)
  • Identify areas of affluence
  • Identify how much more deprived an area is from another.

Data Sources

The data used in this publication is sourced from the Scottish Morbidity Records (SMR) and ISD(S)1 datasets held by PHS.

SMR01 (Inpatients and Day Cases)

The SMR01 dataset comprises episode-based patient records relating to all inpatients and day cases discharged from non-obstetric, non-psychiatric specialties and excluding geriatric long stay records. Data are updated on a monthly basis and include clinical and non-clinical data.

SMR00 (Outpatients)

The SMR00 dataset contains patient-based information on appointments at outpatient clinics in all specialties (except A&E and Genito-Urinary Medicine) in NHSScotland. Data are updated on a monthly basis and include first and total outpatient activity and attendance status, for example Did Not Attends (DNAs).

ISD(S)1 (Beds and Return Outpatients)

ISD(S)1 is a set of aggregated summary statistics on activity and resources in hospitals in Scotland and is derived from monthly and quarterly returns from the NHS Boards. ISD(S)1 also does not record information on the age or sex of patients, nor the deprivation of the area in which they live. ISD(S)1 is the only source of bed occupancy and bed availability data and contains summarised data by NHS Board of Treatment, hospital and specialty. ISD(S)1 is also used for return outpatient activity since completeness for return outpatients in SMR00 is poor.

Data Collection and Validation

Image showing a data flow diagram  illustrating the process of data collection and validation

Communication with Data Supply Partners

The Secondary Care team in Data Management support data providers in the submission and quality of national data sets. The team meet regularly with providers to ensure any issues affecting the data are identified and resolved at an early stage. The team also provides an advisory service to data users, in particular to PHS analytical teams who produce official publications such as this one. PHS analysts are kept up to date with any issues regarding Scottish Morbidity Records (SMR) via the regular SMR Analyst Forum bulletin.

Summary of Data Completeness

The data used in this publication were extracted from the SMR national databases in April 2020, and the following table reflects SMR completeness as at 11 April 2020. Full details of SMR data completeness and timeliness can be found on the SMR Completeness and Timeliness web pages respectively.


NHS Board
SMR00
(new outpatients)

October'19-December'19
SMR01
(inpatients and day cases)

October'19-Dectember'19
Ayrshire & Arran 100% 100%
Borders 100% 100%
Golden Jubilee 100% 99%
Fife 100% 94%
Greater Glasgow & Clyde 100% 96%
Highland 100% 96%
Lanarkshire 100% 98%
Grampian 99% 98%
Orkney 100% 100%
Lothian 99% 96%
Tayside 99% 99%
Forth Valley 100% 5%
Western Isles 100% 100%
Dumfries & Galloway 100% 97%
Shetland 100% 99%
All NHS Boards 99% 94%

 

NHS Boards aim to submit SMR data to PHS 6 weeks following a hospital discharge or transfer, death or a clinic attendance. Completeness is calculated by comparing the number of records submitted to PHS against the total number of records that the Board estimate we should receive. NHS data providers will know how complete their SMR submissions are and the extent of any backlog. PHS calculates backlog as data which are received after 6 weeks.

PHS do not produce completeness levels for ISD(S)1, however, the Data Management team query any unusual numbers with NHS Boards. Note that estimates are applied by PHS to missing data in the ISD(S)1 dataset used for beds and return outpatient figures. Due to problems with the implementation of a new patient administration system, no ISD(S)1 data were submitted for NHS Highland, from quarters ending March 2014 until September 2015. Missing data were estimated using straight line estimation. Please refer to the Methods page for further information on the calculation of estimates.

Data Quality Issues

General SMR Issues

COVID-19

Due to the coronavirus illness (COVID-19), PHS anticipate that there might be some changes in terms of our regular statistical production. The current disruption to Scotland and the rest of the UK could affect the quality of some of our statistics, such as lower accuracy, or it could mean there is less detail available, such as fewer local and regional breakdowns. In some cases, the production of some data series may need to be suspended.

For this publication, it was decided not to contact the NHS Boards directly to ask whether they were aware of data quality issues which may affect this publication, as we would normally do. This decision was taken in order to allow the NHS Boards to focus their efforts locally on their COVID-19 response.

TRAKCare Patient Management System (PMS) Issues

NHS Lothian was the first board to implement TRAKCare PMS, starting in 2005. All NHS boards, apart from NHS Dumfries and Galloway and NHS Western Isles, have now implemented a version of this patient management system. There are known issues relating to SMR data submissions and unfortunately these issues are not standard across NHS Boards; much depends on which system version is in place and how proactive NHS Boards have been in implementing fixes as they become available. Data Management work closely with the NHS Boards and Intersystems (TRAK) to investigate issues as they are identified, however users should be aware that system issues can impact on data quality and often take some time to resolve.

SMR Replacement Records

It should be noted that PHS identified an issue in June 2018 whereby some SMR replacement records being submitted by TrakCare NHS Boards resulted in the original SMR record being amended in error. This caused some data items being over written which may have impacted on some local and national analysis. The Data Management team have developed a process to manually add replacement records to the national file. This will now only add a change where the replacement is less than 2 years from date of appointment or discharge and only for fields relating to admission, discharge and clinical details. Due to the ongoing issues with TRAK generating replacements in error, any records with changes to patient demographics will not be updated but will be held on a separate table for future investigation. Data Management now carry out this process on a monthly basis and the most noteworthy improvement that can be seen is an increase in the completeness of outpatient procedures being recorded.

From 1 April 2020, SMR replacement records will only be accepted from NHS boards if the clinic/discharge date is within 2 years of the submission date. Older replacement records will be rejected; however, it is possible for Data Management to make amendments to historic records on NHS boards’ behalf where appropriate.

It should be highlighted that Data Management are currently unable to approve SMR replacement records remotely. The resulting impact is that some data quality issues mentioned within this publication may have been fixed by NHS boards but can’t be updated in the national file until the replacement records have been accepted.

Auto-populated fields

TRAKCare PMS includes a facility to ‘auto-populate’ SMR data fields which was developed to make data entry easier for the NHS boards. The first episode in a Continuous Inpatient Stay (CIS) will be recorded appropriately, but subsequent SMR records in the same CIS could be completed with the same codes as the first episode. Some of the fields affected are Management of Patient, Admission Type, Admission Transfer From, Discharge Type and Discharge Transfer To. Data Management are unable to quantify how much of an impact this will have. Intersystems have noted that they are unable to correct this at the present time.

Non-NHS Provider Data

Non-NHS provider figures relate to patients treated in non-NHS locations such as private hospitals, hospices, nursing homes, care homes, etc. Patients who receive treatment at a private (independent) hospital which is paid for by the NHS board should be recorded within the SMR by the relevant NHS board. However, if a patient is treated privately (i.e. treatment paid for by patient or private insurer) and there is no NHS involvement then this activity will not be recorded within the SMR. The data recording / completeness of non-NHS provider data varies from year to year, therefore please treat any data provided with caution. It should also be noted that any changes both increases and decreases in activity are based on small numbers which will impact on the percentage change.

General Issues – SMR01

Acute Assessment Unit (AAU) / Ambulatory Emergency Care (AEC) activity -
Definitions

  • Acute Assessment Unit (AAU)

The AAU is a dedicated facility for the acute clinical care of patients that present to hospital as clinical emergencies or who develop an acute clinical problem while in hospital. The units may also carry out some planned healthcare.

Generally, these units have both trolleyed areas and staffed beds which form part of the hospital’s bed complement. Where trolleys are used in lieu of beds, patients should be counted as inpatients.

Acute Assessment Unit (AAU) is the preferred term for services also known as:

  • medical/surgical assessment unit
  • combined assessment units
  • clinical assessment units
  • acute medical (assessment) units
  • paediatric assessment units
  • acute receiving ward/unit admission unit

These cases should be recorded under significant facility 40.

  • Ambulatory Emergency Care (AEC)

An Ambulatory Emergency Care Unit is a multidisciplinary ‘one stop’ service. It provides Outpatient and Daycase services only.

These cases should be recorded under significant facility 39.

National recording of AAU & AEC activity

Currently AAU activity is only being submitted by some NHS Boards within SMR01. NHS Greater Glasgow & Clyde AAU activity stopped in 2017. NHS Highland have been submitting AEC cases via SMR01 using criteria agreed by PHS to ensure that they pass validation rules as an interim measure. NHS Greater Glasgow & Clyde has opted to record these cases differently from NHS Highland since they consider a number of these cases to be non-elective day cases which, due to recording rules, cannot be recorded that way on TrakCare. As such they took the decision to record them as Emergency Department activity to allow them to be able to follow the patient through the system. However, from September 2019 onwards, the Royal Alexandra Hospital within NHS Greater Glasgow & Clyde have started to record these cases using SMR01.

There are ongoing discussions with NHS Boards, the Scottish Government and PHS on the most appropriate way for capturing this activity including AEC cases. AEC is under the scope of SMR00 Modernising Review, and it is hoped that national definitions and guidance on how to record this activity can be agreed by all NHS Boards.

Transfers

Several NHS Boards have experienced changes in internal transfer activity, due to the way in which their data submissions have been submitted. However, it should be noted that any changes, both increases and decreases, are based on small numbers which will impact on the percentage change. For more information, see the Data Trends section of this publication.

General Issues – SMR00

Recording of SMR00 procedures

In the past couple of years, it has been noted that the recording of procedures is not consistent across the NHS Boards. NHS Borders, NHS Dumfries & Galloway, NHS Grampian, NHS Lanarkshire, NHS Lothian and the NHS Island Boards (i.e. NHS Orkney, NHS Shetland, and NHS Western Isles) record procedures on less than 10% of records submitted. Other NHS Boards record procedures on from 10% to under 25% of records. This issue will be looked at as part of the SMR00 Modernising Review.

Recording of Return Attendances

The submission of all return attendances is mandatory regardless of whether or not a procedure is performed. However, there is variation in the NHS Boards submissions of return outpatients in SMR00. For this reason, ISD(S)1 has been used to extract return outpatient attendances.

General Issues – ISD(S)1

ISD(S)1 Return Outpatient and Beds NHS Board coding

Records have been identified with potentially duplicate information coming from more than one NHS Board for the same location. The numbers concerned are very low and the impact is not significant. Additionally, there are issues with the allocation of NHS hospitals to NHS Boards and private hospital activity to the ‘non-NHS Provider code’. Data Management have investigated and found that in many of these cases, this can be attributed to visiting consultant activity and NHS Boards sending their patients to private locations.

Board-specific Issues

Any information provided by the NHS board is included in the Data Trends section of this publication. It should be noted that many trends observed will be influenced by and attributable to the data completeness levels and small numbers. Therefore, caution should be taken when comparing quarterly information.

Data Quality Assurance within PHS

Scottish NHS Boards have a responsibility to ensure their SMR data is accurate, consistent and comparable across time and between sources. The PHS Data Quality Assurance team (DQA) audit SMR data at NHS Boards to determine if they have been properly recorded in accordance with national rules and standards. The DQA team’s assessment web page contains reports from past audits of inpatient/day case data, including findings on the accuracy of submitted SMR01 data items used in our analysis (specialty, admission type, etc).

The Quality Indicators Secondary Care team who produce this publication also carry out quality assurance checks on the data after extraction from the databases. For example, they compare high-level NHS Board figures for the same quarter between the current and previous publications in order to identify any large changes in the data sources. Additionally, they look at trends within the current publication in order to identify any unusual patterns. For changes or patterns in the data which cannot be explained by the known completeness estimates, the team contact Data Management and the relevant NHS Board to highlight the issue and request an explanation. Any information provided by the NHS Board is included in the Trends section of this publication

Areas for Improvement

In December 2016 the Scottish Government published The Modern Outpatient: A Collaborative Approach 2017-2020 (external website) that aims to deliver care closer to the patients’ home, provide more person centred care, utilise new and emerging technologies, and maximise the role of clinicians across Primary, Secondary and Community based services. PHS Data Advice has identified gaps in the existing national dataset (SMR00) that does not allow the full pathway to be appropriately recorded.

In order to meet the objectives that were set out in the “Modern Outpatient” agenda and to ensure our secondary care datasets meet future information needs, PHS has established a modernisation program of all SMR datasets, with an initial focus on outpatients. This would support patient and service management at NHS Board level as well as providing more accurate information at a national level.

The SMR00 Modernisation work may have an effect on the number of SMR’s submitted. In addition, other disciplines of staff (e.g. nurse practitioners) are increasingly carrying out care for patients which may impact on the number of consultant clinics run.

Last updated: 07 October 2022
Was this page helpful?