2015/16 Data Quality Sub-committee Chairman’s report - Mr Martyn Porter
The group met five times in the year, with the overarching focus being the implementation of the NJR’s supporting data quality strategy and initiating the NJR’s retrospective data quality audit of arthroplasty procedures recorded by the NJR for financial year 2014/15.
Read ‘Supporting data quality – NJR strategy 2014/16’
NJR Data Quality Audit
The requirement for more granular data publication under the transparency agenda continues to be the driving factor behind the need to develop a comprehensive programme to identify areas for improvement and ensure confidence and robustness of the data and its analysis.
Confidence in the NJR is based on the assumption of high quality data, rigorous analysis, and strong engagement with stakeholders (surgeons, patients, healthcare providers, implant manufacturers, and the MHRA). Whilst the NJR is fully engaged with stakeholders, and conducts robust statistical analysis, the completeness of data within the NJR is reliant on the input at unit level, which is currently subject to variation across trusts.
It is clear that for surgeons and patients alike, having accurate and complete data is an absolute requirement. Data quality and validation are essential components of any audit or scientific research. Quite simply, if the data is incomplete or incorrect, then false assumptions may be drawn from any analysis. With this in mind, the NJR – through the leadership of the Data Quality Sub-committee – conducted a retrospective data quality audit comparing record for record procedures of trusts’ Patient Administration System (PAS) to the NJR, with the aim to investigate the compliance of local hospital trusts’ reporting of arthroplasty procedures to the registry.
For the first time, through this ambitious project, the registry has been able to gauge levels of data quality at a procedure level, as well as establishing an NJR Data Quality Lead and NJR Clinical Lead in each NHS unit across the country.
NJR Data Quality Audit - outcomes
The NJR’s interim results on the data quality audit programme, compiled on 25 March 2016, showed that for those trusts who had completed the audit: whilst the overall scale of missing records was found to be low (3.4%), the proportion of missing revision records was found to be higher than that for primary procedures. The observed differences suggest an under-reporting of revision procedures in the audited trusts. The ratio of missing primaries for hip and knee replacements is approximately 31:1 and 32:1 respectively, whereas the ratio of missing revision procedures is 18:1 and 12:1 respectively.
To put the importance of this into context, compliance of NHS trusts with uploading revision procedures to the NJR is less than perfect. Whilst this may diminish power to detect failing implants quickly, the large size and wide coverage of the NJR compensates for this. However, when attempting to report results based on NJR sub-groups, such as hospital trusts or surgeons, the reduction in power and introduction of bias may misrepresent trust or surgeon performance.
Further details of the NJR's data quality work can be found here.
Engaging more NHS trusts and the independent sector
The Data Quality Sub-committee will continue to explore how best to further raise the NJR’s data quality initiatives as a priority for all NHS trusts and independent organisations from April 2016.
For example, now set to become an annual project, the data quality audit programme will be one way that hospitals can achieve Data Quality Provider status – designed to recognise best practice in NJR compliance and quality. For 2016/17 the NJR will also be inviting the independent sector to join the data quality audit programme and enrol as Data Quality Providers. During 2015/16 39 NHS trusts were awarded this status, and going forward the sub-committee will be looking to more effectively communicate the scheme.
Additionally, during 2015/16, the NJR completed four external hospital visits. This involved myself as NJR Medical Director, NJR Director of Operations, Elaine Young, and former NJR Chairman, Professor Paul Gregg. We visited hospitals and met with trust board members and senior level trust stakeholders to try and understand and rectify the blockage in communication and engagement with the NJR, as well as supporting hospital colleagues with the data quality audit. The NJR will be maintaining its programme of hospital visits as and when required.
I would encourage everyone involved and interested in this work to keep up to date by signing up to the NJR eBulletin or visiting www.njrcentre.org.uk to review the dedicated data quality area.