Annual progress: Work of NJR Committees

2015/16 Regional Clinical Coordinators (RCC) Sub-committee Chairman’s report - Mr Matthew Porteous

The role of the RCCs is to provide a network of surgeons in all the regions and countries contributing to the NJR.  Their role is to facilitate two way communications between the NJR and its contributors.  They assist the Regional Coordinators in helping hospitals who enter data into the NJR and are experiencing difficulties, and at the same time provide a route for both problems and ideas for improvement to be fed back centrally by contributing organisations. 

Comprising both a geographically representative group of clinicians as well as representatives from HQIP and Northgate Public Services, it also provides a source of interested clinicians to sit on the Research, Editorial Board and other NJR sub-committees.  

The past year has been a particularly busy one for committee members as they have worked with the Regional Coordinators to implement the Data Quality Audit programme (of which more is written elsewhere). Committee members have played a major role in getting all but one NHS hospital in England to contribute their data to the audit.  In order to facilitate this, and better communicate with individual hospitals, RCCs have been responsible for ensuring that every hospital in the UK now has an NJR Clinical Lead, who is our first point of contact for any issues connected with the NJR.  Using this senior clinician to clinician contact, we have been able to resolve many issues quite rapidly and look forward to maintaining and using this network as the audit is repeated in the coming financial year.

At the end of 2015 a number of RCCs retired from the role, many of whom had served on the committee for many years and played a significant role helping get the NJR off the ground in its early years.  I would like to take this opportunity to thank them for their contribution in making the NJR, and indeed the RCC network, the success it has now become.

Eight new RCCs started on the committee in 2016 and their appointments have improved the representation of upper limb and foot and ankle surgeons serving on the committee.

2015/16 Data Quality Sub-committee Chairman’s report - Mr Martyn Porter

The group met five times in the year, with the overarching focus being the implementation of the NJR’s supporting data quality strategy and initiating the NJR’s retrospective data quality audit of arthroplasty procedures recorded by the NJR for financial year 2014/15. 

Read ‘Supporting data quality – NJR strategy 2014/16’

NJR Data Quality Audit

The requirement for more granular data publication under the transparency agenda continues to be the driving factor behind the need to develop a comprehensive programme to identify areas for improvement and ensure confidence and robustness of the data and its analysis.

Confidence in the NJR is based on the assumption of high quality data, rigorous analysis, and strong engagement with stakeholders (surgeons, patients, healthcare providers, implant manufacturers, and the MHRA).  Whilst the NJR is fully engaged with stakeholders, and conducts robust statistical analysis, the completeness of data within the NJR is reliant on the input at unit level, which is currently subject to variation across trusts.

It is clear that for surgeons and patients alike, having accurate and complete data is an absolute requirement.  Data quality and validation are essential components of any audit or scientific research.  Quite simply, if the data is incomplete or incorrect, then false assumptions may be drawn from any analysis.  With this in mind, the NJR – through the leadership of the Data Quality Sub-committee – conducted a retrospective data quality audit comparing record for record procedures of trusts’ Patient Administration System (PAS) to the NJR, with the aim to investigate the compliance of local hospital trusts’ reporting of arthroplasty procedures to the registry.  

For the first time, through this ambitious project, the registry has been able to gauge levels of data quality at a procedure level, as well as establishing an NJR Data Quality Lead and NJR Clinical Lead in each NHS unit across the country. 

NJR Data Quality Audit - outcomes

The NJR’s interim results on the data quality audit programme, compiled on 25 March 2016, showed that for those trusts who had completed the audit: whilst the overall scale of missing records was found to be low (3.4%), the proportion of missing revision records was found to be higher than that for primary procedures. The observed differences suggest an under-reporting of revision procedures in the audited trusts. The ratio of missing primaries for hip and knee replacements is approximately 31:1 and 32:1 respectively, whereas the ratio of missing revision procedures is 18:1 and 12:1 respectively.

To put the importance of this into context, compliance of NHS trusts with uploading revision procedures to the NJR is less than perfect. Whilst this may diminish power to detect failing implants quickly, the large size and wide coverage of the NJR compensates for this. However, when attempting to report results based on NJR sub-groups, such as hospital trusts or surgeons, the reduction in power and introduction of bias may misrepresent trust or surgeon performance.

Further details of the NJR's data quality work can be found here.

Engaging more NHS trusts and the independent sector

The Data Quality Sub-committee will continue to explore how best to further raise the NJR’s data quality initiatives as a priority for all NHS trusts and independent organisations from April 2016. 

For example, now set to become an annual project, the data quality audit programme will be one way that hospitals can achieve Data Quality Provider status – designed to recognise best practice in NJR compliance and quality. For 2016/17 the NJR will also be inviting the independent sector to join the data quality audit programme and enrol as Data Quality Providers. During 2015/16 39 NHS trusts were awarded this status, and going forward the sub-committee will be looking to more effectively communicate the scheme. 

Additionally, during 2015/16, the NJR completed four external hospital visits. This involved myself as NJR Medical Director, NJR Director of Operations, Elaine Young, and former NJR Chairman, Professor Paul Gregg.  We visited hospitals and met with trust board members and senior level trust stakeholders to try and understand and rectify the blockage in communication and engagement with the NJR, as well as supporting hospital colleagues with the data quality audit. The NJR will be maintaining its programme of hospital visits as and when required.

I would encourage everyone involved and interested in this work to keep up to date by signing up to the NJR eBulletin or visiting www.njrcentre.org.uk to review the dedicated data quality area.

2015/16 Research Chairman’s report - Professor J Mark Wilkinson

The Research Sub-committee (RSC) is responsible for delivering the research agenda of the National Joint Registry. The NJR is a uniquely powerful resource that provides a foundation for definitive research into the full range of biological, mechanical, clinical, economic, and social factors influencing the outcome of joint replacement. The data can also help establish the impact of joint replacement surgery on the well-being of patients and the general population.

The RSC is working to maximise the value of the registry by making data widely available to researchers, and also developing means to protect the NJR dataset and strengthen its governance through safe, effective, and efficient data management. The committee takes formal responsibility for the release of data for research through an impartial and objective mechanism and has oversight of the use and reporting of NJR data by research groups. 

Research application pathway

We have a commitment to uphold the standard and consistency of work that is carried out on NJR data in line with UK legislation. Our priority is to ensure that any research question justifies access to the data and offers potential benefit to patients. Research should align to the NJR’s priority framework and be feasible, ethical, relevant and methodologically sound.

RSC has implemented a single entry point and review pathway for all research applications and activity arising within or external to NJR. Applications requesting unlinked and fully anonymised NJR data are directed down the ‘external’ route, whilst applications requesting access to linked datasets (for example PROMs, HES or PEDW data) or patient identifiable data are directed via the ‘internal’ and collaborative project route in association with an RSC or NJR Steering Committee member. 

If you would like to submit a research application, the first step in this process is to complete an expression of interest. This form and further guidance information can be accessed on the research pages of the NJR website [link].

Annual dataset and Data Access Portal

Work is progressing well in delivery of a “research-ready” dataset and online Data Access Portal for researchers. These outputs will enable researchers to utilise the NJR Annual Report dataset more effectively as a research resource, reducing the burden on researchers by providing a single, “clean” source of data that is updated each year with a new cumulative dataset.

Research Fellows

The RSC continues to foster its Research Fellowship scheme in partnership with the Royal College of Surgeons of England, designed to build registry-based research capacity within the clinical orthopaedic community. We are pleased to confirm recruitment of the sixth Research Fellow is currently underway, with aims to appoint for autumn 2016. 

Use of NJR data this year

During the period 2015/16, the committee sanctioned the release of data to ten research projects, seven of which are externally-led projects. The NJR dataset has been used as the key resource in 13 research papers published during this same time period, of which ten were conducted or led by external, independent researchers. Further details are published on the website in the NJR Research Project Portfolio, which catalogues all sanctioned research projects, publications, abstracts, progress reports and NJR updates in relation to research using the NJR dataset together with links to all publications related to NJR data.

We are currently addressing in depth a number of different topics which relate to our identified priority areas, including factors affecting mortality, morbidity and patient outcomes after joint replacement surgery; infection demographics, predictors and outcomes; surgeon education, training and volumes; the provision of arthroplasty according to ethnicity in the UK; mixing of components from different manufacturers in total hip arthroplasty, and outcomes of total hip replacement (THR) in minority patient groups. 

Research is underway to support development of a personalised outcomes forecasting tool for patients considering hip or knee replacement, an example of use of the NJR dataset to inform decision-making at the individual patient and surgeon level. We are also carrying out work addressing data quality and the completeness of the NJR dataset, accuracy, and systematic bias and recommendation on approaches to minimise their impact.

A full list of publications related to NJR data is included in Appendix 4.   
 
Further information

Full details of the NJR’s research application pathway, together with detailed guidance notes for applicants, can be found at www.njrcentre.org.uk.

2015/16 Surgeon Outliers committee Chairman’s report - Mr Peter Howard

The analysis of the registry data has continued on a six-monthly basis. The sub-committee meets soon after the production of each report, and then again three months later to monitor responses. Each report has seen the identification of further potential outliers based on the NJR’s funnel plot analyses. From March 2015 the last five years of data was added as an additional analysis. This is examined on a rolling basis, allowing more contemporary practice to be observed, and also offering a reflection of greater data accuracy. It is particularly useful where a surgeon/unit has made a significant change in their implant usage and/ or practice.

Annual Clinical Report 

An Annual Clinical Report (ACR) is issued to Trusts, Health Boards, independent organisations and units, and to surgeons. The ACR continues to be the mechanism that demonstrates overall performance of the individual organisation, compared to the rest of the registry, on the same measures used in previous reports, namely: data quality; mortality; and revision rates of primary procedures. Funnel plots are shown for the different sub-types of hip and knee replacement as outlined in previous annual reports. The funnel plots now show the position in the previous year’s report, and indicate which surgeons are active and inactive within the unit. 

Now also shown for the last five years of data are revision rates for the unit, its individual surgeons, and separate resurfacing from large head metal-on-metal hip replacement(s) . The sub-types of knee replacement continue to be displayed separately, as well as showing the combined rates.

Clinician Feedback

The NJR’s Clinician Feedback website has several new features of value to surgeons. The reports tab enables all primary procedure details to be accessed and downloaded, and includes the current status of the patient. Similar details of revisions will soon be available, as will access to all relevant Annual Clinical Reports and funnel plots of the latest five years’ data.

Outlier methodology

The outlier methods are under on-going review by the committee, which includes the NJR statistical analysis team, and ad hoc discussions with other organisations such as the Scottish Arthroplasty Project, with its experience with the cumulative summation (CUSUM) test. It is anticipated that the funnel plot method used for the last six years will continue, and will be augmented in the future, possibly with the CUSUM test.

Management process of potential Surgeon Outliers

The process for managing surgeons whose data shows a standardised revision or mortality ratio significantly greater than the expected has evolved over the last six years. It has been formally reviewed and documented this year, and agreed through the Medical Advisory Committee.

2015/16 Implant Performance Chairman’s report - Mr Peter Howard

The Implant Performance Committee comprises members of the Scrutiny Group together with representatives from industry. During 2015/16, the sub-committee held two meetings with a wider invitation to the implant industry: at the 2015 BOA Congress in Liverpool, and again in January 2016. The Scrutiny Group includes surgeons, representatives from the MHRA, Northgate Public Services, the University of Bristol and the NJRSC Chair. Both groups are supported by HQIP.

Examining the data

The NJR data is refreshed and analysed every six months. The Scrutiny Group assesses potential outlier implant performance and notifies the MHRA when there are concerns. This process is refined constantly and, as a result of recent reviews, it is likely that more comprehensive reports will be developed for the future. An implant (and implant combinations in hips) is considered as a potential outlier when there are more than 100 cases and the overall revision rate (based on the patient-time incidence-rates (PTIR)) is more than twice the expected rate, using non-overlapping confidence intervals for statistical significance.

When an implant has been notified to the MHRA because of its poor performance, once a period of three years has elapsed then data for the whole NJR is analysed to give baseline PTIRs with the outlier data excluded. This is similar to the method used in assessing surgeon performance.

Notification from surgeons of poor performance

We continue to encourage surgeons who have concerns about any implant brand or a particular range to contact us directly. This enables us to focus on a specific area that might otherwise be camouflaged by the overall brand if the remainder is performing well.

Data from the other major joint replacement registries is always monitored and used to provide areas of focus for our analyses.

Future development

As always, we are looking to improve our skills at identifying poor performance, whilst at the same time identifying excellent performance. 

Foremost in this collaboration is the development of an international database of implant attributes, which will facilitate improved implant analysis and international comparisons. The database is now established and is being populated with data from the implant companies.

Implants and combinations identified as outliers

Details of implants and combinations identified as outliers are again included in Part 4 of this year’s report. There has been one new notification in knee replacements in the last year, and one hip combination (again a large head metal-on-metal device) – both discontinued several years ago.

2015/16 Editorial Board Chairman’s report - Mr Martyn Porter

The role of the Editorial Board is to oversee NJR communications that include:

1. The timely production of the NJR annual report
2.  A series of publications aimed at patients: Public and Patient Guides to joint replacement surgery
3. The NJR website

Annual Report

The format of the annual report is produced in partnership with Northgate Public Services, the main informatics provider and the University of Bristol, the main analytic provider. The format and contents of the report are continually under review with important contributions are made from the orthopaedic specialist societies, industry and patients. Particular attention is given to the regular or standard reports and the more detailed and research orientated topics that are considered to be of importance to the stakeholder community.

Two years ago we introduced a web based interactive component of the annual report that allowed the user to interrogate the data to produce tailored reports. This reduced demand on the published report and allowed more interaction from the public, as well as offering the potential for more detailed data to be available electronically. 

Enhancing the digital format 

With the continued development of the NJR Reports website, members of the Editorial Board provided invaluable contributions towards the interactive filter options offered in the presentation of clinical activity data for all joint types. The intuitive, flexible options help readers to engage with the data presented and have been well received by users of the NJR Reports website. 2015/16 also saw the inclusion of the interactive NJR’s Public and Patient Guides to joint replacement surgery on the website (see below).

Promoting the 12th Annual Report findings

Other achievements for the year included the launch of the 12th Annual Report at the BOA Congress in Liverpool in September 2015. Sessions organised at the Congress were shaped by two key themes; key messages from the 12th Annual Report including research on ethnicity and mix and match implants, and a discussion on NJR international collaboration. The latter session invited thoughts on International Consortium of Orthopaedic Registries’ (ICOR) development, harmonisation and clinical improvement from Professor Stephen Graves, Director, Australian Orthopaedic Association NJRR, and from Liz Paxton, Director of Surgical Outcomes and Analysis, Southern California Permanente Medical Group.

Public and Patient Guide progress

The Editorial Board has been particularly pleased to see the continued development of the NJR’s Public and Patient Guides to joint replacement surgery (PPGs). The patient-friendly versions are offered in five editions covering hip, knee, ankle, elbow and shoulder surgery separately. This provides greater potential for future development as the data and analysis available for each joint varies. 

The Editorial Board acknowledges that the guides’ readership still only represents a small proportion of pre-operative patients annually. 2015/16 saw a more targeted promotion of the PPGs identifying places where patients would receive information at the most appropriate time, i.e. before their hospital appointment. Alongside providing PPGs to all pre-operative units nationally, the NJR trialled the distribution of the PPGs to all NHS Clinical Commissioning Groups (CCGs). The feedback was mixed but a modest number of CCGs have since added NJR details and resources to their websites.

Aligning in-depth studies and research interests

The Editorial Board has always supported an in-depth research study programme and is delighted to see the developments in the NJR’s approach to research and the strategic direction being shaped by NJR Research Sub-committee Chairman, Professor Mark Wilkinson.  2015/16 saw the Editorial Board sharing developments in the annual report data and outcomes analysis with the NJR’s Research Sub-committee in order to ensure that pertinent in-depth studies can be taken through the internal research application pathway.

The NJR Reports Website

The website is continually reviewed and updated to reflect the requirements of NJR reporting.

2015/16 NJR Medical Advisory Committee Chairman’s report - Mr Martyn Porter

The NJR’s Medical Advisory Committee meets quarterly to provide clinical leadership and oversight to the registry’s work and provide a forum for discussion and engagement with the profession through specialist orthopaedic societies.

In addition to the NJR Steering Committee and NJR Regional Clinical Coordinator Sub-committee, surgeon members, representatives from industry, and representatives from the British Hip Society, British Association for Surgery of the Knee, British Foot and Ankle Society and the British Elbow and Shoulder Society also attend.

Key areas of consultation with the Medical Advisory Committee in 2015/16 included:

• Helpful and collaborative discussion on the Getting It Right First Time national review of which NJR data plays a significant part
• Publication of individual consultant surgeon outcomes
• Publication of hospital-level outcomes through information ‘dashboards’
• NJR’s approach to supporting unit-level data quality 
• Advice on developing relationships with national bodies such as the Private Healthcare Information Network (PHIN) 
• Supporting data requirements for specialised commissioning
• Supporting regulation and service review
• Discussing broader applications for audit data to improve orthopaedic outcomes
• Supporting patient safety

2015/16 NJR Executive Committee report
- Laurel Powers-Freeling 

The NJR Executive committee meets monthly to provide additional operational capability to the NJR Steering Committee’s strategic and governance role. The committee membership alternates on a bi-monthly basis between a ‘core group’, comprising the NJRSC Chairman, NJR Medical Director and NJR Director of Operations, and an ‘extended group’ with additional representatives comprising Chairman of the Sub-committees and main contractors, as required. Please see the online appendices for a record of members and attendance at meetings.

Importantly, the Executive Committee has delegated authority from the NJRSC on operational decisions under a financial threshold of £50,000 and where projects constitute a change in the strategic direction of individual work streams and projects. 

Given the expansion of the NJR’s work in recent years, as set out in the refreshed NJR Strategic Plan for 2015/18, this committee has already provided invaluable support. It is essential that the registry can remain flexible and responsive as stakeholder requirements and demands change.

2015/16 NJR Steering Committee

This section provides brief outlines of the work undertaken by the NJR Steering Committee (NJRSC) and its sub-committees during 2015/16. Although brief, these sections intend to demonstrate the ever-increasing scope of the registry’s work and the commitment required by its members to meet the needs of its orthopaedic stakeholders.

Structure, governance and operating model

The NJR Steering Committee is a NHS England Committee of Experts and is responsible for the work and progress of the NJR.

New members

Working closely with the Department of Health appointments team and NHS England, the NJR management team and the NJRSC Chairman and members have successfully re-appointed and appointed the following members in the reporting period:
 

The co-opted position for the British Orthopaedic Association President was taken up by Mr Tim Wilton on his election in September 2015 and Mr Matthew Porteous, Chairman, Regional Clinical Coordinator Sub-committee, supported the work of the NJRSC as a co-opted member. Professor Tim Briggs continued as a co-opted member of the NJRSC for the 2015/16 year in his capacity as lead of the national ‘Getting It Right First Time’ project. 

NJR must thank all of its members, past and present, for their considerable contribution to the registry. 

The membership of the NJR Steering Committee and membership of all sub-committees for 2015/16 can be found in online appendices. Further information can also be found in the Chairman’s introduction.