Urodynamics quality assurance

Introduction

To be assured of good quality measurements and accurate urodynamics data recording, a guide for standardising urodynamics best practice (see www.fundingawards.nihr.ac.uk/award/15/150/05) was developed in conformity with the ICS Good Urodynamics Practices.69 This guide and process were deemed necessary, as concerns had been raised previously in the literature about the impact of the quality of urodynamic studies on the reliability of diagnostic results.99 A ‘think tank’ had considered it was clear that technique affects the quality of a urodynamic test, and with other factors it will affect the utility and perceived value of that test.99 One publication looked at data from two large male urodynamics trials and found that a significant proportion of sites did not undertake equipment calibration checks or had findings that were affected by artefacts in the signal.100 A robust methodology for obtaining a measure of QA was therefore developed for the FUTURE trial to give an indication of how far this problem of quality might have affected the data collected.

The guiding principle was that the trial includes an appraisal of current urodynamic practices throughout the UK. The role of the QA process was not to force sites into compliance with a rigid protocol – which would be a kind of ‘urodynamics police’ – but to ensure that the data used for analysis were based on a reasonable level of assurance for its quality yet reflected the standard current clinical practice in the UK.

Methodology

A system was established to review urodynamic traces from all participating sites, to give assurance that the quality of the urodynamic technique and interpretation was up to standard for use in the trials data analysis.

Initially, the guide for urodynamic best practice was circulated to all sites and the details were explained during the initial trial meetings. An outline of the contents of the guide is given in Table 27.

The process of urodynamics QA in the trial involved two stages:

• Stage 1: baseline review stage; reviewing two non-participants’ urodynamic traces from all participating sites.
• Stage 2: trial data review stage; reviewing a random selection of participants’ urodynamic traces from each participating site.

Each stage involved a review by a panel of two experts which consisted of the urodynamics engineering expert (FUTURE co-applicant) and one of the clinical co-applicants on the trial.

Stage 1: baseline review

The trial office requested all participating sites to send two anonymous urodynamic traces and the clinician reports including interpretation and diagnoses. These two traces were from their clinical practice prior to being accepted as a collaborating site (i.e. not FUTURE trial participants). Having confirmed anonymisation, the QA manager sent them to two reviewers from the QA review panel. Reviewers commented on the quality of the urodynamics technique and trace interpretation. The feedback was sent back to each site, including any advice and recommendations for improving urodynamics performance for the trial urodynamics stage.

If the reviewers raised minor issues with either the performance or the interpretation done by the site, the trial office arranged further discussion and clarification with the relevant site. If a ‘red flag’ was raised (defined as anything above a minor comment within the filling phase), the FUTURE chief investigator (subspecialist urogynaecologist) further reviewed the urodynamics trace and report, re-discussed with reviewers if required and further discussed the concerns and the remedy plan with the PI in the relevant site. Training sessions by an expert reviewer from the review panel were offered to support the performance and interpretation of urodynamics within the trial. These sites received closer monitoring within the trial (Figure 15).

FIGURE 15 Urodynamics QA flow chart. UDS, urodynamics. (Figure not reproduced here; see the original article.)

Stage 2: trial data review

For every participant randomised to the urodynamic and CCA arm (referred to as the urodynamics arm), the site uploaded a copy of the urodynamics trace onto the trial website (with the clinician report) and completed the FUTURE urodynamics CRF (see www.fundingawards.nihr.ac.uk/award/15/150/05). Once sites had uploaded 10 traces, two were randomly selected for review by the review panel (random 20% check). However, for sites that recieved a ‘red flag’ at baseline (see Figure 15), two urodynamic traces were randomly selected after five traces had been uploaded.

The selected traces were validated and anonymised by the trial office and, with a copy of the urodynamics CRF, submitted to the urodynamics QA manager. The urodynamics QA manager co-ordinated the process thereafter: submitting the trace and urodynamics CRF to two members of the review panel; noting any concerns raised by the reviewer(s) and any actions required by site; co-ordinating the involvement of the chief investigator (similar to the baseline stage) and developing feedback to sites. If minor concerns or issues were identified by the reviewer(s), these were discussed with sites, a consensus was reached and the database updated as required. In situations where there was no response from the site, the chief investigator undertook a further review of the trace and urodynamics CRF, and the database was updated by the trial office as required. These decisions were recorded on a file note and communicated to the site by e-mail.

During the trial, the urodynamic QA reviewers pro forma was developed (see www.fundingawards.nihr.ac.uk/award/15/150/05) to streamline the QA process.

Results

Stage 1: baseline review

One hundred and twenty-two anonymous urodynamic traces and reports were reviewed by two reviewers from the FUTURE team: two urodynamic traces from each site (these were not trial participants, as explained above):

• Seven sites (6%) had minor concerns raised regarding the urodynamic technique/performance. The CI had discussion with the site and agreed on steps for improvement.
• Three sites (5%) were ‘red flagged’ by the reviewers in terms of the urodynamics technique which may have impacted on the quality of the urodynamics traces and results. The chief investigator arranged site training with an expert reviewer within the team.
• The remaining urodynamic traces and report had no concerns or a minor concern raised. This feedback was passed to the sites by e-mail.

Stage 2: trial data review

One hundred and twenty-four anonymous randomly selected urodynamic traces and CRFs from trial participants were reviewed from 61 sites. Four sites did not submit any urodynamic traces because they did not have anyone at their site randomised to the urodynamics arm.

The number of urodynamic traces and CRF reviewed per site were as follows:

• 27 sites had 1 urodynamic trace/CRF reviewed
• 20 sites had 2 urodynamic traces/CRFs reviewed
• 6 sites had 3 urodynamic traces/CRFs reviewed
• 5 sites had 4 urodynamics traces/CRFs reviewed
• 1 site had 5 urodynamic traces/CRFs reviewed
• 1 site had 6 urodynamic traces/CRFs reviewed
• 1 site had 8 urodynamic traces/CRFs reviewed.

Of the 124 urodynamic traces/CRFs that were reviewed by the panel:

• 60 urodynamic traces/CRFs (48%) received reviewers’ feedback that was in full agreement with the site in both diagnosis and data entry; therefore, no further action was required.
• 47 urodynamic traces/CRFs (38%) required minor action, generally an update to the volumes recorded during the voiding phase, which were of limited relevance to the urodynamics diagnosis. The trial office contacted the sites for clarification and the database was updated accordingly.
• 16 urodynamic traces/CRFs (13%) were considered by the reviewer(s) to be suboptimal:
  • 2/16 the reviewers reported the quality of the scan as poor. The trial office contacted the site, who provided clearer traces.
  • 4/16 had reviewer comments on axis scales and 6/16 had reviewer comments on the placement of event markers. These comments were relayed to the sites and CRF data adjustments recommended, and the database was updated accordingly.
  • 8/16 had reviewer comments on poor QA checks during the study, and 9/16 had comments on resting pressures or zero setting. Again, these comments were fed back to sites for technique improvement.
• Two urodynamic traces/CRFs (2%) were considered to deviate from the FUTURE treatment pathway (e.g. recommending BoNT-A treatment in absence of DO). After further review by the chief investigator, file notes were added by the trial office to indicate the pathway deviation.
• Five urodynamics traces/CRFs (4%) had disagreement by both reviewers regarding the clinical diagnosis recorded by site. The chief investigator discussed this with the PIs in the relevant site and a change of diagnosis was agreed. The trial office updated the database on behalf of the sites.

Conclusions

The baseline review process for the urodynamic traces prior to the sites commencing recruitment within the FUTURE trial was very helpful to address different urodynamic techniques and procedures within the collaborating sites. The training and advice provided by the urodynamics engineering expert (co-applicant), to improve the quality of the urodynamics and the process of interpretation, were well received by all collaborating sites.

In the pre-trial review of 122 urodynamic traces and reports, it was reassuring that 116 (95%) had acceptable quality of traces. The remaining three sites (5%) received direct advice and training before recruitment began.

The presence of a well-defined and agreed QA system within the trial was key for keeping all collaborating sites aware that the quality of their urodynamics process was regularly monitored in a supportive and constructive way.

Of the 124 randomly selected traces reviewed from the trial data, 107 (86%) had no or only minor comments raised by the reviewers. Eight (6%) urodynamic traces/CRFs had questions raised regarding the QA checks (considered suboptimal traces by reviewers) and five urodynamic traces/CRFs (4%) required a diagnosis change following their review.

To keep the standardisation of the urodynamics performance and to gain QA, the feedback provided by the two review panel members was vital. In addition, the sites’ responses and communication were key to reaching the aim of consistency and quality in the urodynamic interpretation, thus making the diagnosis in the trial more accurate and meaningful.