Findings from the micro level: pragmatic comparison of patient consultations by physician associates and Foundation Year doctors in the emergency department

This chapter reports on the quantitative evidence identified through one strand of the workstream 3 case study, that is, a pragmatic retrospective assessment comparing the outcomes and service costs for patients attended by PAs or FY2 doctors in the ED, using anonymised patient records from hospital databases. These records were analysed to describe and compare PAs’ and FY2 doctors’ consultation records in the ED, including assessment of the study’s primary outcome, that is, re-attendance within 7 days of initial attendance at the ED, as well as of the secondary outcomes of consultation processes, the clinical adequacy of care and economic considerations.

We report first on a description of the consultation records, followed by a comparison of process and clinical outcomes, then the economic analysis.

Description of the consultation records

Study numbers

In the 16-week period studied, 8816 patients attended by PAs or FY2 doctors were identified by the three trusts within the case studies in which PAs worked in the ED. As each trust could provide different data items requested for the study from their routine electronic databases, a data set containing all the data items required to support our analysis has only been collected for the number of patient records required to meet our required sample size of 304 patients in each group, that is, patients whose consultation was attributed to either a PA or a FY2 doctor, to test a non-inferiority hypothesis on the primary outcome measure, which stated that PAs do not exceed a rate of unplanned reconsultations of 27.4%, with 80% power at 5% significance.

With requests for the same number of consultation records for PAs and FY2 doctors in each site, and some replacement consultation records when personnel coding errors were noted, we slightly over-recruited on FY2 doctors’ consultation records and included patient consultation records for 613 ED attendances in total (Table 12).

Demography of the patients in the sample

The sample is described in terms of the patients’ age bands and sex only, owing to limitations on the provision of other data items.

The patients consulting both PAs and FY2 doctors show a wide range in ages and a fair spread across the sexes (Table 13). No statistically significant differences were found in patient demographics between groups seen by PAs and those seen by FY2 doctors.

Characterisation of the clinical condition of the patients in the sample

The clinical condition of the sample of patients on presentation at the ED is characterised through the measures of the acuity of the condition (the early warning score and/or MTS87) as well as the ED stream or area to which the patient is initially allocated. A description of the distribution of acuity of patients seen by PAs or FY2 doctors is given in Table 14.

Descriptively, no differences were seen in the early warning score (although this was only provided by two of the three trusts included in the data set); however, both the MTS87 and the area of the ED in which the patient is seen show statistically significant differences; compared with FY2 doctors, PAs held a higher proportion of consultations in the standard and immediate categories (rather than the urgent category in the majors section of the ED, although in 150 cases the ED area was not available in the data set). In this sample, the number of paediatric patients seen in the sample as a whole was small; PAs saw a greater proportion of these patients.

Process outcomes

The clinical process outcomes within the ED visit have been measured as follows: length of time in the ED, whether or not the patient had an X-ray investigation carried out, whether or not a prescription was issued, whether or not a discharge summary was completed and the destination from the ED (admission rate).

Clinical process outcomes

The proportion of patient visits with each of these process measures reported is shown in Table 14.

Statistical tests have been conducted to see whether or not there are any differences in the clinical process outcomes between PA and FY2 doctors. The difference was tested using LR tests in unadjusted and adjusted analyses. LR χ² (#) refers to the LR χ² statistic with # degrees of freedom (df). For adjusted analysis, age, sex, MTS and sites were controlled for and interaction between site and profession is also included if the interaction term is significant.

The results suggest that there are no differences in whether or not prescriptions were given in the ED, whether or not the patient was admitted to hospital or whether or not a discharge summary was completed, but patients seeing a PA differed from those seeing a FY2 doctor regarding whether or not X-ray investigations were conducted. Patients seen by a PA appeared more likely to have an X-ray carried out in the ED than those seen by a FY2 doctor (Table 15). As adjustment for case-mix differences was limited to measures of acuity on presentation, it is not known whether or not any difference in the presenting conditions seen by the PAs or FY2 doctors could have contributed to this finding.

Emergency department process: length of stay

The length of stay in the ED was available for all cases in the sample. Table 16 shows the distribution of the times.

A sharp peak at a length of stay of 3 to 4 hours is noted (Figure 5).

FIGURE 5 Distribution of length of stay in the ED (all cases). (Figure not reproduced here; see the original article.)

Unadjusted comparisons of the length of stay for patients seen by a PA and those seen by a FY2 doctor suggest that those seen by a PA have a statistically significantly shorter length of stay in the ED, with a mean difference of 45 minutes (t = –3.916, df 611; p = 0.000).

The difference in expected length of stay between PAs and FY2 doctors was also examined after adjustment of other factors using linear regression, including age, sex, MTS (as a measure of acuity), whether or not they were admitted, whether or not an X-ray was taken and whether or not a prescription was given. Overall, across all sites, those seen by a PA, given everything else being equal, can expect a reduction of 0.61 hours (36 minutes) in length of stay in the ED.

Rate of re-attendance at the same emergency department

Re-attendance within 7 days of the index ED visit was found following 8% (n = 48) of the 610 visits for which these data were available (Table 17). Logistic regression was used to model the likelihood of re-attendance. After adjustment for confounding, no statistically significant difference was found in the rate of re-attendance between PAs and FY2 doctors.

Clinical review of patient records

In the context of the above finding that the study’s primary outcome – the rate of re-attendance within 7 days – was seen not to differ for patient consultations attributed to PAs and FY2 doctors, a review was carried out of a subsample of 40 of the above ED consultation records (20 PA and 20 FY2). Each record was reviewed by four ED clinicians against pro forma questions. In the following sections, we describe the reviews of the consultation record as well as inter-rater reliability.

Appropriateness of the elements of the consultation records

The reviewers’ assessment – either the agreement of the reviewers or the ED consultant’s judgement – is presented in Table 18. The key consultation components, using the criteria used in the Sakr et al.81 study (i.e. patient’s medical history, examination of the patient, request for radiography, treatment plan and decision, advice given and follow-up), are each presented separately.

The results indicate that the reviewers considered the documentation in the consultation record to have been appropriate in the majority of both PA and FY2 doctor cases for each of the key consultation components, and in no component were any errors or omissions seen that resulted in a significant probability that the patient might be harmed.

Nevertheless, in each consultation component a small percentage of consultation records were considered to indicate an error or omission that was unlikely to have resulted in harm or different treatment (defined by Sakr et al.81 as clinically unimportant), or an error or omission that caused a breach in normal guidelines and procedures that would have altered the patient’s treatment (defined as clinically important).81

Reviewer judgement of an error or omission seen that caused a breach in normal guidelines and procedures that would have altered the patient’s treatment was seen eight times in three patient records. Two were records of patients attended by FY2 doctors and one was a record of a patient attended by a PA. For one of the FY2 doctors’ patient records, all reviewers agreed that they had been reviewed by a senior doctor. The reviewers disagreed about whether or not a senior doctor had reviewed the two other records.

An error or omission that was unlikely to have resulted in harm or different treatment was seen 45 times in 18 consultation records. These judgments of error or omission were clustered within consultation records, as shown in Table 19. The extent to which these records accurately provide full information on each of the components reviewed is unknown but audits of routine records in EDs have found omissions.136,137

Review by a senior doctor

The reviewers considered that in just over half of the consultation records, either the treatment plan and decision had not been reviewed by a senior doctor or it was unclear if this was the case (Table 20).

Reviewers’ judgement of who conducted the consultation

The blinding of records to remove any indication of whether the consultation was with a PA or a FY2 doctor was considered to be successful. Reviewers were unable to judge whether the consultation record was that of a PA or FY2 doctor (Table 21).

Overall, 68% (13/19) of the consultations that were actually with a PA were thought, on the basis of the records, to have been with a FY2 doctor. Sixty per cent (9/15) of the consultations that the reviewers thought were by a PA were actually by a FY2 doctor. The raters were 40% sensitive and 46% specific.

Inter-rater reliability

The assessment of inter-rater reliability using Fleiss’ kappa138 for the review of the key consultation components is presented in Table 22.

Taken against the consideration that a kappa score of 0.00–0.20 is ‘poor’, 0.21–0.40 is ‘fair’, 0.41–0.60 is ‘moderate’, 0.61–0.80 is ‘good’ and 0.81–1.00 is ‘very good’139 the inter-rater reliability of the reviewers here was poor when assessing many of the components of the consultation record, improving to fair on request for radiography, the appropriateness of follow-up and whether or not the treatment plan and decision had been reviewed by a senior doctor.

Economic analysis

The economic evaluation set out to compare the costs of using PAs and FY2 doctors in the ED, from the NHS perspective. The main variables of interest, to be extracted from patient records, were also to be analysed as part of the clinical study [these were consultation lengths in minutes, diagnostic tests (blood tests and X-rays) ordered, prescriptions provided, referrals and follow-on care recommended, whether or not the PA or FY2 doctor sought advice from a senior colleague during the consultation, unplanned re-attendance within 7 days (primary outcome) and adequacy/appropriateness of care (obtained through expert review)]. The a priori analysis plan called for calculation of cost differences when statistically significant differences were found in variables, after adjustment for case complexity and consideration of the relative costs to hospitals of employing PAs and junior doctors.

When data collection began, it was found that it was not possible to obtain information on some of the identified indicators. Although time between the first clinical contact and when the patient left the ED was recorded, the patient-facing time of the PA or the FY2 doctor could not be distinguished. Information on blood tests, referrals and follow-on recommendations (other than admissions for inpatient care) and whether or not the PA or FY2 doctor sought advice from a senior colleague during the consultation was not available from all sites.

Of the data that were available, and as reported previously, there were no statistically significant differences between PAs and FY2 doctors with respect to the primary outcome, re-attendance within 7 days, after adjustment for demographic features of patients seen by PAs and FY2 doctors, site and Manchester Triage Score87 (the best available measure of acuity). The patients seen by PAs were observed to receive a significantly higher number of X-rays than those seen by FY2 doctors, after adjustment, but adjustment did not include actual conditions. X-rays could be ordered by any member of the ED team, for example the triage nurse in advance of the patient seeing the PA, and, in any case, PAs are not permitted to independently refer for radiography and have to get requests signed off by authorised colleagues.

Conclusions

Available data for comparing PAs and FY2 doctors were limited. From the information that could be gathered, the evidence suggests that both PAs and FY2 doctors practice equally safely and appropriately in the ED, with no differences in re-attendance rates. Moreover, problems exist in attributing outcomes or processes to individual professional staff because ED care is delivered through consultant-led teams, and both PAs and FY2 doctors routinely take assessments and treatment plans to consultants for checking.

The costs of hiring PAs and FY2 doctors differ. Most PAs are paid at NHS Agenda for Change band 7, although some more-experienced PAs may be at band 8a, and those recently qualified may be hired at band 6.140 In accordance with validated national unit costs, these hourly rates, inclusive of oncosts and overheads, but excluding qualifications, range from £45 (band 6) to £62 (band 8a).89 The equivalent hourly rate of FY2 doctors is £30.89 Note that this figure should be treated with a degree of caution as there is a discrepancy between the mean basic salary for FY2 doctors141 used in the unit cost calculation89 and the national contract basic salary for FY2 doctors.142 With PAs appearing as a more expensive resource, a strictly financial perspective might suggest that FY2 doctors are better value for money. There are, however, other considerations. Junior doctors have training requirements that mean that they are not always available for direct patient care. There is a shortage of junior doctors, and they are insufficient in number to be the sole source of meso-level care in EDs. Where overall staffing ED shortfalls arise, and hospitals bring in locum FY2 doctors, the real costs are higher. Use of PAs can help to relieve staffing pressures in EDs and improve efficiency in the delivery of care. They are able to safely treat patients with a range of conditions, enabling FY2 doctors, and more-senior colleagues, to cover more complex cases, and for FY2 doctors to thereby gain experience that is consistent with the objectives of the junior doctor training programme. Moreover, the role of individual FY2 doctors in the ED is transient, whereas PAs remain long term and provide continuity.

The final chapter provides a discussion of the findings from across the study.