2021 Curriculum Assessment consultation

Assessments consultation

As part of the 2021 curriculum change we have updated supervised learning events and developed some new assessment types to support the development of anaesthetists in training.

Assessments consultation - Initial findings

Thank you to everyone who had the opportunity to respond to the consultation.  The information we have is very useful and will help us finalise the work around the updated and new assessments including creating educational and support material for trainers and anaesthetists in training, and to provide evidence to the GMC for ultimate approval of the curriculum.

Before undertaking a comprehensive analysis of the data in the new year, we wanted to report on the high-level findings.

Supervised Learning Events (SLE) and Supervision Levels

There was general agreement that the use of supervision levels would be an effective way of guiding and monitoring progress (85.7%) and that this wouldn’t be burdensome for assessors (82.1%).  However, a reasonable proportion of respondents felt that the College will need to provide additional support (35.7%) and that there are potential consequences, positive or negative, inherent in the use of supervision levels (53.6%).

The supervision level is intended to record that which the anaesthetist in training requires for that activity at the time the SLE is completed, rather than to record the actual location of the supervisor during that case.  It is an evaluation of the anaesthetist in training’s ability and the supervisor’s confidence in that ability and is intended to promote developmental conversations and therefore learning.

Holistic Assessment of Learning Outcomes (HALO)

There was general agreement that the use of HALOS would be an effective way of demonstrating completion of learning outcomes (75.0%) but there was concern of the burden on assessors, with general disagreement that HALOs would not be unnecessarily burdensome (54.2%).

Since the survey was launched, steps have already been taken to simplify this process.  In the clinical domains of learning, the key capabilities will be ‘clustered’ around common blocks of evidence so that not every key capability will need to be signed off individually.  The development of faculties of assessment will be key in helping assessors manage the HALO process and information about this will be provided as we roll out a ‘train the trainer’ programme next year.  We are also developing a guidance ‘handbook’ and a range of support materials will be developed and hosted on the College website.

Multiple Trainer Reports (MTR)

There was general agreement that the use MTRs would be an effective way of communicating an anaesthetists in training’s progress (76.0%) but there was concern of the burden on assessors, particularly College Tutors, with general disagreement that HALOs would not be unnecessarily burdensome (54.2%).

Initial research had indicated that the College Tutor is currently the person coordinating consultant feedback in most areas and we will ensure that this is covered as part of the support and education programme and materials.  MTRs are expected to provide feedback across the curriculum and can be linked to multiple HALOs.  They will only be required once per year with some exceptional additions to cover specific areas of training, such as the Initial Assessment of Competence. The MTR will be set up on the LLp to ensure that the process is straightforward to initiate and complete.

It is also worth emphasising that the Multi Source Feedback will continue to be a feature in the new curriculum, much as it does now.  The MTR differs from an MSF as it concerns an anaesthetist’s training progress with key capabilities and learning outcomes whereas the MSF seeks feedback from the multidisciplinary team, including consultants, on overall professional behaviour and attitude.

The consultation is now closed, thank you to everyone who had the opportunity to respond.  We will use this information to help structure our educational and support packages for the new curriculum, as well as to provide evidence to the GMC for approving the curriculum.

Please find the original consultation information below.

Supervised Learning Events (SLEs)

The SLEs for the 2021 curriculum are the same A-CEX, ALMAT, CBD, and DOPS and that we are familiar with in the current curriculum but the documentation within them changes.

They continue to be formative assessments, helping the trainee to learn and progress through reflective discussion.

The first section of the forms, which the trainee competes, is a summary of the educative discussion between trainee and trainer.

There is then an opportunity for the trainer to evaluate the level of supervision the trainee requires for the case, procedure or list at that time. If assessing the level of supervision is not appropriate then this does not need to be completed and the trainer can tick N/A.

The trainer is then asked to give guidance for future development. This should include a discussion as to what the trainee needs to do to progress to the next level of supervision.

SLE consultation documents

Example ACEX

Anaesthesia Clinical Evaluation Exercise

Example ALMAT

Anaesthetic List Management Assessment Tool

Example CBD

Case Based Discussion

Example DOPS

Direct Observation of Procedural Skills

A new SLE has been developed for the 2021 curriculum - the Anaesthesia Quality Improvement Project Assessment Tool (A-QIPAT)

Holistic Assessment of Learning Outcomes (HALOs)

The HALO form serves a similar function to the present CUT form. They are the way by which the trainee is assessed as to whether they have achieved the capabilities and learning outcome for each domain at each stage of training. Trainees will need to link evidence to capabilities. In some cases a single capability will need evidence, in others capabilities will be grouped together with evidence needed for that group.

A single piece of evidence such as a WBA can be used to illustrate more than one capability.

As evidence accumulates there will be indication through colour change on the LLp in a similar way to the present CUT forms.

Once the evidence demonstrates that the capabilities have been achieved then the HALO can be signed off.

It will be possible for clinical supervisors to sign off discrete capabilities within the HALO.

Multiple Trainer Report (MTR)

This replaces consultant feedback and can be completed by a range of trainers.

There will be a set range of minimum standards relating to the number of trainers completing it and the timing of when it is done but the College and ES can ask for as many as they deem appropriate.

The purpose of the MTR is to say whether the trainee is progressing satisfactorily or not and to provide feedback on areas of good practice or areas that need development.

Performance in all the domains of training can be assessed.  Once hosted on the LLP, each of the areas for feedback will be linked to curriculum requirements for that domain of learning at that stage.