Assessment Strategy for 2021 Anaesthetics Curriculum

Validity is an important consideration in measuring the quality of an examination and demonstrating that each examination component plays a vital role within the integrated framework of the wider programme of assessment.

The Validity theory recommended in Designing and maintaining postgraduate assessment programmes (GMC. (2017). Designing and maintaining postgraduate assessment programmes) is Kane’s Validity Framework (Kane, M.T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, special issue: Validity, 50:1) (KVF). KVF requires awarding bodies to state the purpose of the examination component within the examinations framework and asks for evidence to support the choice, content and format of the examinations. This strategy document sets out a purpose statement for each examination component in the FRCA examinations framework. It provides justification and evidence of how the examinations overall and at component level, ensure the safe management of anaesthetists in training through critical progression points within the anaesthetic training programme, and clearly communicates the levels of performance expected.

As defined in the GMC standards, examinations must consist of “an integrated set of assessments […] which are blueprinted against, and supports, the approved curriculum. It may comprise of different methods”. To this aim, the components that make up the FRCA are not only blueprinted to the curriculum, but have also been selected to assess three of the four levels of Miller’s pyramid (Miller, G. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65); knows, knows how [and why] and shows. The fourth level, ‘Does’, is assessed by workplace-based assessments and relates to behaviour in real life situations.

Figure 3 – examination components against Miller’s’ pyramid

Whilst the written examinations and the SOE examinations are aimed at the same levels of Miller’s pyramid (knows and knows how), the SOEs are designed to test a candidate’s knowledge beyond recall and recognition of facts. The structured interactions allow examiners to explore a number of higher order domains within Bloom’s revised taxonomy (Anderson, L. W. & Krathwohl, D.R., et al. (2001). A taxonomy for learning, teaching and assessing), see figure 2 below, such as understanding, application of knowledge, analysing and evaluating.

Figure 4 – Bloom’s revised taxonomy (BRT)