While apprenticeships have existed in the UK since medieval times, they have been in decline since a peak in the 1960s.
In 1993, a new modern apprenticeship scheme was announced in which apprentices would be considered employees, paid a wage and required to work towards a UK National Vocational Qualification (NVQ) level 3 (an A level or American advanced placement equivalent) qualification. Higher apprenticeships were introduced in 2010, which were equivalent to foundation degrees and above. Graduate level/masters (MSc) level 7 advanced clinical practitioner (ACP) apprenticeships were introduced in 2018.
Academic qualification is a key aspect of the learning process throughout an apprenticeship (Figure 1).
Apprenticeship assessment
The MSc is assessed through the successful achievement of 180 level 7 academic credits, 20 of which are for the assessment of the apprenticeship, which is conducted through the compulsory end point assessment (EPA). The purpose of the EPA is to assess whether the learner is occupationally competent to perform the role that they have been trained in. After graduation, every graduate will receive a MSc certification from their university, as well as an apprenticeship certificate, the classification of which is based solely on the EPA result. Every apprenticeship, regardless of the academic qualification, has an EPA.
The EPA must be an independent assessment carried out by an end point assessment organisation (EPAO) within 3 months of those participating completing the on-programme learning. The learner must also meet the gateway criteria to start the EPA.
Recently, it has been agreed that the qualifications that are professionally regulated by the Nursing and Midwifery Council (NMC)—such as the nursing associate, pre-registration nursing and district nursing—can have their EPA linked to programme-board processes. Essentially, this means that such qualifications do not require any additional assessment (the EPA), as the EPA criteria can be shown to have been met by the programmes' assessment. As the ACP role is not yet regulated in the UK, it still has a distinct EPA.
The EPA must be assessed by an independent assessor—a suitably experienced, independent person who has not been involved with the on-programme learning. Any potential assessors that have been involved in the teaching on the ACP MSc cannot assess the EPA. Many programmes have an independent assessor who is involved in the delivery of advanced practice programmes at another higher education institution.
End point assessment requirements
The Institute for Apprenticeships and Technical Education set out the requirements of an end point assessment (Institute for Apprenticeship, 2018a). It must include two distinct assessment methods, which can take a wide range of forms (e.g. written work, presentations, interviews, etc).
At least one of the methods must be synoptic (assessing the learner on all the knowledge, skills and behaviours (KSBs) that were developed during the apprenticeship, as set out in the ACP apprenticeship standard).
The KSBs for the ACP standard (Institute for Apprenticeships, 2018a) mirror the four pillars of advanced practice used in Health Education England's Multi Professional Framework for advancing practice (Health Education England (HEE), 2017): clinical practice, leadership and management, education and research. Although the ACP apprenticeship KSBs are similar to the capabilities discussed in HEE's framework, they are not identical.
Synoptic assessments
The UK Quality Assurance Agency for Higher Education (Quality Assurance Agency for Higher Education (QAA), 2006) define a synoptic assessment as one that ‘encourages students to combine elements of their learning from different parts of a programme and to show their accumulated knowledge and understanding of a topic or subject area’.
These types of assessment are not widely used in higher education in the UK but have been used in the US since the mid-1980s, where they are often called ‘capstone modules’ or ‘projects’ (Southall and Watson, 2016). These modules are a way of assessing a students' grasp of the whole syllabus. Synoptic assessments are currently a requirement for all academic and vocational courses in England.
Challenges for education institutions
Higher education institutions (HEIs), as the end point assessment organisations, are responsible for developing and administering EPA materials in accordance with the EPA assessment plan (Institute for Apprenticeships, 2018b).
Early adopters of the ACP apprenticeship determined that a collaborative approach would be beneficial in producing the materials required, as it would help ensure that there was a consistent implementation across England.
It is also a stipulation in the EPA assessment plan that EPAOs need to participate with an EPAO network, to share and discuss areas of improvement and to report on best practice. To this end, the Association of Advanced Practice Educators in the UK (AAPEUK) hosted a network for HEIs delivering the ACP EPA, which has developed into a community of practice. Communities of practice are defined as ‘groups of people who share a common concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis’ (Wenger et al, 2002).
The AAPEUK network has developed a comprehensive range of resources (Box 1) to support HEIs in delivering the ACP EPA, as well as provide support for educators, independent assessors and practitioners from all professions involved in the delivery of the ACP EPA. These resources are highly valued, as the apprenticeship route and the EPA usually come under scrutiny by several external parties, including the:
- Institute for Apprenticeships and Technical Education
- Office for Standards in Education, Children's Services and Skills
- Designated Quality Body, which was appointed by the Office for Students.
Box 1.Resources Available on the Association of Advanced Practice Educators UK Members-Only WebsiteEnd point assessment (EPA) plan addendumEPA question bankRubrics for each of 3 elementsEPA sample questionsExample case studies
The ACP EPA consists of two parts and three elements, all of which must be passed to pass the EPA:
- Part one: an open-book exam, with eight questions to be answered using three pre-prepared case studies
- Part two: a practice change report and a presentation.
Challenges for educators
The biggest challenge in translating the EPA assessment plan into materials for administering the EPA came with the open book exam. The EPAO must develop a question bank of 24 questions to be used in unique combinations for every exam for a maximum period of 3 years. The EPAO network worked collaboratively to produce this question bank for all members to use.
During its development, it became clear that the original sample questions provided in the EPA assessment plan document were too complex to be answered in the time allowed. The wording of the questions was, as expected of a level 7 assessment, reflective of the higher levels of Bloom's revised taxonomy (Anderson et al, 2001).
Bloom's taxonomy is a framework for categorising educational goals into a hierarchical order of six cognitive levels, which range from lower-order thinking skills, such as remembering and understanding, to higher-order thinking skills, such as analysing, evaluating and creating. However, with approximately 12 minutes for each of 8 questions, this level of assessment was deemed unrealistic.
Therefore, the network drew from the lower levels of the taxonomy to make the task more achievable, while ensuring that it still met the ultimate aim of linking evidence from the pre-prepared case studies to the standard KSBs to demonstrate achievement. Instead of being asked to ‘critically explore’ or ‘critically evaluate’, questions require students to ‘describe’ or ‘explain’ how their actions in the case studies showed evidence of competence for the specific KSB referred to.
This reflects the appropriate use of the taxonomy to describe what the learner is expected to achieve in this specific task, rather than assuming that certain levels of study should restrict themselves to certain levels of the taxonomy.
This is not to say that the attributes in the higher levels of the taxonomy are not a feature of this assessment overall. The synoptic nature of the presentation of practice (part two) requires learners to apply the skills of critical appraisal, critical analysis and synthesis, which have been developed over the course of their MSc degree. In this case, the wording of the KSBs was key to ensure that these graduate skills were required (Institute for Apprenticeships, 2018a).
The EPA assessment plan (Institute for Apprenticeships, 2018b) specifies that the marking criteria provided in the plan must be used in place of local HEI criteria for the MSc programme. Criterion-referenced assessments using marking criteria or rubrics (scoring guides) are the most common way of assessing students in higher education.
They are designed to measure what, and how well, students have learned, as well as help staff make judgements about students' performance in a consistent and reliable way. A judgement is made as to what extent a student has met a criterion (usually based on the learning outcomes for the course of study) and how well.
The EPA marking criteria were unusual, in that they addressed the achievement of the KSBs in a binary way. They simply assess whether the evidence in the case study meets the criteria or does not—there is no judgement of how well a KSB has been met. To pass each part, the learner must have met all the KSBs in bold, as set out in the EPA assessment plan (Institute for Apprenticeships, 2018b).
A higher classification is given depending on how many of the non-bold KSBs have been met, as this tends to establish that the learner has provided a broader range of evidence or gone beyond the minimum standard expected.
This style lends itself to a rubric-style checklist, as assessors can indicate whether the KSB has been met, not met, or (for the purposes of feedback) partially met. It is therefore necessary to ensure that all those involved in the assessment of learners are introduced to this rubric, even if they are experienced educators. Generating a numerical grade from the rubric has been a challenge for some HEIs, as pass/fail grades are not always permitted by local HEI regulations. Members have been able to draw on other member of the EPAO network's experiences and expertise to resolve such issues.
Many ACP MSc programmes have an assessed portfolio that require learners to maintain evidence of their learning and reflection, as well as demonstrate that they have met their professional requirements.
In programmes with an assessed portfolio, educators have commented that the EPA appears to repeat this assessment.
This is not necessarily the case. As a synoptic assessment, the EPA brings together the learning achieved across the four pillars of advanced practice, and therefore could be considered as a fitting final piece (or capstone) for a trainee professional's advanced practice portfolio.
These issues and others identified by the network may inform a review of the EPA assessment, but also the on-programme assessment set by HEIs. The EPAO network hosted by AAPEUK, with its ever-increasing representation, knowledge and experience, is in an ideal position to contribute to a review of the ACP standard.
The end point assessment
The open-book exam
In the first part of the EPA, the exam is ‘open book’—learners are able to bring in three case studies from their own practice that they have written in advance, and which have been overseen and verified by a practice supervisor.
These case studies must be used to answer eight questions, based on the KSBs from the clinical practice pillar of the ACP standard, over a duration of 2 hours.
The case study itself does not contribute to the assessment of the open-book exam. They are made available to the assessors and must be referred to in the exam answers and demonstrate safe practice. As such, what makes a good case study relates to its utility for the learner for the purpose of the exam. The case study must show occupational competence and are best drawn from practice in the final year of the programme. It is not necessary to show development, (e.g. to compare and contrast performance over time). In the exam, a learner will have approximately 12 minutes to write their response to a question; therefore, the less time a learner takes to identify which part(s) of a case study or studies will best meet the KSB, the better.
Each criterion in the ACP apprenticeship standard has a knowledge and skill statement (Institute for Apprenticeship, 2018a). The AAPEUK rubric for the open-book exam sets out the clinical practice criteria separately as ‘understands’ for the knowledge and as ‘able to’ for the skills with those that must be met as a minimum in bold.
The behaviour criteria apply to all parts of the EPA and are set out in the rubrics, with those required to pass written in bold. The questions are designed to give the learner the opportunity to provide evidence against all the KSBs that are bold in the rubric. The EPAO network provide three examples of exam questions for educators and students to use in preparing for the exam to replace those provided in the EPA assessment plan, as previously discussed. Learners should consider how they might also demonstrate achievement of the bold behaviours to pass, and the non-bold KSBs and behaviours in order to gain a better grade.
Figure 2 shows an excerpt of an example case study developed by the EPAO network that can be shared with learners undertaking the EPA:
- Column one contains the headings that will be familiar to practitioners as the components of a health history
- Column two sets out the anonymised details as per a written record of the case
- Column three explains why that KSB is met, and serves as an aide memoire to the learner in the time limited environment of the exam
- Column four identifies the KSBs that are being claimed.
Many KSBs have several aspects; the mapping is most effective when the specific part of the KSB being claimed is identified.
Column three might also include quotations, such as definitions of terms (e.g. autonomy, or person-centred care) around which an answer could be framed, or details of policies or guidance that might be referred to in an answer. The format in this example is not exactly as that set out in the assessment plan, but all elements are included, so it is therefore acceptable. The word count relates only to column two and three, and while the case study needs to be comprehensible, it can be in the form of notes to make the best use of the word count. All bold KSBs need to be mapped at least once across the three case studies; a matrix might be a useful preparatory exercise.
The presentation of practice
The second part of the EPA is a presentation of practice which has two elements:
- Element one is a clinical practice change report, in which the learner outlines the evidence-based background for a proposed clinical practice change relevant to their ACP role and area of clinical practice. It should be noted that the learner is not required to carry out the practice change for the assessment, as this is a proposal. This element particularly addresses KSBs in the advanced clinical practice and research pillar of the standard
- Element two is a presentation, which draws upon the report and discusses the implementation of the proposed practice change. This element particularly addresses KSBs from the clinical leadership and education pillars in the ACP standard. As with the open-book exam, the learner must provide evidence against the KSBs attached to these elements in bold as a minimum to pass.
Despite having separate rubrics to show which KSBs relate to which element, these two elements are marked as one piece of work.
The role of the supervisor
The case studies for the open book exam must be verified by an approved employer signatory, who confirms that the case study is an authentic example of the apprentice's practice and has provided an hour of direct observation during the period of the case study. Learners are often encouraged to produce case studies throughout the apprenticeship as a formative opportunity and to provide a basis for discussion during supervision sessions.
A learner's supervisor or line manager may be able to assist with identifying an appropriate topic for a practice change that contributes to an area already highlighted for improvement. Alternatively, it may be that the learner's final project or dissertation contained a recommendation for practice that can then be used for the proposal. This has the benefit of being an area of interest for the learner about which they already have extensive knowledge and shows them as a researcher implementing their findings (learners should be careful to avoid breaching any HEI regulations about self-plagiarism or duplication). The more practical the change, the easier it may be to generate the proposal.
Box 2 shows real examples from learner's work (shared with permission), many of which have been taken forward in practice.
Box 2.Example Practice Change Topics (Shared with Permission)Introduce a psychological assessment tool to help identify women who require support for psychological distress after pregnancy loss soonerA standard operating procedure for the use of adrenaline auto-injectors to treat anaphylactic reaction in oral challenges and in the nurse-led clinicPost operative physiotherapy protocol for patients undergoing arthroscopic surgery for femoralacetabular impingement syndrome
The learner should draw on learning from previous modules to use theory and research to support their writing (for example, change management, leadership, and education theory), as well as the skills of critical appraisal, analysis and synthesis developed during the programme.
The role of the academic team
While members of the academic team cannot be involved in the assessment of learners on the programme, they are permitted to support them with the production of the materials for the EPA. That being said, there must be no new learning (for example, about theoretical modules or concepts). The use of the example case studies and exam questions can enable formative opportunities. The formative submission of drafts of case studies, the change report and a formative presentation are often offered, in the same way that draft work is often used, to provide formative feedback in other modules.
If a learner does not pass one or more elements on their first attempt, they are entitled to an opportunity to resubmit. The resubmission must be made within 3 months of the confirmation of the failed assessment. The learner is only required to resubmit any elements of the EPA that they did not pass. In the case of the open-book exam, the network has agreed that the resit should use a new version of the exam paper.
Conclusion
Learners tend to be daunted by the prospect of the EPA, but as long as they are in an appropriate patient-facing trainee ACP role, they should have no difficulty in producing the materials to show their occupational competence.
The apprenticeship route for ACP development is still relatively new and there is an ongoing need for information and support for all those involved in teaching, supporting and developing trainees. The AAPEUK EPAO network has become a valuable source of information, consensus and support for all involved in the delivery of this apprenticeship standard.
The collaboration of a team of educators from a diverse range of universities in a competitive environment have worked together and overcome the challenges to successfully implement this new assessment.
The apprenticeship system, while having idiosyncrasies for a MSc level programme accessed by senior professional staff, has contributed to the governance of advanced practice workforce strategy. Increasingly, learners on all funding routes are expected to be in a trainee role for a clearly identified and justified role that has been mapped against the four pillars of advanced practice.
As a synoptic assessment, the EPA is beginning to be recognised as having value for both the trainee and employer as a capstone that marks the successful completion of the programme.
By drawing on their own practice, it helps learners and employers appreciate and gain confidence in the learner's abilities, during what can be a challenging transition to advanced practice—one which is dogged by imposter syndrome for many.
Despite some flaws, the ACP EPA stands as a good example of a synoptic assessment of a MSc level vocational qualification. As part of the apprenticeship programme, it is helping to align professional development with workforce need and contribute to improved workforce transformation.
KEY POINTS
- The end point assessment (EPA) for the advanced clinical practitioner (ACP) apprenticeship is an integrated synoptic assessment of occupational competence
- Implementing the EPA assessment plan held challenges for educators. These were addressed by the end point assessment organisation (EPAO) network, which is hosted by the Association of Advanced Practice Educators UK (AAPEUK)
- Since the first EPAs in 2020, more is known about what preparation is required prior to the EPA and how educators and supervisors can support this
- The ACP EPA is a constructive example of a master's level synoptic assessment, which allows both learners and employers to appreciate and have confidence in their abilities during the transition to advanced practice roles.
CPD / Reflective Questions
- Why is the end point assessment (EPA) important in the education of the advanced clinical practitioner?
- What challenges did its initial implementation hold for educators?
- How do you think the EPA could be developed further?