Developing, approving and maintaining qualifications: selected international approaches

There are lessons for Australia in the key approaches to the development, approval, maintenance and quality assurance of qualifications adopted in countries overseas. This research takes into account a range of approaches used in selected European Union (EU) member states (Germany, Finland and Sweden), the United Kingdom (England, Northern Ireland and Wales, Scotland, and Ireland) and the nations of New Zealand, Singapore and South Korea. The processes used in Ontario, Canada, and selected accreditation agencies in the United States were also investigated.

This work serves to inform the vocational education and training (VET) sector in Australia about practices used overseas. It provides a useful reference document for agencies charged with developing and reviewing qualifications and showcases the different approaches used to ensure that qualifications remain current for the industries they serve.

Key messages

The report highlights some important issues for Australia, including:

• The introduction of qualifications frameworks, implementation of competency-based or learning outcomes approaches to learning and assessment, recognition of prior learning, and effective regulation and quality assurance processes are all being debated overseas, with varying solutions to the perceived issues being applied.
• The development of hierarchical national qualifications frameworks comprising progressively higher qualification levels is relatively widespread, and increasing. In the main, countries start with existing systems and review these to adapt to new concepts and practices. The use of credit accumulation or credit point systems based on the number of hours typically required for qualification completion is also prevalent.
• The referencing of national qualifications to regional framework models, especially in the European Union but increasingly discussed in our own region, is favoured for improving the transparency, portability, comparability and mutual recognition of qualifications. The main aims are to ensure that qualifications coming from overseas are of the same quality as those attained in the home country and to facilitate labour and student mobility.
• Collaboration between governments (or their delegated agencies) and industry stakeholders is key to developing and/or approving competency standards, educational standards and content that align with labour market needs. Stakeholders almost always involve representatives from industry; in some systems representation is also sought from education and training practitioners and experts, academics, professionals and community groups.
• Removing or retiring qualifications is an issue for systems where there has been a proliferation of qualifications. New Zealand and the United Kingdom have implemented systematic review processes which target for removal those qualifications that have experienced zero or very low uptake over a specified period of time (usually two years).
• Regulatory frameworks reflect their cultural and economic environments. The focus is increasingly moving away from top-down regulation (except for serious transgressions) to a system of collaboration between regulator and provider. Regulators provide advice on what is required to meet specific standards, and providers make internal arrangements to implement and monitor their progress against these standards. Self-appraisal is combined with regular or predetermined external evaluations by the appropriate government agency.
• Countries are keen to ensure that qualifications and skills gained are valued in the labour market by employers and students. This is done by aligning national qualifications and training needs with comprehensive labour market analyses, and applying outcomes-based quality assurance and/or inspection frameworks (including for equity groups). Rates of participation, qualification completion, employment, unemployment, movement into higher qualifications and progression through employment are some key indicators.
• There is a concern about the quality of teachers and teaching, in particular in the European Union states and the United Kingdom, with some countries increasing the level of qualification required for teaching in a VET institution or program.

This paper investigates some approaches to the development, approval, maintenance and quality assurance of qualifications of selected international comparators, including the European Union (EU) member nations of Germany, Finland, Sweden, England, Northern Ireland and Wales, Scotland and Ireland, and the nations of New Zealand, Singapore and South Korea. The processes used in Ontario, Canada, and selected accreditation agencies in the United States have also been investigated.

Qualifications and frameworks

Across jurisdictions, the design of basic and advanced vocational education and training (VET) qualifications is focused on the skills and knowledge required for the current and future world of work. This comprises developing specific occupational knowledge, skills and competences, as well as more generic skills that can be transferred to different contexts, such as, increasingly, communication, technological, team-working and problem-solving skills. The portability of qualifications across national and international jurisdictions to facilitate the mobility of workers and students is especially important in Europe. Seamless vertical and horizontal pathways between qualifications (known as permeability of qualifications in Europe) are supported by national qualifications frameworks (NQFs) in European member states and the referencing of these frameworks to the European Qualifications Framework (EQF).

The learning outcomes approach is currently being adopted in the VET systems of many European countries. Like competency-based training the learning outcomes approaches is focused on demonstrated performance (outcomes) to denote the acquisition of skills and knowledge primarily related to the world of work.

In some countries, such as England, Scotland, Wales and Northern Ireland, VET qualifications are based on national occupational or industry competency standards. In others, for example, Germany, the training regulations and ordinances spell out what is required for qualifications in the dual system of apprenticeships. Occupational standards are based on functional analyses of jobs undertaken in certain occupations or workplace roles. Among these the major functions are divided into units of competency, made up of a set of elements, which describe the accomplishment of certain activities. Performance criteria identify benchmarks for assessment. These units of competency can be grouped together to create a broad qualification, as in the case of Germany, or stand-alone qualifications, such as the national vocational qualifications (NVQs) in the United Kingdom. In other countries, education or program standards are used to identify the content of qualifications. These standards are commonly developed through close collaboration between governments, the relevant industry stakeholders, professional experts and training providers.

To ensure that the qualifications developed continue to be relevant to the needs of industry and society, and to identify emerging industries or occupations, governments and industry bodies also usually undertake comprehensive labour market analyses. In some countries the results of these analyses must be attached to any submissions for the development or delivery of new qualifications.

Another approach to ensuring qualification relevance is to reduce the number of available qualifications. It has not been easy to identify the specific processes used to reduce the number of qualifications sustained by specific countries, although there is reference to this as a task entrusted to the committees that develop qualifications. We have however been able to identify the specific approaches used by the Scottish Qualifications Authority (SQA), the New Zealand Qualifications Authority (NZQA) and the systems used by the Skills Funding Agency in the United Kingdom. The approach is based on identifying for removal qualifications that, for two years, have had zero or very low uptake. The United Kingdom Vocational Qualifications Reform Programme has also used this approach to identify qualifications for removal, but it has combined this with agreement from the relevant sector skills councils (SSCs). Selecting zero- or low-uptake qualifications for removal may not always be the right solution, especially for those occupations which historically have relied on only small numbers of qualified individuals in their industry (for example, funeral directors, mortuary theatre practitioners, grave diggers). This is why consulting with industry stakeholders about the continued need for particular qualifications is important.

There is continuing debate about the quality of qualifications based on the principle of unitisation. In the main this debate is concerned with the undermining of the broad qualifications traditionally perceived to be required for the holistic development of tradespersons. In Germany, the use of the learning outcomes approach is especially criticised in relation to entry-level dual system qualifications. Although learning outcomes have been generally accepted for prevocational training and continuing education, they have been criticised for subverting the concept of ‘beruf’ or vocation in the German dual system. There is a fear in some places that unitisation resulting from the learning outcomes approach will open the way to partial qualifications. This fear of partial qualifications seems not to be important in the United Kingdom, where units can be accumulated over time.

Stakeholder involvement

Broad stakeholder consultation and involvement in the development of qualifications and qualifications frameworks is observed in many systems, especially those undergoing qualification-related reform and those hoping to open up access to and movement through different education sectors and qualification types. Such consultations and activities are generally driven by government ministries and involve a range of regulatory agencies, industry peak bodies, trade unions (including students), professional associations, public and private VET providers, school and higher education sectors (including practitioners), experts in the field, and research agencies. The sector skills councils in the United Kingdom and the chambers of commerce and chambers of crafts in Germany formalise the participation of industry in the training system. In Finland and Sweden industry and community stakeholders are involved in the development of qualifications in the national and sectoral committees charged with this function. They are also involved in the assessments of skills demonstrations, which generally take place in workplaces.

Quality assurance

The current move to outcomes-oriented learning, based on learning outcomes, competency standards, or learning objectives, places the focus on assessment activities rather than on delivery or learning techniques. When learners obtain qualifications for the knowledge, skills and competencies acquired in a range of formal, non-formal and informal situations, uncertainty may arise about the quality of the qualifications obtained. This uncertainty can reduce trust and confidence in the value of the qualification and its acceptance by employers and the individuals themselves.

In traditional education systems the general approach has involved a system of inspection. In reformed or systems undergoing reform the inspectorial approach has either been superseded by or been combined with a quality assurance approach. The inspectorial approach continues to have currency in many EU member states. The quality assurance approach has generally been adopted in South Africa, New Zealand, Ontario in Canada, and with accrediting agencies in the United States. Concepts of quality assurance have been combined with concepts of inspections in Scotland and other parts of the United Kingdom. In Germany quality assurance has been largely focused on continuing education.

On the whole, quality assurance and regulatory frameworks have specific requirements that need to be addressed by those seeking initial or continuing accreditation. These are generally specified in quality standards, objectives or criteria. Standards-based systems, as well as those based on objectives and criteria, require regulators to have a clear idea about the intentions of the standards and objectives to be applied or pursued; regulators also need to ensure that these are clearly communicated and are understood by those implementing them. In many jurisdictions the issues covered by quality assurance and those relating to accreditation standards converge.

Regulators and free providers are increasingly concerned about the bureaucracy associated with regulation. The use of risk-based assessment to reduce bureaucratic burden and to provide responsive regulation is also being implemented across systems.

Conclusions

Many of the issues being faced overseas in the construction of qualifications frameworks, the development of the qualifications themselves and quality assurance frameworks have already been debated in the early development and recent revisions of first-generation frameworks. Of interest, however, are the practices used in some jurisdictions to prepare providers for accreditation and the use of risk-based approaches to identify and apply regulatory action. There is a need to build on earlier reforms; for example, there has been a range of national initiatives aiming to improve the quality of assessments. Findings from these initiatives should be published more widely and considered when developing guidance and resources for training providers. The following are the major issues for consideration:

• The types of risk-based quality assurance mechanisms that are based on decreasing regulation for high-performing institutions and increasing investigations for those considered to be of higher risk should be more closely investigated. Risk-based approaches involve the identification of triggers for evaluation. Different systems have identified a range of these triggers. The triggers identified by the Florida Education Department, Ofqual (Office of Qualifications and Examinations Regulation) in the United Kingdom, and the New Zealand Qualifications Authority are useful approaches.
• Involving stakeholders in the design and assessment of qualifications is a feature of many systems at the national level, where it is sometimes legislated. However, it is important to note that stakeholder involvement may be constrained by the ability and availability of stakeholders to meaningfully participate in the process. Identifying the type and extent of involvement that can be reasonably expected from industry, the community or student stakeholders may be an important step to ensuring their valued input into the design of qualifications, especially where these stakeholders do not have a formal legal role in the education and training system. This approach can also help to identify the role of stakeholders in external assessments in improving the validity and reliability of assessments. The practices used for the verification of assessments in the United Kingdom or skills demonstrations in Finland may be useful points for further investigation.
• The integrity of qualifications is not only dependent on the capacity of teachers, trainers, assessors and verifiers to make valid and reliable judgments about the learning that has occurred, but also involves ensuring the availability of up-to-date facilities and equipment for learning and/or assessment. All groups need initial and continuing training in their sector-specific knowledge and skills and in teaching and/or assessment pedagogies. Initial training and continuing professional development of quality auditors and inspectors is also required.
• Qualifications systems and regulatory approaches reflect the traditions of the cultural contexts in which they are located; however, the need to ensure that qualifications have integrity and currency holds true for all nations in this study.

Limitations

This paper has provided information on what selected country comparators do to develop, approve, maintain and quality-assure their qualifications. This has been achieved through a desk-top analysis of readily available information on public websites and publications. Although obtaining information in this way for some areas, for example, qualifications frameworks, qualification approval processes and quality assurance mechanisms, has been relatively straightforward, it has been more difficult to obtain information on how different jurisdictions remove qualifications that are either no longer useful or where few or no individuals enrol. Furthermore, we can never be sure that the descriptions of processes and policies we have sourced from public websites at the time of writing are the most accurate, comprehensive, up to date and complete. When we rely on descriptions of overseas systems and processes that use English terms that are not easy to interpret, there is a danger that the processes may be misunderstood or misrepresented. Despite these limitations, we consider the information we have accessed will provide a useful and relevant comparative context.