Regulating and quality assuring VET: international developments

The opening-up of the market for education and training, including vocational education and training (VET), has increased the importance of regulation and quality assurance mechanisms in ensuring the integrity of qualifications. This report investigates approaches to the regulation and quality assurance of vocational education and training in a number of countries: New Zealand, selected European member states (Finland, Sweden, United Kingdom), Canada (province of Ontario) and two accrediting agencies in the United States. The insights gained from this investigation into the practices applied overseas could be used to inform the development of VET regulatory and quality assurance approaches in Australia.

Key messages

• Increasingly, training systems are implementing principles of responsive regulation and risk analysis to help ensure compliance and to reduce the burden on regulators and regulated populations. However, the implementation of these approaches requires regulators to have access to sufficient and robust data collection mechanisms to help them to identify effective triggers for risk-based reviews.
• The voice of industry (including employer and employee associations) is commonly heard in the development of qualifications, assessment for qualifications, the provision of practical work experience, and validation of assessments.
• Debates about the quality of teaching are gaining momentum, not only in Australia, but also overseas. The aim is to implement mechanisms to improve the quality of teacher preparation and to ensure continuing professional development.
• External assessments conducted by third parties possessing relevant occupational knowledge and expertise can be used to assure the integrity of assessments and qualifications.
• The New Zealand external evaluation review (EER) approach is worthy of attention, especially as it aims to help providers to develop their capacity for self-assessment. However, a lesson from their experience is to make clear decisions about how to promote the approach to providers to ensure that trust between regulators and providers is maintained.
• The preparation of institutional self-reviews or reports helps to embed self-monitoring mechanisms into the routine actitivities of providers. However, such processes, if not well managed, can become so resource-intensive that they may draw valuable resources away from core teaching and learning tasks and so hinder the achievement of real continuous improvement.
• Outcomes-based measures of institutional performance can help individuals to make informed choices about where they want to study, and governments to make policy and funding decisions. Their usefulness is highly dependent on the robustness and accuracy of participation and outcomes data and the mechanisms for data collection.

Dr Craig Fowler
Managing Director, NCVER

The general principles and practices of regulatory and quality assurance systems for vocational education and training (VET) and other education sectors at home and overseas are converging. In this paper we review relevant Australian and international literature relating to the regulatory models and frameworks applied in the training sector but also in other industry sectors with relevance for VET. This is important to ensure that the lessons learnt by others in implementing regulatory standards in similar contexts are recorded and understood.

Overview

The review of the literature has found that, although the details of the regulatory processes followed may differ, many of the issues confronted by and important to VET regulation in Australia are often being experienced in education and training systems overseas.

Common in many systems are provisions for: initial provider registration and accreditation (including provisional registration or equivalent); frameworks for qualifications; and institutional self-reviews (self-study, self-assessment) combined with external reviews (often including desk-top audits, on-site visits and third-party assessments). In some countries site visits include observations of teachers and learners in institutions, as well as visits to employers and observations of learners at workplaces.

The adoption of objective measures on the outcomes of performance for systems, providers and students is also gaining traction and encompasses student learning and competency outcomes, graduate destinations, and employer and student satisfaction with training. In some countries output measures (such as numbers of participating students, teachers and employers, and hours of training delivery) continue to be used to signal quality. Although there is increasingly a shift towards continuous improvement, as opposed to strict compliance with rules regimes, in many systems the need for and use of external accountability combined with internal quality assurance regimes remains high.

Risk-based approaches to reducing the regulatory burden on the regulated and the regulator and to ensuring the efficient use of resources are also being commonly applied. This is especially the case for those systems which have given substantial independence and autonomy to providers. The identification of key risk factors helps regulators to establish audit or review schedules and to focus reviews on specific issues.

Not surprisingly, the key quality standards and/or areas of focus in many quality systems cover similar issues. These include ethical and legal considerations and good business and financial management (including ownership and financial sustainability); competent staff for administrative operations (including management and leadership); appropriately qualified teachers, trainers and assessors; accurate and timely information to students prior to enrolment to enable them to make informed selections; and non-misleading marketing strategies.

There are also definite moves to increase the transparency of information about the expectations and outcomes of service provision. Across systems and sectors, transparency initiatives are promoted as ways of helping governments, systems, providers and clients to make informed decisions. Nevertheless, what are considered to be traditional approaches to ensuring quality continue to operate (for example, defined curriculum and qualifications, external examinations and inspections). Mechanisms to regulate and quality-assure providers eligible for government funding are also used in voluntary systems, as in the United States.

Key observations

It is clear that any quality system and its associated standards need to address the mission and related objectives of an organisation. Standards-based systems, as well as those based on objectives, require regulators to have a clear idea of the intentions of the standards and objectives they want to apply or pursue, and to ensure that these are communicated to those who are to implement them in a clear and easily understood manner. Developing standards or objectives, however, is not just about the language used or the level of prescription required. It is also concerned with identifying what these standards should cover. Arriving at these decisions cannot be done in isolation, and across industry sectors at home and overseas there is substantial evidence to show that broad consultation before implementation is critical.

Increasingly, both educational and regulatory systems are focusing their efforts on improving quality, including the qualifications of the VET workforce. The standards for entry-level teacher training and continuing professional development emphasise the need for teachers and trainers to understand their specific disciplines or trade areas as well as the pedagogy of training and assessment. Suggestions are made for the creation of a ‘master VET practitioner’ or ‘master teacher’ category and for providing qualifications that would enable those with general teaching qualifications to acquire VET teacher qualifications. There are also suggestions for a professional association for Australian VET, which would eventually be responsible for accrediting or registering teachers and trainers; a mentoring system for beginning teachers; and a formal teaching scholarship centre, which would focus on the pedagogy of VET. The qualifications of teachers differ across jurisdictions, with some jurisdictions requiring teachers to have higher education qualifications as well as industry experience.

The quality of assessments is also a key concern. This includes the consistency of assessment judgments across different assessors and for all individuals being assessed. Also important to the debate is the validity of assessments. Assessment burden and cost are identified as key disadvantages for systems that rely heavily on external assessments. A related issue is the undervaluing of underpinning knowledge and understanding in favour of demonstrated practical competency. This is a consideration for those countries that have adopted a competency-based assessment approach. Skills demonstrations by learners are also being used in external assessments.

Engagement with industry stakeholders is considered to play a key role in the development and implementation of quality systems and associated quality standards and/or objectives at both regulator and provider levels. In some countries industry has a formal role, not only in the setting of examinations, but also in the assessment of outcomes. Involving those to be regulated in the development or review of standards is also being practised and/or promoted. As well as helping to build trust, good relationships between the regulated and the regulators can assist in the development of locally relevant standards. It is acknowledged that a system should be vigilant in maintaining standards that reflect changes in social and economic conditions.

Responsive regulation (including light-touch approaches combined with increasingly more punitive measures, if required) is also gaining favour and is being adopted in a number of public agencies, including the Australian Taxation Office and the National VET Regulator (the Australian Skills Quality Authority [ASQA]). Responsive regulation is based on the belief that the natural inclination of the regulated community is in fact to comply rather than not with standards and regulations. A light touch is characterised by increasingly more favourable treatment for compliant behaviour and increasingly more severe sanctions for transgressions. This approach is supported by the findings of a number of empirical studies which have shown that praise has resulted in increased levels of compliant behaviour, while ‘shaming’ resulted in decreased levels. The general practice is to address early non-compliant behaviour by attempting to understand the reasons for the behaviour and providing support designed to return the organisation to a situation of compliance. Clarity of purpose, transparency of expectations, trust between regulators and the communities they regulate, and reduction in both regulatory effort and the costs of compliance are promoted as key features. These features are to be reinforced by clearly defined and consistently applied regulatory sanctions of increasing severity for transgressions.

Adopting a risk-based approach to regulation is also favoured by commentators on regulatory compliance. Such an approach ensures that complying providers are not subject to unnecessary regulatory burden; this approach also assists in concentrating resources where they are most needed. To make best and efficient use of regulatory resources, a regular scan of the regulatory environment is suggested to help to identify and prioritise emerging risks, which can then be addressed. It is also important for regulators to understand the root causes of issues and to address these via a combination of regulatory strategies, including sanctions. In addition, regulators may need to understand how the policy of one sector may impact on behaviour in another. For example, market mechanisms in the VET sector have provided an opportunity for exploitation and abuse (with regard to international education).

Becoming more common is combining self-assessment or review processes with external review by regulators and other third parties. This approach is also promoted as a way of reducing unnecessary burden. In systems which make significant use of a self-review or self-study process (Ontario, New Zealand, United Kingdom, South Africa and United States), the training organisation must undergo self-review to determine whether it considers it has complied with requirements.

No matter what approach is taken in regulatory practice or assessment of training, the need for developing standards and practices that allow the consistent interpretation and implementation of standards remains. This includes having in place clearly articulated standards (or their equivalents), presented in accessible formats, and ensuring that auditors and those to be audited are adequately prepared for their respective roles. In some international systems students become part of the review process. The use of peers (acting in a critical friend capacity) is also being practised. Suggestions are also made to have teaching staff ‘deeply engaged’ in the process by giving them a greater role in the preparation of the documentation required by auditors and especially in relation to addressing key audit criteria.

The move towards improving transparency is gaining traction across a range of industry sectors and education and training systems, including VET. Transparency is especially reliant on the generation and publication (often in online formats) of data on performance. The aim is to improve the provision of information to enable clients and consumers (including students and their parents, workers and employers) to make better choices in the purchase of services, and the action to take if the purchased services prove unsatisfactory. Ensuring transparency of information about the outcomes delivered also enables governments to make suitable funding decisions.

A variety of fees (including annual dues) are commonly applied for different services by regulators and accrediting bodies or agencies. Fees are charged for the lodgement of applications for initial, provisional and renewal of, or continuing, registration and the assessment of supporting documentation. These fees will vary according to the volume of qualifications, courses or units to be accredited or the locations of delivery sites. Charges are levied to cover the costs associated with the on-site visits of evaluators or auditors on external review teams or panels. Fees are levied on applications requesting the change of scope of registration or accreditation.

It is also clear that those systems that are revisiting their own regulatory frameworks need to consider the extent to which these require a major transformation and overhaul, or whether changes can be accommodated within the existing practices, keeping in mind that significant change can take some time to bed down. It is also important to note that implementing regulatory change involves considerable cost for both the system and the community, and that community fatigue with constant change can reduce trust in the abilities of regulators and in the effectiveness of implementation. These issues explain the staged approach to implementation that has been adopted in many sectors. Key to these decisions is good information about patterns and trends in regulatory behaviour to help to identify priority problems and issues. Understanding the root causes of problems (as promoted by many experts and commentators on regulatory frameworks) will be useful in directing the efficient use of resources.

Limitations

In this paper we have provided information on how some selected countries regulate or
quality-assure their VET systems and qualifications. We have conducted a desk-top analysis of the information (including of the VET system in Australia) that is easily accessible through public websites and publications. Although it has been relatively straightforward to obtain some information on overseas developments via these means, we can also never be sure that the processes and policies posted on public websites are comprehensive, up to date or complete. Relying on descriptions of systems and processes from this distance means that our interpretations of what is intended or what actually occurs may well be compromised. Despite these limitations, we believe we have provided a range of potentially useful information.

As well as examining Australian materials that address VET and other education and training systems, we report on developments in those international systems that have implemented substantial regulatory reforms. We also draw from an extensive range of research materials and from other relevant websites. This paper cannot, however, be considered to be exhaustive in its coverage. Nevertheless, it is sufficiently extensive to provide key insights and lessons to inform the discussion on effective and efficient regulatory frameworks for Australian VET.