Cover image for Delivering high-quality VET: what matters to RTOs? report

Delivering high-quality VET: what matters to RTOs?

By Hugh Guthrie, Melinda Waters Research report 22 March 2022 978-1-925717-94-5

Description

This report is the second part of a project investigating the quality of delivery in VET and how it might be better defined and measured. Based on consultation with high performing RTOs, the research finds that definitions of quality depend on an RTO’s purpose, mission, student characteristics and operating context. RTOs use a wide range of information and data to build a comprehensive a picture of delivery quality throughout the student life-cycle or learning journey, although the measures used varies by RTO type and size.

Summary

About the research

A priority for governments is ensuring public confidence in the quality and value of vocational education and training (VET) available to learners throughout their lives. The delivery of high-quality teaching, learning and assessment is an important element of this and is known to directly impact on outcomes for students. However, little research has examined what high-quality training delivery looks like in practice and how it might be measured.

Based on consultations with registered training organisations (RTOs) from the public, private, adult and community education (ACE), and enterprise segments of the sector, this research investigated how the quality of delivery in VET is currently defined and measured. It also set out to identify the barriers to high-quality delivery, as well as approaches that might better encourage and sustain high-quality delivery into the future.

Key messages

  • The definition of high-quality VET delivery differs among RTO types, depending on their purposes, missions and goals, their student types, the courses and qualifications they offer, and the context in which they operate.
  • The key principles underpinning a definition of high-quality delivery in VET, which are common across the RTOs participating in this project, are that it is:
    • transformational: how well students are achieving
    • student-centred: how well students are supported and encouraged to learn
    • fit for purpose: how well stakeholders’ needs and purposes are met
    • evolutionary: how well delivery adapts to changing stakeholder and workplace needs.
  • The size and type of an RTO influences the ability to define and measure the quality of VET provision.
    • Quality appears to be most easily described and measured in enterprise-based RTOs, smaller private RTOs and ACE providers, where the scope of delivery tends to be narrower and there is direct oversight of the teaching and learning environment. The resources and expertise required to collect and analyse data, however, can be limited in smaller RTOs.
    • Larger RTOs tend to have more resources to collect and analyse data, but monitoring quality in organisations supporting a broad spectrum of students with diverse backgrounds and needs, a large suite of courses and qualifications and multiple delivery sites, can be challenging.
  • RTOs use a wide range of information and data to evaluate quality, including a mix of quantitative data, qualitative data, and information gained through informal ways.
  • High-quality delivery depends on many factors, some of which are beyond the control of RTOs. The barriers identified by participating RTOs include a compliance view of quality, funding, the quality of training packages and difficulties in recruiting, developing and retaining teachers and trainers.

Executive summary

This report is the second part of a project investigating how the quality of delivery in vocational education and training (VET) in Australia is currently defined and measured, and how registered training organisations (RTOs) use and value measures of quality. The project also set out to identify the approaches that might be more effective in encouraging and sustaining high-quality VET delivery into the future. This report builds on the first component of the project (Guthrie & Waters 2021) and is focused on the views of RTOs from the public, private, adult and community education (ACE), and enterprise segments of the sector.

An emphasis on the views of RTOs is timely as they are the principal instruments for success for VET students, employers, industries and communities and in implementing the national reforms to the sector currently underway.

We use the term ‘delivery’ throughout this report to encapsulate all of the activity involved in producing high-quality learning experiences and outcomes for students; that is to say, not only the more visible teaching, learning and assessment practices, but also the wraparound administrative and support services that enable students to engage fully in learning, which include course planning, design and development. We use the terms ‘teacher’ and ‘trainer’ to apply to all of those who deliver VET courses and qualifications to students.

What quality means to RTOs

The RTOs with whom we interacted for this project, all of which are recognised for good and high-quality delivery, are strongly focused on improving learning experiences and outcomes for students and meeting their needs, along with those of employers (when employment outcomes are involved). This indicates that Harvey’s (2007) fitness-for-purpose view of quality — which judges quality on how well a product or service meets its stated purpose — is primarily driving their delivery efforts.

We also heard quality described by RTOs in more aspirational terms, such as ‘exceeding expectations’, ‘striving for excellence’ and adding ‘real’ value for stakeholders, particularly students. Comments such as these reflect Harvey’s exceptional (excellence) and transformational views of quality and suggest that both indicators are also motivating RTOs to do a good job. The transformational view in particular conceptualises quality as a process of change that adds a personal growth dimension to VET learning experiences. This leads to two notions of transformative quality: one that enhances learning experiences and outcomes for students and another that empowers those students to succeed in life (Harvey 2007).

The three views of quality (fitness-for-purpose, excellence and transformational) are moderated by the need for consistency (Harvey’s perfection view) and accountability (a value-for-money view). All of Harvey’s views of quality are valid and important to delivery in VET and are exemplified by what our informants told us. The challenge for RTOs is finding the right balance between them to meet their specific purposes and objectives. And this was the first important message we received during our consultations for this project: definitions of quality differ, or at least the emphasis does, across RTO types. The right balance for each depends on their purpose, mission and goals, their student types, the courses and qualifications they offer, and the context in which they operate.

Overall purpose has a strong bearing on how RTOs define quality. We found Euler’s (2013) three purposes of VET (the individual, social and economic) useful in demonstrating that, while all RTOs have a strong individual purpose (focused on students), some have a broader social purpose, whereas others have a stronger economic (business) purpose. The balance between these purposes determines how they define and measure quality.

Size matters too. We found that the quality of delivery is most easily described and measured in enterprise-based RTOs, as well as in smaller private and ACE RTOs, most of which have narrow scopes of delivery and more direct oversight of daily teaching and learning activities and environments. Small size, however, means that fewer resources and expertise are available to implement the comprehensive data-collection and analysis systems used by larger RTOs to attain consistency in quality across multiple courses and delivery sites.

Large size also has its drawbacks, especially when RTOs service a broad spectrum of students from different backgrounds, offer a large suite of courses and qualifications, and have varying purposes and cultures across multiple delivery sites. This makes the job of describing, measuring and monitoring quality especially challenging and means that, for larger multi-purpose public and private RTOs, a one-size-fits-all approach to measuring quality of delivery by using aggregated results is not that useful, especially at course level.

When participating RTOs were asked to define high-quality delivery, their responses revealed common points of view, also confirming our earlier findings (Guthrie & Waters 2021). It was generally accepted that good-quality delivery:

  • has a transformational aspect for students, in that it changes what they know and can do, and how they see themselves, to varying degrees
  • is student-centred,[1] in that it motivates and supports students of all types to learn and achieve their purposes and aims
  • prepares students for work (or other destinations) and for life
  • develops an occupational identity, where applicable
  • is closely tied to industry and the workplace
  • meets the needs and expectations of employers, where applicable
  • results in employment or other desired outcomes.

We conclude from our discussions with RTOs that the key features or principles defining high-quality delivery in VET can be summed up as:

  • transformational: how well students are achieving and developing
  • student-centred: how well students are supported and encouraged to learn
  • fit for purpose: how well stakeholders’ needs and purposes are met
  • evolutionary: how well delivery adapts to changing stakeholder and workplace/industry needs.

The clear message from the participating RTOs is that high-quality delivery involves much more than contact hours between teachers, trainers and students. It depends on many dynamic factors, many of which are beyond their control, as their feedback in this report shows. They unanimously agree that highly capable and passionate teachers and trainers, with currency in their dual teaching and training and industry professions and engaged in continuing professional development (CPD), are absolutely key to high-quality delivery. They also confirmed the importance of RTO leaders and managers driving and supporting a culture of quality improvement and excellence to delivery quality.

How RTOs currently measure quality

Our participating RTOs use a wide range of information and data to gauge and evaluate the quality of delivery. These include:

  • The impact of delivery on business aims, objectives and purpose: according to RTO type and operating environments
  • The effectiveness of a course or qualification: measured by enrolments, the extent to which it met student purpose and employer needs, student achievement, completion rates and graduate outcomes
  • The quality of learning experiences for students; levels of satisfaction, attendance, retention, attrition, engagement, student—teacher ratios and student support. The quality of facilities, equipment and organisational culture is also considered
  • The impact of delivery on learning; student progress and achievement
  • Engagement with employers and industry: their involvement in course design and delivery, satisfaction and confidence in an RTO and its students, employment of graduates and work-placement students, repeat business and recommendations to other employers
  • The capability of teachers, trainers and others supporting delivery: level of qualification, industry currency, participation in CPD, levels of satisfaction and engagement with students, in some cases at a team level but mostly at an individual level.

Our discussions with RTOs revealed that the most important aspects of delivery quality are students’ performance, experience, engagement, satisfaction and progression towards their goals, as well as their personal growth and wellbeing.

Many well-known quality measures are used collectively and over time to evaluate current delivery performance and performance trends. The larger RTOs tend to focus on quantitative data, collected through sophisticated systems (mostly online), while also making good use of qualitative information, collected formally via surveys and other means. Smaller RTOs tend to place more reliance on less-formal qualitative information, to ‘keep a finger on the pulse’ of delivery quality on a daily basis, obtaining the information largely through observation, casual conversations, word of mouth and gauging the ‘the vibe’ of a learning environment.

The value of this wide range of information to the RTOs lies in its immediate relevance to specific students and its ability to highlight emerging issues, flagging the need for appropriate action to address the issues. While longer term and wide-ranging data can show performance trends, anecdotal information, although often not recorded or reported externally, can tell a richer story of students’ experiences and the quality of learning environments — as well as the factors enabling or constraining their progress — than more formal and standardised measures of delivery quality.

Irrespective of how RTOs collect data on delivery quality, all those we interviewed use the information to paint a comprehensive picture of learning environments and how they are impacting on their students, including how well they are progressing and barriers or impediments to learning. How RTOs interpret and use this information to improve delivery is a hallmark of their quality performance (Ofsted 2014). Benchmarking, observation and peer review of delivery practices are valuable approaches to quality improvement for some RTOs but are not used systemically. Expanding this capacity will be critical to improving quality assurance across the sector. Networks and communities of practice are used systematically and are effective in building delivery capability and quality.

Challenges to delivery quality

The RTOs interviewed for this project are finding it difficult to deliver high-quality learning experiences and outcomes for students within the current regulatory, policy and industry environment. The common challenges of concern to them (and crucial for some) are:

  • Funding; specifically formulae, levels, duration of contracts and inconsistencies between jurisdictions
  • compliance-driven regulation, with multiple regulatory bodies; for example, VET regulatory bodies, industry bodies and jurisdictional contract managers
  • the quality of training packages (when seen as too prescriptive and not meeting their purpose), coupled with the significant cost and effort associated with updates to remain ‘compliant’
  • difficulties in recruiting, developing and retaining teachers and trainers, especially attracting industry professionals into teaching roles and professional development for casual and part-time teachers and trainers. Working conditions and initial and ongoing teaching qualifications remain major issues.

Some of these challenges are the focus of reforms currently underway being led by the Commonwealth Department of Education, Skills and Employment (DESE), including the development of a VET Workforce Quality Strategy; the transition to self-assurance by the Australian Skills Quality Authority (ASQA) in response to the Rapid review (mpconsulting 2020); and reviews of VET pricing arrangements and costs of delivery across jurisdictions, being conducted by the National Skills Commission (NSC). These reforms are welcomed by the RTOs we consulted due to the opportunities they will provide for: building effective quality systems, tailored to their specific operations, missions, purposes and delivery circumstances; improving funding arrangements; and relieving what they see as a disproportionate amount of time, effort and resources devoted to compliance. This compliance focus does not accord with their efforts to improve quality through innovation, collaboration and capability-building initiatives.

Ways forward

The participating RTOs offered several suggestions to assist them to provide high-quality delivery into the future. These include, at a system level:

  • Establishing a national VET body to coordinate, promote and support high-quality delivery
  • Developing a genuine approach to quality assurance, one based on trust that proven RTOs will continue doing a good job, while actively supporting others to improve
  • Developing funding models and formulae that support and incentivise efficient but excellent delivery
  • Taking greater heed of the learnings from VET’s significant body of research, conducted by experts and practitioners, to inform policy reforms.

At an RTO level, we found many examples of good and excellent quality practices that could be shared more broadly across the VET sector, especially as RTOs prepare for quality self-assurance. These include:

  • building a culture of quality improvement within RTOs
  • using good data for short-term quality improvements
  • building a longer-term evidence base of high-quality delivery at RTO and system level, particularly for use by smaller RTOs
  • fostering greater collaboration and partnerships between RTOs to facilitate the sharing and benchmarking of good practices and innovation in delivery for specific student groups and learning environments
  • improving workforce recruitment, retention and development strategies.

This report notes that, despite best intentions over many years, substantive improvement and change in VET delivery has been difficult due to the sector’s diversity, its many stakeholders (with divergent expectations) and an inherent inability to change (Guthrie 2021).[2] Many of the RTOs consulted expressed frustration and fatigue that the major issues impacting on delivery quality, especially those beyond their control, have not been adequately addressed, despite a long history of ongoing sectoral change. There is still only so much they can do to improve delivery quality in the current system.font-size: 170%

VET’s stakeholders are entitled to expect high quality in the delivery of VET, but we see real tensions between the sector’s regulatory and bureaucratic requirements and demands on RTOs to be innovative, and flexible and responsive to the needs and objectives of a wide range of student and employer groups. The expectations for high-quality delivery, as some of our informants told us, are almost impossible to meet within current settings. From an RTO perspective, all aspects of VET’s quality system need to work together to enable, encourage and sustain high quality.


[1] In our first paper we used Cedefop’s (2015) description of learner-centred as meaning being responsive to learner interests and needs and slowly increasing their ability to be independent learners. This approach places an emphasis on learner outcomes, communication skills and capability for learning and is less reliant on teacher-led pedagogies.

[2] Guthrie (2021) describes this inertia as an inability to make substantive changes to improve the sector despite good intentions to do so. The problem, he suggests, are piecemeal and short-term approaches to change, which have resulted in battles between jurisdictions and interest groups over whom has responsibility for implementing the changes.

Download

TITLE FORMAT SIZE
Delivering high-quality VET: what matters to RTOs .pdf 1.4 MB Download
Delivering high-quality VET: what matters to RTOs .docx 1.7 MB Download

Related items

For registered training organisations (RTOs), high-quality delivery of vocational education and trai… Show more

This webinar will review the findings of the recently released NCVER report - Delivering high-qualit… Show more

This report is an interim publication for a project that outlines what is known about the quality of… Show more

New research shows the definition of a ‘quality’ VET sector can vary widely by stakeholder group and… Show more