By John King, Managing Director, NCVER
This piece is based on a presentation given at the Victorian TAFE Association's TAFECreates conference on 7 August 2025.
From Markets to Stewardship
Coming back to the VET sector after some years away, it feels noticeably different.
Through much of the 2010s, we were well into VET’s ‘Age of Markets'. [1] We placed our trust in the invisible hands of choice and competition to deliver good outcomes.
The language of the time reflected that mindset. Designing reform was like engineering, with policymakers pulling ‘levers’ and adjusting ‘settings’ to control a complicated machine.
Our big innovations in evidence in that age were about confirming compliance: showing the past performance of training providers.
Today, in this new ‘Age of Stewardship,’[2] we’re relying more explicitly on the visible hands of governments, providers, students, employers, and communities to shape outcomes.[3]
We need different metaphors and the mindsets that go with them: less like engineers controlling machinery, more like ecologists nurturing a complex system.
The same shift applies to evidence.
Traditional performance measures still matter, but our big innovations should be about illuminating impact: helping us see not just what has happened and why, but also the future outcomes of learners.
Looking Back with Dials and Tin Openers
There are two long-recognised types of performance indicators for measuring past performance:[4]
- Dials: monitoring tools that show us, like the temperature gauge on an oven, what’s happened in the VET system. They are simple in nature and few in number.
- Tin openers: diagnostic tools that prompt us to investigate, opening the ‘can of worms’ so we can explore why things happened in the VET system. They are complicated in nature and large in number.
Qualification completion rates are an important dial, monitoring a national priority for the sector.
NCVER’s annual Student Outcomes Survey is a tin opener: it tells us, for example, that around one in five don’t complete qualifications for training-related reasons, and another one in five due to personal circumstances.[5] It opens space for more questions about what’s going on and what could be done to improve training and support for students.[6]
The Role of NCVER
Improving VET measures is priority work for NCVER, responding to the clear expectations of our Members.[7]
For example:
- Deepening surveys. This year, we’re asking non-completing students not only why they withdrew, but what might have helped them stay the course.
- Improving timeliness and accuracy. Through internal system and process improvements and the VET Data Streamlining program, data reporting lags will shrink, data quality will lift, and data richness will grow.
- Changing standards. With changes to national data standards and supporting systems, we’ll be able to report Free TAFE at the student, not aggregate level; track completions as reported, not lagged; and report individual and apprenticeship completions, not just training contract completions.
- Tailoring insights. Our portal lets users view and benchmark their own performance, a step towards providing more personalised, actionable insights to help them make better decisions.
- Additional metrics. With Jobs and Skills Australia, we’re exploring complementary measures of performance beyond completion rates. For example, our research shows students who don’t formally complete but finish all their enrolled units often achieve employment outcomes just as strong as completers.[8]
We work alongside our partners, individually and collectively making important improvements to measuring performance better: skills and training departments; Jobs and Skills Australia;[9] and Jobs and Skills Councils and their State-based equivalents.
Looking Ahead with Crystal Balls
With innovations in data and analytics, we can be more confident in a third measurement tool at our disposal: the crystal ball.
These are predictive indicators, with the best built on linked data, careful analytics, and deep research.
They don’t just reflect pasts, they forecast futures.
These indicators are the next frontier for VET and I look forward to NCVER playing an important role,[10] working closely with our shareholders and partners.
Done well, crystal balls could help:
- Shift policy from reactive to proactive.
- Target initiatives where they deliver the greatest return.
- Estimate and assess the impact of reforms.
To make sudden leaps in measuring performance this way, we need at least four things:
- Strong sponsorship. Championing evidence, demanding it, and supporting it.[11]
- Infrastructure. Linking the full administrative data chain of inputs, outputs, and outcomes,[12] as foundations for both bespoke analysis and predictive models to understand, assess and forecast the impact of changes on student outcomes.
- Safeguards. Guarding against data bias, model blind spots, and overconfidence in probabilities.
- Trust. Trust to link and share data and insights across jurisdictions and sectors; trust to evaluate programs, even when results are mixed; trust to be transparent with the community.
Seeing Impact Confidently
The evidence base in VET is improving.
With dials, we monitor performance.
With tin openers, we diagnose it.
With crystal balls, we can anticipate it.
Together they enable certainty about the past, confidence in the future, and curiosity about why.
Our focus can shift from system efficiency to effectiveness, and ultimately, impact – including return on investment.
Nurturing an Evidence Ecosystem
We can’t rely on data and modelling alone.
Administrative datasets and models based on them will always be partial. They can’t show the complete picture.
People’s pathways are messy, full of twists and turns that data can’t fully explain.
The training system is complex, full of loops and dynamics that data can’t fully reveal.
We also need the ‘thick’[13] and ‘small’[14] insights.
We need qualitative research[15] and mixed method evaluations.[16]
We need sector perspectives, student voices,[17] and community stories.[18]
They are essential to interpret, triangulate and challenge the data.
It’s this mix of tools and methods that produces good and useful knowledge about what works, where, for whom, when, and why.
It is only through nurturing an evidence ecosystem that VET can understand, anticipate and improve its contribution to outcomes.
NCVER looks forward to playing its part.
[1] See, for example, Don Zoellner 2022, Mature Australian VET markets: a data-driven case study of public policy implementation, 14,5 Empirical Research in Vocational Education and Training.
[2] See the National Skills Agreement.
[3] The new national VET Outcomes Framework tracks progress of the VET sector. See also NSW Interim Skills Outcomes Framework 2024.
[4] Dials and Tin Openers comes from Neil Carter et al, How Organisations Measure Success: The Use of Performance Indicators in Government (1995). See also Gwyn Bevan and Christopher Hood (2006) What’s Measured is What’s Matters: Targets and Gaming in the English Health Care System vol 84, issue 3 Public Administration 517-538.
[5] The proportions are around one in four for government funded qualifications. It is important to note that the survey shows many students who do not complete qualifications still achieve good outcomes.
[6] See, for example, Andrahannadi, U & Griffin, T 2025, Student support: enabling the learner journey, NCVER, Adelaide.
[8] Wibrow, B, Hall, M & Griffin, T 2024, The student journey: the many faces of completion and non-completion in VET, NCVER, Adelaide.
[9] See, for example, JSA’s work on the National Skills Taxonomy.
[10] NCVER currently projects completion rates and is a custodian of data in the VET National Data Asset.
[11] The National Skills Agreement’s emphasis on data, evidence and evaluation is welcome.
[12] Data assets like NCVER’s statistical collections, the VET National Data Asset, Person Level Integrated Data Asset and state linked data sets are valuable.
[13] See Tricia Wang 2016, Why Big Data Needs Thick Data.
[14] Abhijit Banerjee and Esther Duflo 2011, Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty.
[15] Here, for example, is NCVER’s recent research.
[16] See, for example, the Australian Centre for Evaluation’s rapid review of the Boosting Apprenticeship Completions and Completing Apprenticeships Commencements programs, summarised in Ian Ross and Lisa Paul 2024 Strategic Review of the Australian Apprenticeship Incentive System - Department of Employment and Workplace Relations, Australian Government.
[17] See, for example, Brown, J 2017, In their words: student choice in training markets – Victorian examples, NCVER, Adelaide
[18] See, for example, Thea Snow et al 2024, Storytelling for Systems Change: Listening to Understand.