Measuring the outcomes of VET: Review of research

This review is a consolidation study describing the range of outcome measures used in the vocational education and training (VET) system and examines a range of literature concerning these measures. The first section of the paper aims to identify various perspectives on outcome measures, those of governments, of employers, of students and of the broader community; the paper describes how these perspectives influence the types of measures in use.

The paper establishes the role of outcome measures as one of three key dimensions of system performance and examines in some detail the seven key performance measures (KPMs) for the VET system established by the Australian National Training Authority (ANTA) Ministerial Council. Four of these seven KPMs are judged to be outcome measures. Other outcome measures that impact upon the VET system and of interest to other government agencies, such as the Productivity Commission through its Report on government services, are also considered.

The community, employers and students all share an interest in ensuring that the VET system produces skills needed in the labour market, although their perspectives can differ. The community has a social and economic interest in seeing low youth unemployment and high levels of training investment by employers. Employers require the VET system to provide workers with the right skills mix in sufficient numbers. As most VET students are motivated by vocational objectives, they too want to acquire skills relevant to the labour market. The views of employers and students on the VET system are gained by examination of the main source documents the employer satisfaction survey (ESS) and the graduate destination survey (GDS) (now the student outcomes survey). The analysis of these sources undertaken by ANTA and the Productivity Commission is also briefly reviewed.

Separate consideration is given to the delivery of VET in schools. Outcome measures in this sector are less developed than in the post-school sector and to some extent are merged with outputs.

Having established the various perspectives on VET outcome measures, the paper reviews a range of literature within these contexts. Labour market outcomes for VET students are impressive in terms of accessing employment; however, there are marked differences for some groups of graduates. The paper canvasses the issue of how much responsibility the VET system has for employment outcomes and compares measures internal to the system,, such as participation rates and module completion rates, with external measures such as unemployment rates..

Women, indigenous people, people with a disability and those from a non-English-speaking background (NESB) achieve poorer employment outcomes than other graduates. Much VET research relating to equity has identified a need for more specific outcome measures for equity target groups, such as culturally appropriate measures for indigenous people. Another important point to emerge from the research on equity target groups, of more general relevance, is a need for more specific evaluation studies of courses, rather than a reliance on wide-ranging data collections.

Several researchers (Gonczi 1998; Anderson 1997) have traced the origins of the greater focus on economic outcomes to the globalisation of the Australian economy and related anxieties over Australia's international competitiveness. Fitzgerald (1994) directly links the need for meaningful outcome measures to the growth of the training market. A theme arising regularly in the literature is the apparent conflict of values between what are seen as economic outcome measures and those identified as educational .

The growth of the competitive training market has been the catalyst for what is probably the central theme of the debate over VET outcomes; that is, the balance between educational and economic values. Several important related themes are also reviewed the role of VET in reducing youth unemployment and industry investment in training.

Considering the central role that the training market policy continues to occupy, it is surprising that there appear to be no measures to determine how much this policy has actually increased choice and diversity within the VET system. Given also that a key rationale for introducing the competitive market approach to VET funding was to produce a reduction in VET unit costs, it is surprising that the Productivity Commission found government expenditure per module load completion rising between 1996 and 1997 (Industry Commission 1999).

Burke (1998) identifies reducing the youth unemployment rate as an important objective of the training reform process. Robinson and Ball (1998) show that young people s participation in VET rose between 1990 and 1996, despite reductions in apprenticeships and traineeships. However, during the 1990s the rate of youth unemployment rose. There is a legitimate view that the VET system cannot be regarded as accountable for the decline in youth full-time employment. Some writers however have questioned why the large increase in VET enrolments has largely been in older students rather than in young people. Others (for example, Curtain 1998) have pointed out that Australia, on international standards, is well supplied with university graduates but compares less favourably at both upper and other post-secondary levels, hence implying that increased participation by teenagers in VET could assist the nation to improve its overall educational standards.

The paper explores some of the issues relating to skills shortages from the employer's perspective and industry investment in training. The evidence appears to indicate that skill shortages in the VET area are not widespread. There is also considerable interest in the literature in the degree to which industry views investment in skilling the workforce as a path to productivity gains, with several writers suggesting that industry generally fails to link training expenditure to productivity gains.

Finally, the paper identifies a range of issues that may warrant further research. The longer-term outcomes of VET are identified as one area with limited coverage to date. Graduates have shown some level of dissatisfaction with the provision of information on courses and careers; however, there seems to be little research into how to improve this aspect of VET delivery.

Given the relatively slow rise in teenagers take--up of post-school VET compared with older students, research into reasons for this seem warranted. Related to this might be further research into the development of outcome measures for school students undertaking VET courses. Are they, for example, more likely to progress into post-school VET and what other outcomes are achieved?

The training market remains a foundation of VET policy, yet the assessment of its benefits, both in terms of cost reduction and increased choice and diversity of provision remains largely unknown. Some evidence suggests costs have not declined, while the only measure of choice appears to be the number of registered providers. Research into user choice and diversity might embrace both employer and student perspectives.