Towards evidence-based vocational education and training: A discussion paper

Background

The changes that have occurred in vocational education and training over the last decade have profound implications for the role of evaluation. There has been a transition from a supply-side system to one which is moving towards demand-side structures, with funding more tied to outputs rather than inputs; the range of training providers is becoming more diverse; and there has been a change in emphasis from external controls to quality assurance mechanisms. Equally significant is that these transitions have occurred in a system in which there has been considerable integration of policy-making and practice which crosses the boundaries of both traditional federal/State jurisdictions and government/industry responsibilities. All of these developments lead to a strong case for the development of new forms of accountability.

To ensure that future decisions on vocational education and training are well founded the National research and evaluation strategy has been developed, administered by the National Centre for Vocational Education Research (NCVER). This discussion paper is intended to inform the further development of the national strategy by strengthening its focus on evaluation.

The state of evaluation in Australia

Unlike research and development, evaluation is not justified in its own right. More or better evaluations will not necessarily improve the quality of education and training that people receive: it is the way in which evaluative information is used that is the key. For this reason the most important need is for structures, systems and ways of thinking that are based on informed judgement?for which competent evaluations are the fodder.

An analysis of evaluation at different levels of the vocational education and training system suggests that there is an uneven distribution of levels of evaluative activity and expertise:

Extending the evaluation aspects of the national strategy

Evaluation is not always worth doing: sometimes it is not an effective use of resources, it may not always be an appropriate response and, in a number of cases, the will to evaluate does not exist. What a national strategy can achieve, however, is a number of outcomes which will assist the use of evaluative methods to support decision-making.

This could be done by extending the existing National research and evaluation strategy to achieve the following additional outcomes:

• A useful level of evaluative activities The vocational education and training sector should aim for a situation in which training providers and enterprises plan and make decisions on the basis of a sensible and systematic use of evaluative techniques, with such information forming an integral part of their operations. This goal could be achieved by aspects of national training policy being formally 'referred' to the National Research and Evaluation Committee (NREC) or other bodies for evaluation, and developing 'showcase' examples of quality assurance mechanisms which incorporate evaluative approaches.

• A capacity to evaluate
  All levels of vocational education and training need sufficient people skilled in carrying out evaluative activities and interpreting the results?particularly staff within State training authorities, industry training advisory bodies (ITABs), and training providers. This could be achieved by facilitating forums of managers to discuss quality assurance issues, by arranging for the critical appraisal of evaluative studies, and by other mechanisms to develop expertise within all groups of stakeholders.

• An informed and robust critique
  Public policies and practices should be subject to better-informed critique?both publicly and, when deemed necessary, in confidence. This could be achieved by writing this expectation into contracts for national key research centres.

Over the last few decades evaluation has been used only sporadically in vocational education and training?and less than in other sectors of education?to improve educational outcomes. There is now a real opportunity for it to make an effective contribution to decision-making in vocational education and training through the national strategy, and this discussion paper isintended as a first step.