Research

Summary page

Research

Technical report: Survey methodological options for a) non-completers of VET and b) apprentices and trainees over time

This report investigates the design and development of appropriate methodologies for two separate surveys: firstly, a national survey of students in vocational education and training (VET) who leave without completing any courses; and secondly, a longitudinal survey of VET students who pursue the apprenticeships and traineeships pathway.

There is a need for more information on both completion data as a key performance measure in VET and the impact of training on career paths, income and labour market experiences. Current destination surveys of VET students tend to focus on course and module completers; there is no national follow-up of those students who drop out of VET programs. Similarly, there is no adequate longitudinal data on apprentices and trainees to assist in estimating the long-term impact of contracted training on career paths and earnings.

The investigation was undertaken in several stages. The first stage involved an extensive review of research and literature detailing similar surveys conducted in Australia and overseas. The second stage comprised a series of workshops and interviews with a range of key agencies (including the Australian National Training Authority [ANTA], the Department of Education, Science and Training, the Victorian Office of Training and Tertiary Education, and the Tasmanian Office for Post-Compulsory Education and Training) to canvass views on the options and issues raised from the review of methodologies. The work undertaken as part of the initial stages was used to help prepare this report outlining the strengths and weaknesses of alternative methodologies and provide options on ways of conducting the two surveys.

The report is organised in three sections. Following the introduction, methods for undertaking the survey of VET non-completers are investigated. This section provides a broad overview from the literature of methodologies adopted for similar projects within Australia and overseas, including countries such as Austria, Canada, Denmark, England, France, Germany, Ireland, Italy, Netherlands, Norway, Portugal, Spain, and the United States. It also raises definitional issues that need to be addressed in the design, development and conduct of the survey. It concludes with options for establishing and conducting the survey, based in part on the advice and views of the experts interviewed in the consultation phase of the project.

The third section of the report looks at longitudinal surveys of apprentices and trainees. It presents information and raises issues that have emerged from a review of similar surveys undertaken in Australia and overseas. Options for establishing and conducting the survey are given in this section, again based in part on the advice and views of the experts interviewed in the consultation phase of the project, as well as on international work.

Survey of VET non-completers

There are several main aims in undertaking a survey of VET non-completers. One is to obtain information on the characteristics of the non-completers—their backgrounds, courses of study, and enrolment patterns. Another is to find out the reasons why these students do not complete their study or training, in order to assess whether the rate can be reduced through targeted support and other measures. A further aim is to examine the outcomes of non-completers to assess the effects on individuals of withdrawal from VET. Some of this information does not have to be collected from individuals themselves, but can be obtained through an analysis of administrative provider data on students assembled as part of the national VET statistics collection, and examined in the context of the Australian Vocational Education and Training Management Information System Standard (AVETMISS).

In analysing administrative provider data on students to determine non-completion rates, it will be necessary to target the non-completers more comprehensively to gain some measure of students’ intentions for study. While it is still possible for students to change their intentions over time, the inclusion of an intentions question on the student enrolment form would provide a reasonable foundation for assessment of non-completion.

Through the analyses of administrative provider data on students, withdrawal or non-completion can be measured. The National Centre for Vocational Education Research (NCVER) has developed a methodology to link a student’s records across different years using a number of demographic and other variables. The method assigns a unique identification to each person and thus allows links to be made between an individual’s enrolment records across years. The course code, which is unique to each course across all years, allows identification of whether a student is continuing in the same course or not. Using the matched records it is possible to analyse non-completion by tracing the enrolments and records of students over a defined period.

The analyses of administrative provider data on student withdrawal or non-completion need to be supplemented by a survey of non-completers every three years to collect information on the experiences, views and outcomes of students who are identified as non-completers in the previous year.

Administrative data can tell us very little about the reasons for non-completion and the outcomes of non-completers. This information requires surveys of the non-completers themselves, surveys which attempt to establish the reasons for non-completion, the experiences of non-completers, and their outcomes. Such surveys do not need to be undertaken as regularly as those used to derive estimates of rates of non-completion using administrative data. However, they are important because they can provide valuable information on some of the underlying behavioural factors leading to withdrawal and non-completion of VET study.

The definition of non-completion is difficult in VET. Not all students who enrol in a VET course intend to complete the full qualification. Some undertake their study with a view to completing only some elements or modules of a course rather than the full qualification. This means that there are various groups of VET participants who have different intentions, and the notion of non-completion will vary according to the group of participants. For this reason, one possibility in a survey of non-completion is to undertake analysis based on different definitions, one option being for the survey to be undertaken in conjunction with and using the same methodology as NCVER’s Student Outcomes Survey, but augmented to include students who meet the following criteria:

those who were enrolled in a course in the public VET system through a technical and further education (TAFE) institute in Australia two years previously and were not enrolled in the target year and had not successfully completed any module
those who failed at least one module two years previously, had not completed a qualification, and were not enrolled in the target year
those who had not completed all of the modules required to graduate for a qualification two years previously, and were not enrolled in the target year.

Longitudinal survey of apprentices and trainees

The main aim of a longitudinal survey of apprentices and trainees is to provide data on their backgrounds, progress and outcomes in order to have a better understanding of the long-term impact of contracted training on career paths and earnings. The profiling of apprentices and trainees currently undertaken using administrative data provides important information on issues such as completion and non-completion. However, it does not provide data on the behavioural factors associated with completion and withdrawal, nor on the impact of apprenticeships and traineeships on career paths and earnings. This requires longitudinal survey data.

It is suggested that the Canadian National Apprenticed Trades Survey (Statistics Canada 1997) be used as a model for the survey. The Canadian survey collects information for comparing and contrasting the labour market activities of completers and discontinuers. It examines the relationship between apprenticeship training and occupation two to three years after apprenticeship, and the factors associated with discontinuing an apprenticeship program. The target population for the survey covers both completers and non-completers from a given period. Data are collected by telephone interview approximately two years after leaving or completing apprenticeship training. In an Australian context the sample could be drawn from NCVER’s National Apprentice and Trainee Collection.

If based on the Canadian model, the approach requires selection of a stratified randomly selected sample of apprentices and trainees for a given target year. In 2003, according to national figures on Australian apprentices and trainees, there were 117 800 completions and 122 700 withdrawals from apprenticeship and traineeship programs (NCVER 2004). If the survey sample were to be stratified by state/territory, age and sex, then to derive state and territory estimates with a confidence level of 95% and a 5% confidence interval would require an achieved sample of approximately 3200 completers and 3200 non-completers. This would be based on an individual state and territory sample size of 400 completers and 400 non-completers. These sample sizes would provide robust and reliable estimates at a national level. If further stratification were desired to disaggregate the sample by training package or qualification level, then larger sample sizes would be required.