Does scored VET in Schools help or hinder access to higher education in Victoria?

The systematic introduction of vocational subjects to the secondary school curriculum in the 1990s — VET in Schools — was aimed at helping to retain less academically inclined students at school and to provide students with a broad range of post-secondary options and pathways. The early 2000s saw a broadening of the VET in Schools programs with the scoring of vocational subjects. This meant that particular vocational subjects could count towards both nationally recognised training and a university entrance score. The anticipated benefit of scored VET subjects was an improved status for vocational education and training (VET) in the secondary school curriculum and a further benefit was that it offered viable options to those students who were not entirely certain of which pathway to take — university or vocational training.

This study, which focuses on the experiences of Victorian secondary school students who completed ‘scored’ VET subjects — counting towards the Victorian Certificate of Education (VCE) — looks at whether taking these subjects affects their entry to university in terms of university entry score, receiving a university offer or receiving an offer in a preferred course. Victoria is the focus for two reasons: since the early 1990s, VET subjects have been highly integrated into the Victorian secondary school curriculum; and, secondly, Victoria was the first state to allow scores from some VET subjects to count fully towards a national vocational qualification, the Victorian Certificate of Education and a university entry score. In this study, those who take scored VET subjects represent fewer than 10% of all Year 12 completers in 2011.

Key messages

• For Victorian students who intend to go to university and who complete a scored VET subject this research indicates that there is a sizeable penalty. The average university entry scores for these students are estimated to be six points lower than they would have been had a general subject been chosen, representing around a 5% reduction in university entry scores, on average.
• This reduction in university entry scores negatively impacts upon the chances of receiving a university offer, from 79% to 67%, on average.
• The largest negative impacts on average university entry scores are found with engineering and technology; community, outdoor and recreation; and hospitality subjects.
• The authors suggest that the down-scaling of scored VET subjects may partly explain this impact and they offer an alternative scaling methodology for consideration.

This is an important study as it is the first to attempt to examine any impacts on university access of taking a scored VET subject (in Victoria). In doing so however it highlights an apparent adverse outcome of a pathway originally intended to offer students the best opportunities to pursue the post-school studies most suited to their ability and motivation.

Dr Craig Fowler
Managing Director, NCVER

Despite comprising only a small fraction of all VET in Schools enrolments, programs that count towards both national vocational education and training (VET) qualifications and university entry potentially fill an important role in the upper-secondary school curriculum.

The aim of this study is to take a first step in gaining an understanding of the efficacy of VET in Schools courses by estimating the relationship between enrolling in Victorian Certificate of Education (VCE) VET subjects and university access. We choose to examine the outcomes from VCE VET courses because they represent a model of assessment, known as ‘scored VET’, that closely resembles that applied in general courses. In particular, the assessment involves both written exams and numerical assessment of performance in job-specific tasks associated with units of competency. In this study, the analysis is carried out on a sample of school completers in Victoria in 2011 who lodged a first preference for enrolment in a university course prior to sitting their final year exams.

To meet the aim of this study, we address the following key research questions:

• What is the relationship between taking a Victorian Certificate of Education VET subject and university entry scores?
• To what extent is any relationship explained by scores in VCE VET subjects (direct effect) as opposed to scores in all other subjects taken by VCE VET students (indirect effect)?
• Does any relationship depend upon the type of VCE VET subject chosen?
• What is the relationship between participating in VCE VET subjects and the chances of being offered a place at university?
• What is the relationship between participating in VCE VET subjects and the chances of attaining a first, third or sixth university entry preference?

As far as we are aware, this is the first study to estimate the relationship between taking a VCE VET subject and university access. In estimating the relationship, we compare the outcomes of students who do and who do not take VCE VET subjects, using a rich dataset and econometric techniques to control for differences in a large number of characteristics. Most importantly, we control for differences that may have an important bearing on university entry scores, such as prior academic performance, using Year 9 NAPLAN scores; academic aspirations, using field of education and course cut-off scores for each student’s first preference submitted for university entry; student and peer socioeconomic background; and regional and school-level factors¹. Although we control for a large range of differences between the two groups, to the extent that there are differences that are not controlled for or are imperfectly controlled for the relationship between taking a VCE VET subject and university access may not reflect a causal impact.

The econometric technique used in this study is propensity score matching, a quasi-experimental method that simulates random assignment into VCE VET by selecting, for each student who chooses a VCE VET subject, a control group with the same observable characteristics but who did not enrol in a VCE VET subject. The outcomes from the matched control group represent counterfactual outcomes, against which the VCE VET outcomes are compared to isolate the relationship between taking a VCE VET subject and university access. A key feature of this study is the use of a unique Victorian dataset that links, at the individual level, administrative population data on university preferences used in the university admission process; university entry scores; characteristics of students, schools, parents and place of residence; and university offer information (the last from a large survey of graduates).

Another key feature of this study is the development of a decomposition approach to help explain any estimated relationship. The decomposition approach splits any relationship into direct and indirect effects. For those who take a VCE VET subject, the direct effect is defined as the difference in the score in a VCE VET subject relative to the score if a general subject had been chosen instead. The indirect effect, or spillover, is defined as the average score in all general subjects when a VCE VET subject is chosen, relative to the average score in these subjects had an alternative general subject been chosen. The results are also estimated for scored VCE VET subjects across seven subject groupings: business and finance; community, outdoor and recreation; dance and music; engineering and technology; information technology; hospitality; and equine industry. The impacts are estimated across subject groups because there are insufficient observations to allow robust estimation by individual subjects.

Results

We find among students who intend to go to university, controlling for a range of differences between those who do and who do not take a VCE VET subject, that those who take a VCE VET subject have a six-point lower score on average than those who do not take a VCE VET subject (111 compared with 117 out of a possible 205). This represents around a 5% lower university entry score on average and is robust to the range of alternative key assumptions that underpin the analysis. Across VCE VET subject groupings, we find some variation in magnitude of the estimated negative association, with significant negative associations found in four of the seven subject groupings (engineering and technology; community, outdoor and recreation; hospitality; information technology) and no statistically significant negative results found for the rest (business and finance; dance and music; equine industry).

Consistent with a lower entry score, we also find that those who take VCE VET subjects have a lower chance of receiving a university offer. In particular, on average, taking a VCE VET subject is associated with a 12-percentage-point lower chance of receiving a university offer. In other words, 67% of students who take VCE VET subjects who apply to go to university receive an offer, but it is estimated that the chance of receiving an offer would be 79% (12 percentage points higher) if these students had taken a general course instead. The large difference in the chances of attaining a university offer, despite only relatively small differences in entry scores, is because VCE VET students are on average around the middle of the entry score distribution. Therefore, their chances of receiving an offer are sensitive to small changes in entry scores, including changes associated with course choice. Similarly, taking a VCE VET subject is associated with a seven-percentage-point lower chance of attaining a top six preference, from a total of 12 preferences.

We estimate that most of the six-point lower entry score associated with taking a VCE VET subject (around four points or 70%) can be attributed to a negative direct effect, while the remainder is due to a negative indirect effect. The dominance of the direct effect over the indirect effect reflects the relatively low scores attained in VCE VET subjects among students who intend to go to university. On average among VCE VET students who intend to go to university, their scores in VCE VET subjects are lower than those attained in their other subjects. In this study, we cannot be precise about the underlying causes of the negative direct and indirect effects, but they may include the disruption associated with accessing training off campus, poor suitability of VET training for students who intend to go to university and down-scaling of VCE VET scores.

Although we cannot precisely pinpoint the source of the direct effect, an exploratory analysis suggests that the down-scaling of VCE VET subjects may be a key source. Scaling ensures that scores across different subjects can be compared on an equal footing so that students do not gain an unfair advantage by choosing any particular combination of courses. Existing scaling arrangements in Victoria correct for differences in the strength of competition or difficulty in attaining the mean score in a given subject. Competition in a given subject is measured as the mean score in all other subjects taken by students in that course. In subjects where competition is less than average (mean score in other subjects is less than 30), the scores are scaled down; the opposite holds when competition is above average. In VCE VET subjects, scores are scaled down, but we find evidence to suggest that the extent of the down-scaling may be greater, on average, than in many general courses because a large proportion of the students in these courses do not intend to go to university but appear to focus their effort on these courses, to the detriment of performance in other courses. Therefore, the measure of competition used in the scaling may underestimate the true difficulty of attaining the mean in VCE VET courses. Unlike general courses, VCE VET students who do not intend to go to university have an incentive to focus their effort to attain credit for, or attainment of, a national qualification.

In this study, we cannot rule out the possibility that the lower average entry score for VCE VET students is due to the differences in factors that are not controlled for and which affect entry scores. That said, the dominance of the direct effect suggests that this is unlikely. If there are uncontrolled differences in factors that lead to lower entry scores for VCE VET students, then to explain the entire gap they must have a disproportionately negative effect on scores in VCE VET subjects compared with scores in other subjects. Such factors may be differences in preferences, such as a preference for hands-on learning, but they are likely to disproportionately increase performance in VCE VET subjects. If we assume that uncontrolled-for factors have the same effect on all subjects, then at the most they are no larger than the per-subject indirect effect, which means they would explain no more than a third of the estimated gap in university entry scores².

Implications

The stated aim for the introduction of VCE VET subjects to the academic curriculum was to enhance the status of VET programs by recognising performance in these subjects in the same way as performance in other VCE subjects (Victorian Curriculum and Assessment Authority 2010). As well as enhancing the status of vocational education and training, these subjects potentially also serve dual purposes: to provide vocational options for those who do not intend to go to university and to allow students in the middle of the academic distribution to pursue vocational options without closing off their pathways to university. In theory, these two purposes are compatible, but in practice the results presented in this paper suggest that there may be trade-offs.

To better meet the dual purposes of VCE VET subjects, we suggest two courses of action. First, more research is needed into the extent to which different student motivations in scored VET subjects leads to greater down-scaling than would otherwise be the case. If further investigation were to confirm our over-scaling explanation, then one possible response would be to adjust the scaling of VET subjects to account for differences in motivation. One way to do this would be to restrict the scaling to only those students who intend to go to university, measured by whether or not they lodge an application for admission to a university course prior to sitting their final exam. Scaling subjects in this way should have relatively minor impacts on scores in general subjects where a high proportion of students intend to go to university. However, there are some general VCE subjects outside the VET system, such as industry and enterprise studies, that also attract relatively high proportions of students who do not intend to go to university but who may also focus their effort in a particular course because the course is especially useful for employment preparation. Given that the main purpose of scaling is to allow students to be equitably ranked for university entry, then the scaling should be based on the scores of students who are competing for university entry.

Second, while the negative indirect effect is small and may be potentially explained by uncontrolled-for factors, there are a number of possible low-cost precautionary measures that could be taken by government and schools. For government, collecting data on the time spent in off-campus training may help in assessing and monitoring any academic impacts. If necessary, these data would also allow better coordination of training across local school clusters where close-by options in TAFE (technical and further education) institutes may be inadequate. For schools, an appropriate response may be to encourage their students to complete VCE VET subjects in Year 11 rather than in Year 12.

Importantly, we stress that this study only examines the impacts of taking VCE VET subjects on direct access to university. Other important outcomes from VCE VET programs, such as indirect access to university (for example, by completing a diploma course), participation in post-school VET study, retention in post-school study, and employment outcomes are not investigated here, but should be considered in any overall evaluation of these programs. Previous studies by Anlezark, Karmel and Ong (2006), Lamb and Vickers (2006) and Polidano and Tabasso (2013) have demonstrated positive benefits to school retention and initial labour market outcomes from unscored VET in Schools courses.

¹   The course cut-off score to control for the level of academic aspiration is from 2010, which was the latest available at the time students lodged their university course preferences. This control is not included in the standard results because there are a number of missing student preferences. This control is added as part of the sensitivity analysis (see appendix E).

²   If we assume that all of the indirect effect is due to differences in uncontrolled-for factors and that these factors have the same negative effect on all subject scores, then we can say that 0.408 (1.632/4) percentage points of the 3.985-percentage-point gap due the direct effect is also explained by uncontrolled-for factors. Therefore, at the most, uncontrolled-for factors would explain 2.04 out of 5.618-percentage-point gap.