Is the stock of VET skills adequate? Assessment methodologies

Whether the stock of vocational education and training (VET) skills is adequate to meet the needs of industry is a question that has interested and perplexed policy-makers for a long time. Four approaches have been used to try to answer the question:

• manpower requirements
• international, national and industry comparisons
• labour market analysis
• indicators

This report considers examples of applications of these approaches and details why they provide less than fully satisfactory answers to the question whether the stock of skills can be regarded as adequate.

The report proposes that an electronically-upgraded indicators approach is likely to prove the most satisfactory option for VET decision-makers.

The report also proposes a method for building on such an approach to provide an aggregate measure of the adequacy of VET skill formation.

The manpower requirements approach to assessing the adequacy of stocks of skills attempts to derive skill requirements implicit in national and industry forecasts, to compare these with projected supplies of skilled workers, and to formulate training programs to eliminate the forecast excess demands or supplies of skills.

Three attempts to use the manpower requirements approach to assess the adequacy of the stock of skills are discussed in detail, including the Organisation for Economic Co-operation and Development's (OECD) celebrated Mediterranean Regional Project. Imprecision in forecasting shortages or oversupplies of skills is shown to be important in each of the attempts.

It is concluded that the 'quasi-engineering' notion of manpower requirements does not rest on secure foundations. The manpower requirements approach to bringing about 'skills adequacy' is seen to be ambiguous, and even illusory. In consequence, the importance of ensuring both flexible training systems and labour markets cannot be overestimated if rapid and relevant adjustments are to occur to offset inevitable errors of foresight in the supply of skills.

The international, national and industry comparisons approach compares national, State or industry skill profiles with model, target or yardstick profiles and formulates training programs to attain such model, target or yardstick outcomes.

The work of Cullen in work skills and national competitiveness (1997a, 1997b, 1998), is given detailed consideration as an example of how comparative analysis has been applied to assess the adequacy of skills formation in Australia. A group of international comparative studies presented by Ryan (1991) is also considered.

Cullen uses two principal analytical tools: correlations between work skill measures of international competitiveness rankings and frequency distributions of qualifications classified by State and industry to reach conclusions about the adequacy of national, State and industry skill stocks.

Australia's performance in the formation of skills is benchmarked internationally, by State and industry. 'Gaps' which emerge from comparisons between Australian skill stocks and the proposed benchmarks are offered as guiding information for policy-makers. Such a 'gap', benchmarking approach is shown to be unable to provide the necessary guidance, however, most obviously in cases where Australia has 'over-achieved' in comparison with the proposed benchmark. For example, Australia has a higher 'people factor' rank than its overall competitiveness rank, implying that Australia's competitiveness is not presently being constrained by 'people factors' (such as skills) but by other factors.

The lack of strong statistical evidence to support the asserted relationships between education variables and economic performance suggests that the detailed guidance that such studies can provide to VET policy-makers (and work skills providers and consumers) is likely to be small.

For example, after analysing regressions of national competitiveness against national qualification profiles, there does not appear to be any strong reason to support the expansion of any one particular level or type of education, as the effect of the various qualifications on overall competitiveness appears to be quite similar to each other.

The lack of strong statistical evidence supporting relationships between education variables and economic performance indicators does not mean that education variables exert no influence on economic performance. Rather, other factors may be as important, or even more important, and their relative importance may change from time to time. Hence, it is not possible to define an appropriate VET strategy independent of considering the detailed circumstances that apply. This is not done in the rather simple 'comparisons' approach.

Studies presented by Ryan (1991) suggest that the 'fungibility' (i.e., transferability) of experiences and policies from one country to another is critical in determining the relevance of comparative analyses. Unfortunately, the studies reject the notion of 'fungibility' or transferability of policies and practices across countries. If VET policies and practices are 'non-fungible', statistical analyses of international cross-sectional data are of questionable relevance and value.

The practice of simply copying policies and programs from one country to the next is inappropriate. Developing practices for one's own country based on lessons drawn from other countries is what is required. Assessments of the adequacy of stocks of skills can only be made in the context of the interdependence of the stocks of skills with other social and economic factors, including the demand for skills.

Labour market analysis

The labour market analysis approach employs models of markets for skilled labour to forecast future excess supplies and excess demands and to formulate policies to correct such imbalances, if such policies can be identified and implemented.

R B Freeman's classic study, The overeducated American (1976), is discussed, together with an important collection of papers on the topic edited by H Metcalfe (1995). It is noted that forecasting the future is difficult in the social sciences because the models used as the basis for forecasts can easily be faulty. A model that is appropriate for one period may omit factors or changes to the social system, and turn out to be inapplicable in another period. Well-established generalisations about markets may not adequately describe the behaviour of some sub-markets, despite adequately describing the broader market of which they are part. Even with the correct model, forecasts can err because of unexpected events that falsify inferences about the future. Social scientists do not have an enviable record for predicting the future.

The phenomenon of dynamic 'cobweb' adjustments of labour markets to imbalances between supply and demand may lead to inadvertently counter-productive responses by governments?a classic example (in a non-training field) being Australia's red grape 'vine pull' of the 1980s. Grapes that were thought to be in chronic excess supply turned out to face a prolonged and massive market shortage.

While the economic modelling of skilled labour markets can be instructive, it can also lead to wrong policy prescriptions. The development of work skills should be treated as part of a broad process of economic evolution, not as a factor that can lead economic progress on a forced march.

The indicators approach looks at indicators of the present and likely future condition of skilled labour markets and forms judgements as to the most appropriate courses of action to be taken to improve expected skilled labour market outcomes. This approach to assessing the adequacy of skill has been the most widely used in the context of Australian VET skills.

In the long run, if left alone, labour markets will adjust, changing relative wages, hours and qualifications required for jobs so as to reduce imbalances in the supplies of and demands for skills of various kinds. However, in the short run, mismatches between the supply and demand for skilled labour can lead to costs for individuals and for the economy. Manpower forecasting has the potential to reduce the costs of these mismatches. Both quantitative and qualitative indicators of skilled labour market imbalances are presented as a basis for forecasting likely trends.

Three Australian studies using the indicators approach are discussed. These support the view that qualitative 'indicator' frameworks for assessing the adequacy of future supplies and demands for skills are preferable to strictly quantitative approaches. Analyses making maximum use of current and historical information made available to knowledgeable policy-makers are likely to lead to the most useful solutions to problems of imbalance between the supply and demand for skills.

One of these studies, Occupational outlook, (Department of Employment and Industrial Relations 1987) is regarded by the authors as one of the more sensible and valuable aids to understanding skilled labour markets in Australia. It presents a large range of relevant data clearly and succinctly while not concluding or forecasting too much. The need to disaggregate the data contained in Occupational outlook can be addressed by publishing the material it contains in an electronic format.

The above review of various approaches to considering the adequacy of the stock of skills indicates that technocratic 'solutions' to meeting the skills needs of nations, States and industries all turn out to be ambiguous and illusory. There appears to be no substitute for sensible people as key decision-makers, faced with appropriate incentives, making informed judgements based on an array of relevant and accurate indicators, to provide options for reform.

Flexibility is also required in the education and training system and in the labour market in order to allow adaptation and adjustment to occur to offset the inevitable errors in investments in skills that result from imperfect foresight.

Further, appropriate solutions do not always lie within the realms of education and training. Therefore, a range of policy options, including options lying outside the training system and the labour market should be considered when attempting to redress mismatches in the market for skilled labour.

It is proposed that the tool most suited to assessing the adequacy of stocks of VET skills would be a highly detailed and upgraded electronic version of Occupational outlook. This would present a large variety of indicators of imbalance in the demand and supply of skills needed by enterprises, by linking a series of regularly updated databases.

This presentation platform should be made available to a range of informed people at 'key decision-making points' within a flexible labour market. The centrepiece of such a tool would be an occupational/industry spreadsheet, containing data in time series, in addition to commentary regarding the data and occupations and lists of relevant contacts and references. Australian Standard Classification of Occupations (ASCO) and Australian and New Zealand Standard Industry Classification (ANZSIC) classifications are suggested as the basis of the data to be included in the spreadsheet.

It is noted that, if the approach is to be successfully adopted, industry would be required to contribute to determining which indicators were chosen to represent the skills position in each industry or occupation. Twenty-five relevant indicators including vacancies, training costs per employee and rate of geographical / product market change are suggested.

The VET sector can be measured in one of two ways, consistent with national economic accounting principles:

• by adding the values added by VET
• by adding the incomes received by employees and owners involved in VET

To ensure that skill development is of genuine value to the economy it is necessary to ensure that the users or final consumers of each good are the final arbiters of its value?not the producers or providers. It is necessary, in order to avoid Soviet-style accounting, to ensure that economic production is of genuine value. Two methods for ensuring genuine value are:

• if the product or service has been sold to a willing buyer
• if the product or service has been provided by governments whose performance can be subjected to open scrutiny and free debate

Valid concerns about the Australian VET system include whether decision-making processes are sufficiently responsive to what Australia's enterprises and citizens (consumers) really want. The proposed VET grass roots planning process is expected to ensure the outputs and wants from the VET system are qualitatively matched.

To assess the adequacy of VET skills a four-step calculation is proposed:

1. Activities contributing to the formation of VET skills must be identified.
2. Identified skill formation activities should be valued by their values added.
3. Values added should be summed to provide a measure of total value added by the VET sector.
4. The annual additions to the stock of VET skills could be aggregated to provide a measure of the value of the stock of VET skills each year. Total value of additions to stocks could complement actual annual additions to stocks.

When training is distinct from production, the identification of costs is relatively straightforward. However many training and learning activities are learned and taught in the course of normal production activities. Including informal on-the-job training and learning in aggregate measures of VET skills is consistent with the emphasis on being responsive to enterprise needs.

Value added by the formal (structured) training of VET is relatively straightforward to measure, and is presently identified in the national accounts. Informal (structured and unstructured) training are not explicitly contained in the national accounts. Wage payments associated with employees undertaking informal training should be identified and allocated to enterprises' training accounts rather than their productive accounts.

Enterprises that provide training should be considered to have two activities-a production activity and a training (human capital investment) activity. Resources devoted by an enterprise to investments in training and learning should be counted as value added in training, not as value added in the enterprise's production. Methods are proposed by which on-the-job investments in the skills of employees can be estimated.

How big is the annual VET sector if measured using the proposed methodology? Australian Bureau of Statistics (ABS) figures show that formal training expenditure by enterprises accounts for two per cent of wages and salaries annually. It is assumed that informal training by enterprises also accounts for 1.5 per cent of wages and salaries. In addition to enterprise expenditure ABS data shows government spending on VET is about 0.6 per cent of gross domestic product (GDP) annually. Thus the total size of VET is about 2.35 per cent of GDP which is about double the size of the university sector.

After reviewing various methodologies for assessing the adequacy of VET skills, this paper develops a methodology that attempts to maximise the likelihood that the stock of VET skills being developed will provide skills of value to Australian enterprises. The methodology incorporates grass roots decision-making-considered essential to guarantee quality-and outcomes that can be aggregated to summary measures of value of stocks of skills.

The methodology is consistent with national accounting principles and is capable of providing an aggregate measure of Australia's skills development that can be compared in a consistent way with other countries' skills development, as well as enabling comparison of Australia's performance in skills development with Australia's performance in other forms of investment.

What is proposed is a flexible aid to better, more value-creating, VET decision-making and assessment, one that recognises the futility of attempting to undertake assessments using the technocratic frameworks adopted by the centralised planning frameworks of the past.