Assessing and acknowledging learning through non-accredited community adult language, literacy and numeracy programs

Beyond the obvious acquisition of the three ‘R’s’—reading, writing and arithmetic—many other benefits derive from learning. Increases in self-esteem and confidence have often been cited as consistent, if unanticipated, outcomes of learning and have been achieved even when there has been limited gain in the competency being learned. Further, such confidence-building can have a positive impact on how and what an individual learns. While the measurement of objective competencies acquired through learning is well developed, the measurement of the wider benefits of learning is still in its infancy.

This report by Darryl Dymock and Stephen Billett identifies the extent to which indicators might be developed for a range of learning outcomes in non-accredited community adult language, literacy and numeracy programs. It follows Dymock’s earlier work, Community adult language, literacy and numeracy provision in Australia: Diverse approaches and outcomes (NCVER, 2007), which attempted to gauge the extent of provision of this type of training in Australia.

Key messages

• It is possible to assess the wider benefits of non-accredited adult language, literacy and numeracy learning. However, different types of assessment instruments, which cater for the diversity and complexity of learners’ needs, motivations and outcomes, are necessary. Special attention must be paid to the language adopted in the instruments so that these can be easily used by tutors and their students.

• Both learners and tutors derive personal and educational outcomes by participating in the process of assessing and acknowledging learning outcomes.

• Tutors in this field, many of whom are volunteers, are in need of professional development to fully understand the purpose and language of the assessment instruments.

• The right approach to assessment can build the self-confidence of students, many of whom are not suited to formal education settings.

Tom Karmel
Managing Director, NCVER

Purpose
This research explores the potential for developing instruments that assess and acknowledge the wider benefits of learning for adults participating in non-accredited community language, literacy and numeracy programs. The extent and variety of the learning needs and motivations of participants in these programs are not yet sufficiently accounted for in other assessment frameworks, such as the National Reporting System (NRS). ¹ Therefore, it is necessary to identify approaches and procedures for assessing and acknowledging the outcomes that participants achieve in these courses. For this study, ‘non-accredited’ learning is that for which no nationally accredited vocational education and training (VET) qualification is awarded to learners.

Approaches

The research comprised three stages. Firstly, a review of national and international research was undertaken to map the wider benefits of learning that have been identified in non-accredited adult learning. This review was augmented by an analysis of data collected in a 2006 study that mapped the provision of non-accredited community learning in adult language, literacy and numeracy in Australia (Dymock 2007a).

Secondly, following that review, a number of potential assessment instruments that could be used to assess and acknowledge the outcomes of learners in non-accredited adult language, literacy and numeracy programs were identified by the authors. Through collaboration with five research partners, six of these instruments were selected as being most appropriate for these purposes. ² The research partners were community education providers in Queensland, Victoria, South Australia and Western Australia and were selected for reasons of diversity in user groups and locations.

Thirdly, the selected instruments were trialled and evaluated in the field at the partner sites. After the trial period, partner representatives reported on their experiences in using the instruments to assess and acknowledge wider learning outcomes in non-accredited adult language, literacy and numeracy.

Findings

An understanding of the full extent of the learning arising through learners’ participation in non-accredited language, literacy and numeracy programs takes account of the following key factors: the purpose or motivation for learning; the processes of assessment, if it occurs at all; and the competence of voluntary tutors. As a result, the study found that a range of instruments and approaches would probably be required to assess and acknowledge learning outcomes that are specific to particular learners or cohorts of learners.

Diverse needs

People who participate in these programs range from learners with significant disabilities, to those whose language development has never been fully realised, through to those developing their language competence because of social, family or work-related necessity. Other reasons for participating include changes in attitudes and wanting to develop a greater sense of self and personal competence (that is, being confident enough to accept new challenges). Such diversity of learner and learning goals warrants assessment processes commensurate with these goals.

Diverse outcomes

From the review of literature the research identified seven categories of ‘wider benefits’ of learning. These were: self-confidence; engagement with others; attitudes to learning; agency/pro-activity (actively learning through experience); life trajectories; personal growth; and social capital. This study confirmed those categories and the importance of acknowledging the wider benefits of learning. In general, these benefits include enhanced personal confidence, heightened social competence, and the opening-up of possibilities and prospects that were not options prior to participation in these programs. Outcomes such as these are not the focus of or are captured in accredited learning frameworks like the National Reporting System. Importantly, the findings of this review also pointed to the development of a ‘learning identity’ as a key element of engagement in the learning process, a concept that needs further research.

Diverse kinds of instruments

Given the diversity of learner outcomes arising from non-accredited language, literacy and numeracy programs, the approaches to assessment need to be correspondingly diverse to ensure that the wider benefits of learning are captured. In the practical component of the research, all six instruments were used to some effect in different teaching and learning contexts. However, no single instrument was identified as being the most preferred across all sites. Instead, each of the instruments in different ways had particular qualities and uses, which underlined that these assessment processes are likely to require a range of instruments and usages.

The instruments were most useful when the level of language used to capture the learning was easy for the learners to understand—or at least for the tutors to interpret for them. Moreover, the relative value given to each of the wider benefits will vary at a local level, and, consequently, so will the choice of indicators. A ‘portfolio’ of instruments is therefore likely to be necessary to address diversity in learners’ and tutors’ needs. In other words, greater consideration needs to be given to the processes involved in assessing and acknowledging learner outcomes. The simple adoption of uniform measures is unlikely to prove satisfactory or be appropriate.

How such instruments are used instead of or in conjunction with existing frameworks is a decision for those responsible for the programs, but their administration has the potential to be particularly useful in formative as well as summative learner assessment. External reporting (for example, to management or a funding body) may best be done through the aggregation of individual assessments of outcomes, the nature of this summary depending on the instrument used.

Role of tutors

The instruments trialled were used to identify individual achievements; they also enabled tutors to identify areas of strengths and weaknesses in those achievements, this process informing subsequent learning and assessment activities. The assessment processes found to work best were those that comprised a joint activity between learner and tutor. Moreover, there are potential benefits for both learner and tutor from engagement in processes that extend beyond what is captured by the instrument. However, like the learners, the largely volunteer tutors in these programs have different levels of competence and readiness to assess these kinds of learning outcomes. They possess diverse levels of language and literacy and instructional and assessment competence. So, training tutors to understand the purposes and language of the instruments is likely to be essential.

Conclusion

The importance of establishing assessment measures and approaches commensurate with the diversity of learner needs, motivations and outcomes in non-accredited language, literacy and numeracy programs is a key finding of this study. Allied to this is the value of tutors engaging directly with participants in completing these assessment instruments. This study has shown that it is possible to develop and use instruments to assess and acknowledge the benefits of learning through these programs; the benefits are of the kind that goes beyond the outcomes addressed by the National Reporting System. Indeed, the diversity of outcomes necessitates having access to a range of instruments from which to select, in order to cater not only for the learners’ individual needs, but also for the abilities and preferences of tutors and coordinators.

The findings also identify potential benefits for both learners and tutors from joint engagement in the assessment process. However, tutors may need training to better understand and use the instruments available to them. The instruments also need to be accessible and comprehensible to the learners, particularly those who are second language learners.

While this study has highlighted the diversity of purposes, processes and outcomes from non-accredited community language, literacy and numeracy training, it also draws attention to the potential for developing a national scheme for reporting on the wider outcomes of this type of learning. However, the desirability and practicality of developing and implementing such a reporting scheme warrants further investigation. At the individual level, the development of ‘learner identity’ as part of engaging in the learning process also needs further exploration in non-accredited adult learning generally, because this appears to be a key factor in engaging disengaged and disadvantaged adults in learning and helping them into more active, productive and satisfying lives.

¹ The National Reporting System (NRS) is a mechanism for reporting the outcomes of adult English language, literacy and numeracy provision in the VET system in labour market programs and in the adult and community education (ACE) sector. The National Reporting System takes into account the complexity of language, literacy and numeracy competence, considering not just competence with linguistic and mathematical systems, but also the ability to shape language and numeracy use according to purposes and contexts. The National Reporting System has been recently reviewed. It will be known as the Australian Core Skills Framework and will be released in late 2008.
² The selected instruments were: (i) Changes in confidence; (ii) Learner outcomes: personal, social, economic; (iii) Individual learner profile; (iv) Learner indicators of success; (v) Criteria for achievement; (vi) Skills and wider outcomes. Each of instruments (i) to (v) was based on existing measures and adapted by the participating research partners to suit local needs. Instrument (vi) is a composite instrument developed by the authors from their review of research and the interview responses. The ‘Findings’ section provides further information on each of the selected instruments. A copy of each instrument can be found in appendix 2.