Opinion piece
By Jo Hargreaves, Team Leader, Research Program Management
Intermittent efforts to come up with a universally accepted way forward on assessing the wider-benefits arising from university-based research have taken place in Australia over the last decade. It is currently being discussed again with submissions to the Australian Research Council and Department of Education and Training inquiry into creating a measure of research impact and engagement closing on 24 June 2016.
NCVER recently published its latest study assessing impact against two research themes: apprenticeships and the nature of vocations and competencies required by industry. Our endeavours in this area date back to the 1990’s, with a seminal report highlighting the complexity and practical difficulties involved. Those challenges are well-known and talked about more widely across various platforms and sectors, as are the success stories.
The challenges have not prevented us from continuing on in a pragmatic way. Our model has evolved into a structured case study approach combining bibliometric analysis, various metrics, documentary reviews and interviews with end-users to identify where and how impact and engagement is occurring.
Interestingly, submissions from the Australian Technology Network and the Group of Eight to the Australian Research Council inquiry suggest a case study (or vignette) approach is required. This is coupled with the approach taken by Innovative Research Universities who emphasise the importance of compiling data on end-user involvement[1].
Central to our model is talking to the end-users of research to gather information about how the research has been used and what influence it has had. This is also particularly critical in uncovering citations of the research in ‘grey literature’ such as unpublished government reports or policy papers that do not always cite the evidence used. Our approach also makes it possible to unravel realised impacts from intention and shift the focus from one of research process to sharing the outcomes of research in practical and engaging ways.
Such a model recognises that research can be used either directly or indirectly, and the role of research in the decision-making process may be diffuse but not necessarily less influential. It is not perfect, but assessing impact is an imperfect science. Our philosophy has been that it is better to get on with having a good attempt rather than trying to come up with a picture-perfect solution to challenges that have to be navigated rather than solved.
The findings from our most recent study demonstrate the diversity in the use of research as well as end-users identifying various impacts on policies designed to address apprentice completion and commencement rates, the way in which pre-apprenticeship programs are structured, industry training strategies and mentoring support and pastoral care strategies for apprentices as well as professional development programs for VET practitioners.
Our impact studies have also highlighted the importance of NCVER’s efforts across:
- how research projects are selected and framed
- defining the problem and ensuring stakeholders are involved
- rigour in research design and methodologies
- sophisticated stakeholder engagement and knowledge translation strategies, which are required to assist in knowledge transfer and uptake
We will continue to support our researchers in their impact and engagement efforts, as well as enhance our model for assessing impact along the way.