How can we rate and measure knowledge valorisation?

Using Altmetrics to measure the impact of grey literature.

02/13/2019 | 12:14 PM

Knowledge valorisation is one of the three core tasks of VU Amsterdam, but I notice many of my colleagues here are struggling with what we actually mean by valorisation and what we want to achieve with it’, observes Jutka Halberstadt, assistant professor in childhood obesity at the Health Sciences Department. Jutka Halbertstadt-klein 'To me, societal relevance is a vital aspect of my university work, which is what started me on a quest to answer these questions and define how my own activities fit with VU Amsterdam.'

National health care standard
Among those activities is nationwide project management of the VU project Care for Obesity. Her team's work field takes place at the intersection of research, policy and practice, which has yielded various products of practical use for the professional field. 'One of them is a national health care standard that we developed in conjunction with organisations of health care professionals, health care insurers and patients. This document has been the accepted foundation for the diagnosis and treatment of childhood and adult obesity since 2010. Without VU Amsterdam, this document would not have come about.'

Building a better society
'To get a sense of the significance of valorisation for VU Amsterdam and what we actually understand it to mean, I went around to talk with colleagues all over the university, from Finance to the Administrative Office and from the University Library to Health Sciences. It became clear to me that this is something very important to almost everyone, including our new Executive Board president. Also, I found out that it's not about money, but about using our research and education to help build a better society.'

Rating knowledge valorisation
'The question then is, how do you rate knowledge valorisation? And there's the rub, because the university assesses on the basis of research output, citation scores and teaching; valorisation activities are a nice bonus, but they won't help you get ahead in science. To me, it's not the impact factor that matters, but rather what benefits society.'

Showcasing and measuring societal impact
'I got together with Maurice Vanderfeesten and Joeri Both from the University Library to think about ways to make the societal contributions of individual scientists more visible and measurable. If we can measure that, it could also provide input for assessments and to showcase the societal impact of VU Amsterdam as a whole. If I want to gauge societal interest in my scientific publications, I can use Altmetrics to look at the grey literature (policy documents), news media and social media, but a product like our health care standard doesn't get picked up by PubMed, and therefore not by Altmetrics  either, even though the societal impact is immense.'

Author name
'Maurice agreed to take on the task of making the impact of these kinds of valorisation products quantifiable as well. We've already come up against some technical hurdles that are teaching us more about what's needed to be able to measure that impact. To give an example, the health care standard wasn't published with the name of a person as the author, but on behalf of the partnership of involved organisations. For findability, it's important to have author and journal names. Maurice’s research   will give us pointers on how to measure valorisation effectively in future.'

Ideas on rating valorisation?
'I would like to get a conversation going with my colleagues at VU Amsterdam about the significance of valorisation and ways we can measure and evaluate it. In my research group, for instance, we issue recommendations to the ministry of Health, Welfare and Sport and collaborate with municipalities. Besides peer reviewed articles, we develop usable products for the Dutch field of practice like manuals, blogs and workshops. I myself am active on LinkedIn, regularly give interviews and co-author with Prof. Jaap Seidell popular science books and an article for the Het Parool newspaper once a month. We've also just launched a website and set up a five-member coalition, including the RIVM (National Institute for Public Health and the Environment), to generate awareness at the municipal level about tackling childhood overweight and obesity with a care chain approach. And, last but not least, we publish Open Access as much as possible. So how should each of these things be valued? I would love to hear what others think!'

Want to learn more about measuring the impact of grey literature?

Read: 4 tips on making grey literature citable and traceable

VU Library Live on rating academic researchers
In March 2019 the University Library is organising another VU Library Live talk show, this time spotlighting new approaches to rating and remunerating academic researchers, also extending to Open Science and societal impact. In the year ahead, the VSNU (Association of Universities in the Netherlands), NWO (Netherlands Organisation for Scientific Research), NFU (Dutch Federation of University Medical Centres) and ZonMw (Netherlands Organisation for Health Research and Development) want to join forces to advance this issue in a dialogue with the university field.

Among the talk show guests this time will be NWO President Stan Gielen and Frank Miedema, dean of the Faculty of Medicine at Utrecht University, vice-president of the UMC Executive Board and active in UU's Open Science Programme.