New Masters Thesis: A Portfolio of indicators [...]

Last week Birger Larsen’ MSc student, Danielle Chiosso Liu, defended her thesis entitled ‘A Portfolio of indicators: Broadening the scope of bibliometric research evaluation of the humanities – a mixed methods exploration.’ The thesis work was begun last year and inspired by the initial ACUMEN project proposal.

Abstract
Funding allocation schemes are increasingly incorporating bibliometric indicators into evaluation of research. Evaluative bibliometrics have proven to be especially problematic when applied to the humanities. The paper sets out to determine whether or not a portfolio of indicators can better represent humanities scholarship quantitatively, compared to the standard productivity/citation analyses often used in research evaluation practices. A literature survey, qualitative interviews, and bibliometric methods are combined in a mixed methods approach. The proposed portfolio extends source data and includes unconventional indicators of more informal aspects of scholarly communication. The indicators are tested on a small sample, taking demographic variables into consideration. This study finds that while a portfolio of indicators has potential, it is still a ways away from being viable in an evaluative capacity. Technical and theoretical developments are needed. Interview results show an overall ambivalence towards evaluative bibliometrics in research evaluation; scholars are wary of normalizing effects any quantitative analysis might have when applied to evaluation. A pragmatic mixed methods approach like the one used in this paper is recommended for future studies in issues of research evaluation, making sure to take into account the individual perspective of humanities scholars.

Birger uploaded the full text to SURFgroepen, here.