[This post was originally published on SciDev.Net and has been reproduced (with some modifications) with permission and in accordance with the Creative Commons license used by the source website.] Sustainability science is becoming increasingly collaborative and global, a recent report found, but researchers say cooperation between scientists from the North and South remains scarce and unbalanced. Publication and citation data show that […]
Almost everybody uses some social networking platform, e.g., Twitter, LinkedIn, ResearchGate, Mendeley, in some way or other today. More and more people are talking about how social media interactions can be advantageous to researchers in several ways. In this interview, I talk to a researcher who is sultry about studying the online interactions of academics on social media and about understanding the potential of altmetrics to capture research influence. Meet Dr. Kim Holmberg, a postdoctoral researcher at the Research Unit for the Sociology of Education (RUSE) at the University of Turku. Dr. Kim Holmberg is also Honorary Research Fellow at the Statistical Cybermetrics Research Group at the University of Wolverhampton and an Associate Professor at the Department of Information Studies at Abo Akademi University. His research involves altmetrics, webometrics, scientometrics, Web Two.0, Library Two.0, and virtual worlds in education. He also uses quantitative methods to explore various phenomena on the web in general and in social media in particular. Dr. Holmberg has authored several journal articles, book chapters, and conference proceedings. He has also published books on topics close to his heart – altmetrics and information exchange in the social media age.
In this interview, Dr. Holmberg talks about how social media platforms play a pivotal role in facilitating useful networks and opportunities for academics. The highlight of this conversation is Dr. Holmberg’s explanation of how the scholarly community is at a nascent stage of understanding and exploring altmetrics and how it is too early to assume that altmetrics are definitive indicators of the societal influence of research. Dr. Holmberg also talks about the main problems with citation-based indicators and mentions some rookie mistakes researchers tend to make when using social media.
Your research interests are fairly fascinating. Could you tell us more about each of them?
My research interests are all related to two things; the web and quantitative research methods. I could basically say that I’m keen on understanding a diversity of different online phenomena that can be investigated using mainly quantitative methods. The choice of quantitative methods is fairly natural due to the enormous volumes of data that the research and publishing community is producing and sharing online. More specifically, I’m presently interested in altmetrics, that is investigating online scholarly communication and how research output (articles, data) is being discussed, collective, and mentioned online and what that could perhaps tell us about the influence of that research or the attention it may have received from a broader audience beyond academia.
Based on your observations, what are the most distinct characteristics of the way in which researchers use Twitter?
I’d say that there aren’t any distinct characteristics in how researchers use Twitter. Some use Twitter professionally while others use it more personally, but most likely, most use it in a way for both professional and private reasons. This duality also makes it difficult to interpret anything from how researchers use Twitter.
Are there any disciplinary differences in researchers’ use of social media?
Yes, in our paper Disciplinary differences in Twitter scholarly communication, we found some evidence on how researchers from certain disciplines use Twitter more for scholarly communication than others. Even tho’ our analysis did not include all disciplines, our results displayed clear differences in researchers’ use of Twitter across disciplines. We also took this to suggest that even altmetrics on Twitter should be field-normalized. [Holmberg, K. & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, vol. 101, no. Two, pp. 1027-1042. DOI:Ten.1007/s11192-014-1229-3.]
What role do social media play in facilitating and/or enabling collaborative research today?
Social media has an significant role in both facilitating and enabling collaborative research. Through academic social networks or social reference managers researchers can find other researchers with similar research interests or the algorithms these platforms use can point users to articles that are similar to those they have authored, thus connecting researchers with each other. Social media can also enable collaboration, for example, through open discussions on various social media platforms or collaborative authoring.
What are some common rookie mistakes researchers make during online interactions on social media platforms?
A good chunk of advice for any researcher using social media platforms is to reminisce that these platforms (most of them by default) are open to everybody, so anyone can see what you write or share. Albeit you can create closed groups or chose which of your connections sees what you write on some platforms, it might be difficult for you to permanently keep track of the selected groups you’re sharing your information with. For this reason, I keep my Facebook profile downright open to public viewing. When I know it’s open and never even attempt to share anything with a closed group, I always keep in mind that anyone can read what I write. This helps me choose what I am telling and ensure that I am communicating what I want to. This also increases the effectiveness of that platform for my purposes.
Most discussions about social media usage concentrate on researchers. How can other members of the scholarly publishing community use social media to their advantage, e.g., academic institutions, libraries, funders, journals, publishers, etc.?
I do think a lot of these already use social media very innovatively. Libraries for example have been at the forefront of using social media. In fact, the Library Two.0 phenomenon that came to be widely discussed a few years ago was just about that – about how libraries use social media to connect with their patrons and to promote their services. And that was even before the concept of altmetrics was introduced. Libraries can use social media platforms to promote academic work (articles and other types of academic output), to share information about academic events (academic conferences, guest lectures, public talks, seminars, etc.), and to interact with their patrons, customers, audiences, or stakeholders.
What are some of the current altmetric indicators used to evaluate research influence? How effective are they?
I’d say that no altmetric indicators should yet be used to evaluate research influence because I feel that we’re not yet ready for it. Our understanding of alternative metrics as research influence indicators is limited, even tho’ a lot of research work on this topic is being undertaken in institutions in Montreal in Canada, Leiden in The Netherlands, Wolverhampton in the UK, Bloomington in the US, and Turku in Finland. We still need to resolve a lot of issues with data quality and standards of influence measurement. There are questions about normalizations of the altmetrics data. And to fully understand what the altmetrics (e.g., mentions of a research article on Twitter, Reddit, or Facebook) reflect we need to devise a reliable system to understand more about the people who generated the altmetric event through their interactions with the research output (e.g., sharing an article or mentioning it in a tweet) and in what context they have collective it. I think that, presently, altmetrics are best used for showcasing specific research that has received a lot of attention online. Altmetrics can also point to research that has received a lot of attention outside academia but that doesn’t typically show up on the radar when using citations only to assess research influence. And altmetrics can be used to create narratives around specific research, i.e., how the research has been collective and discussed and by whom. But more research needs to be done before we can talk about reliable altmetric indicators or before we can use them for research evaluation.
Today, altmetrics are becoming increasingly popular. Yet, several researchers are hesitant to use them. Based on your observations, what are some of the common misconceptions researchers have about using altmetrics to measure research influence? How can these be dealt with?
I think one problem is that “altmetrics” as a term includes a very broad multiplicity of different platforms and sources where we can trace and collect mentions of research products. Some of these are promising sources for research evaluation purposes, while others should very likely not be used at all. It is significant to know the distinction inbetween these different platforms and sources. It is also significant to understand that while Twitter is a big data source for altmetric events, it is not the only source and certainly not the most reliable source.
What are some of the problems with citation based indicators? Conversely, what are some of the weaknesses of alternative indicators?
There are basically three well-known major problems with citation-based indicators. The very first problem is that they concentrate only on research articles. In today’s digital age, research products can take many forms, from datasets to presentations and computer code to research articles. All of these research products should be considered as contributions to science, yet only research articles are considered as “important”. Also, there are excellent disciplinary differences in publishing and citation traditions, because in some disciplines publishing in books or conference proceedings is more frequent than in scientific journals.
The 2nd problem has to do with time. The actual act of scholarly communication can span over a long period – from formulating a research idea to creating a formally published research article that can be cited by others. When using citations to evaluate research influence we can either wait for a few years and wait for citations to accumulate or we can judge an article based on in which journal it has been published in. The very first option is thus an evaluation of past success, while the 2nd is a tentative prediction of the future potential influence of the article.
The third problem with citation-based evaluation is that citations are created only by other researchers, thus displaying how the scientific community has used the published research. This then gives an indication only of the scientific influence of the research. At the same time, there’s enlargening request to showcase broader societal influence of research, such as economic influence, influence on health, cultural influence, educational influence, or influence on the environment.
The weaknesses of alternative indicators or altmetrics presently are mainly connected to data quality. This has been worked on, for example by NISO (National Information Standards Organization), but there’s still a lot of work to be done in this area. Another weakness is that we don’t always know the context in which the altmetric event has been created, i.e., whether is it an endorsement of the research or a criticism of it.
What is the best treatment to measure research influence?
Peer review. Only through peer review can the influence of specific research be evaluated. However, peer review cannot be performed at a very large scale, or perhaps it can, but that would be very expensive! Therefore, we need other methods as well. Citation-based indicators can help and support peer-review, while altmetrics (presently) can highlight interesting research or research that is worth more attention by the reviewers.
Which online platforms do you think are more effective for researchers – broad platforms like Twitter or researcher-specific forums like ResearchGate?
This depends on what you mean by “effective.” Effective in reaching a broad audience? Then I would say Twitter. Effective in enabling you to share your own work with other researchers or to get in touch with other researchers working on similar projects? Then I would vote for an academic social networking site or social reference manager. So it all depends on what you would want to accomplish, and as we have seen, researchers use social media for a broad multiplicity of different purposes.
How do you see the relationship inbetween social media and the research community evolving over the coming years?
Social media has come to stay and we may as well accept it. But nothing will remain constant. While certain platforms may be discontinued, others will take their place. In the near future, I see more and more researchers beginning to use social media platforms (for different purposes) as they see the benefits these interactions suggest.
Thank you, Dr. Holmberg, for sharing such interesting perspectives on this freshly emerging topic of discussion in scholarly communication circles!