Transcript: Research & The Real World

Interview with Camille Gamboa, Sage

For podcast release Monday, January 8, 2024

KENNEALLY: In laboratories and libraries, ivory towers and coffee shops, scholars of all fields work out innovative answers to pressing problems. The setting for such research, on subjects from economics to the environment, may seem remote or even removed from our lives, yet the impact will eventually reach us directly.

Welcome to CCC’s podcast series. I’m Christopher Kenneally for Velocity of Content. For researchers, knowing where and how their work matters is critical. This information can determine the direction of future investigations and the trajectory of careers.

SAGE Policy Profiles, a new web-based tool from SAGE and Overton, lets researchers uncover and understand the influence their evidence-based research may have on public policy by identifying citations of the work in policy documents, think tank publications, and working papers. Camille Gamboa, associate vice president of corporate communications with SAGE, joins me now to explain how SAGE Policy Profiles works. Welcome to Velocity of Content.

GAMBOA: Thank you. It’s great to be here.

KENNEALLY: Researchers and academics have for decades monitored the impact of their work by following citations of published articles in the scholarly world. Camille Gamboa, does the SAGE Policy Profiles tool now let them calculate a real-world impact?

GAMBOA: Yeah, great question. SAGE Policy Profiles was intended to allow researchers to easily see where their work is being used in the real world and specifically in global policy, and to illustrate and share that work’s impact graphically. Since we’re simply reporting on the trail of citations, there are no calculations involved. That said, the data the tool presents provides a rich narrative and a more complete picture of research impact than citation-based metrics alone.

You may have heard of Goodhart’s law, right – so the idea that once you create a measure, it ceases to become useful, because it distorts behavior in undesirable ways. We aren’t trying to create a new metric to fetishize with the Policy Profiles, but we are absolutely trying to make it easy for researchers, their communities, institutions, and funders to understand more thoroughly the value of their work.

KENNEALLY: Who are the researchers that will find SAGE Policy Profiles most useful, and how can they work with the tool?

GAMBOA: Yeah, I would say that any researcher interested in informing policy through their work would find it useful, even those who just started out in their careers. Of course, early-career researchers may not discover their own work cited in policy yet, but the tool would lead them to resources on the community site Social Science Space, where the tool is housed, to help them get started.

KENNEALLY: And do researchers in the social and behavioral science community view the impact of their work differently than those, say, working in the hard sciences? Why do policy documents matter to them?

GAMBOA: Yeah, I’m not sure that social and behavioral scientists view their impact differently, but what I can say is that these researchers make an outsized impact on the policy world, and the data shows this. In fact, I asked our friends at Overton to pull some data on the topic, and using their own data and the OpenAlex database, they found that when you compare mean scholarly citations per publication to mean policy citations per publication, in the STM disciplines, there are a lot more scholarly citations than there are policy citations, but in political science and psychology, sociology, the difference is pretty small. And then, interestingly, in business and economics, there are actually more policy than academic citations. So if you rely on citation-based metrics to measure the success of research, you’re ignoring another massive area of impact. Of course, that’s doubly the case for social and behavioral science.

KENNEALLY: What kind of reporting is possible with the SAGE Policy Profiles tool, Camille Gamboa, and how might a researcher use that?

GAMBOA: Yeah, so using SAGE Policy Profiles, researchers can export data, PowerPoint slides with their policy mentions, and even – my favorite part – a personalized, shareable dashboard. It has maps, graphs, etc., that they can use to showcase to their communities that their academic work is helping to build evidence-based policy. In early testing, researchers have cited interest in using the results on personal websites, in front of tenure and promotion committees, and in grant proposals. So we’re really excited to see where it goes.

KENNEALLY: Is SAGE collecting data related to the Policy Profiles tool? If so, what will you be looking at?

GAMBOA: Honestly, SAGE is not collecting data at this point except for contact information for those researchers who want to be in the know as the tool’s updated. And let me be clear – I’ve been asked before if we plan to commercialize this product, and I am proud to say that the answer is no. SAGE Policy Profiles was created as a service to the individual researcher and to the community at large, and we plan to keep it that way.

KENNEALLY: If a researcher wants to use the tool, Camille Gamboa, where do they go to find it?

GAMBOA: Yeah, researchers can go to socialsciencespace.com/sagepolicyprofiles, and they’ll get the instructions they need there to get started. It’s fast, it’s easy, and as we said, it’s completely free.

KENNEALLY: SAGE has said that the definition of research impact needs to evolve. The SAGE Policy Profiles tool is one way to boost that effort. Why does all of this matter to SAGE, Camille Gamboa?

GAMBOA: Yeah, so SAGE is a mission-driven company founded on the belief that social science – sorry.

So SAGE is a mission-driven company founded on the belief that science and education can and should improve the world. We’re also an independent company, which means that we can experiment. We can invest long-term in tools like the SAGE Policy Profiles because we believe it is the right thing to do. As I’m sure your listeners agree, currently academia over-relies on one citation-based metric which does little to recognize or incentivize the work of researchers or research institutions to connect their work with the world outside of academia. We hope that tools like the Policy Profiles used in combination with other needed action will disrupt our citation-obsessed culture so that we better recognize the real-world impact of research.

KENNEALLY: Camille Gamboa, associate vice president of corporate communications with SAGE, thanks for speaking with me about SAGE Policy Profiles.

GAMBOA: It’s been great to speak with you.

KENNEALLY: That’s all for now. Our producer is Jeremy Brieske of Burst Marketing. You can subscribe to the program wherever you go for podcasts. You can also find Velocity of Content on YouTube as part of the CCC channel. I’m Christopher Kenneally. Thanks for joining me.