Transcript: Confidence In Research

Interview with Anne Kitson, Sr. Vice President and Managing Director, The Lancet and Cell Press

For podcast release Monday, August 29, 2022

KENNEALLY: As a result of the COVID-19 pandemic, the public and governments have focused increased attention on scientific research, hoping to find not only solutions, but also certainty. Yet science advances through trial and error. That conflict of expectations is understandable and ultimately may be unresolvable. A broad coalition hopes to address the challenge with confidence-building activities, beginning with the researcher community.

Welcome to Copyright Clearance Center’s podcast series. I’m Christopher Kenneally for Velocity of Content.

In July, Elsevier launched a global collaboration to understand the impact of the pandemic on confidence in research and to learn how researchers may better maneuver in a rapidly changing scientific landscape.

At Elsevier, Anne Kitson is senior vice president and managing director of The Lancet and Cell Press. She leads the Confidence in Research project and joins me from London. Welcome to the program, Anne.

KITSON: Thanks for having me, Chris, and hello to your listeners.

KENNEALLY: Well, it’s a pleasure to have you join us, Anne Kitson. You represent two of the world’s most important and widely respected scientific journals. How serious is this crisis of confidence in scholarly publishing and research generally?

KITSON: First and foremost, I just want to reiterate your introduction – my remarks about the Confidence in Research initiative is that the scope is specifically assessing researchers’ perspective on their own communities with regard to process and output, the challenges, and possible solutions as a consequence of their experience during the pandemic. Also to mention that this is very early days, so my comments will focus around themes observed so far. But these will be subject to evaluation and scrutiny by the other stakeholders and their experts involved – the broader coalition, if you like, that you referred to.

The main opportunity here is that the pandemic and our experiences of the last two years are a chance to reinvigorate discussions, how the various players associated with the research environment can really drive confidence in research, and what, of course, those drivers are, or indeed reaffirming them.

But to your question, I believe labeling it as a crisis is overstating this, but there are some growing strong challenges it’s incumbent on all of us to address together. We collaborate already, but we need to continue to do so and even more now in this very rapidly evolving landscape.

So what happened in the past two years is firstly the speed of scientific endeavor and progress, if you will, increased tremendously as policymakers, scientists, and the industry collaborated on a scale like never before. That led to very effective treatments being developed and regulated for use in a fraction of the time. However, that said, we’ve also seen an exacerbation and an acceleration of trends that have been known for many years. The pandemic very much shone a spotlight on science and its practice, and public engagement is now higher than ever before, and some concerns were raised.

Some of the concerns – regional differences. So in health, deepening inequity in voices being heard, high for rich countries, but poorer countries not having a platform and suffering even more in the pandemic. Inequality between disciplines, some being badly served and under supported. And the nature of publication and collaboration as it evolves is becoming increasingly challenging for researchers to judge the credibility and the trustworthiness of research. We observed that this is leading some to report good practice in conducting and communicating research. It’s also prompting concerns and affecting how researchers perceive their role in society.

And if we talk about reproducibility first, the point you made at the start that the process of science is essentially messy by its nature – researchers sometimes get things wrong, but they like that. It helps them with their investigations. Continuous self-correction is an integral part of scientific progression. However, at times, science didn’t do a good job around explaining changes through the course of the pandemic. So we do need to be self-critical.

I’d also like to say that science is very complex and difficult, and work at the cutting edge often involves methods that are very sensitive to small changes. We know that subsequent research studies often replicate some or part of the previous research to build on the work of others so that the body of accepted knowledge grows. Having said this, we are aware that when such cutting-edge research is translated in, say, pharma or engineering, some of the results may not be replicable and that funders and governments do wish to address these points.

Other areas for attention – clear approaches to the use of associated research data is a sensitive topic with respect to clinical trials, of course. Better annotation of research data would also help. But is that a lack of confidence or just reality?

I’d also like to make a comment that reporting is much more transparent, more than ever before, but because science is complex and fast-moving, openness and transparency to support the surety and the research integrity becomes ever more critical.

The last point I’d like to make about undermining confidence is the issue of paper mills. There’s a new study, actually, by COPE and STM of data submitted from a variety of investigations by leading publishers to get a sense of the scale of the problem, and it basically concludes that the submission of suspected fake research and also associated with fake authorship is growing and threatens to overwhelm the editorial process of a significant number of journals. So there’s deep concern amongst a range of stakeholders, including publishers, research investigators, and others, and a realization that all stakeholders need to work together across the scholarly communication process to find solutions, and of course, those predatory journals and the misuse of AI.

Sorry, Chris. Long answer. (laughter)

KENNEALLY: Well, there’s a lot to discuss here, and you’ve identified several areas where there are gaps in credibility. So how will the Confidence in Research project begin to close those gaps?

KITSON: The Confidence in Research collaboration includes a field study conducted by one of our partners, The Economist Impact, and the idea in a nutshell is that it’s going to culminate in an expansive report to be released this autumn. Just to go into some of the aims of the study, if you will, we want to evaluate researchers’ perceptions of the impact of the pandemic on the production and communication of research, assess whether and how researchers have changed their behavior as a result of the pandemic, identify gaps in researchers’ confidence, and identify interventions that might help them effectively bolster both scientific practice and confidence in research.

But we’re not just doing the study. We’ve also got some of the core elements. So we have a global advisory board who inform the design of the study. We’ve done expert interviews, in-depth interviews with the research community. We’re doing in-country roundtables. And we’ve got a global expert panel.

Just a very small point – we started back in March. That’s when we convened the global advisory board to inform the study, which we finalized in April, and then we deployed it into the field. And we’ve been collecting quantitative and qualitative data, and we got some interim findings surfacing the core themes in July. We’re also partnering, like I said, with regional organizations around the world to ensure that we capture the perspectives as we discuss the survey results to really get the rich context there.

This month, we’re in the midst of roundtables. We did one in the US about 10 days ago. And we’re beginning to develop collaboration outputs from these. Also, in September, we’ll finish these roundtables. We’re having another global expert panel in mid-September, more regional convening partners meetings, to basically continue building those outputs. And then finally in October, we hope to basically share those outputs, and we see it very much serving as a jumping-off point to work on possible interventions with the coalition of stakeholders involved.

Why are we doing this? I think it’s fair to say Elsevier has got quite a long history, actually, of working with researchers, and our aspiration and our hope is basically that we can put forward meaningful actual commitments and recommendations to support researchers. Of course, we have to do this with our stakeholders. We can’t do this alone.

KENNEALLY: So this is a global effort, Anne Kitson. Tell us about the partners and those you’re working with. Who are the advisors and the experts who are going to help you form the questions you hope to answer?

KITSON: So our global advisory board consists of science and research NGOs and think tanks, researchers in research-intensive universities, policymaker organizations, editors – as in editors of academic journals – associations, and funders. We’re trying to be as inclusive as possible. We’ve got a representative from South Africa and Rwanda as well as the Western countries, and we’ve got a representative from the WHO.

And then in terms of the convening partners, these regional partners, we’re working with Research!America. We’re working with Sense about Science, the Körber Foundation in Germany, and the Chinese Association for Science and Technology Policy in China and a couple of others.

KENNEALLY: And a first step is this ambitious survey to identify researchers’ views about the impact of all this increased public attention on science. So what kinds of pressures are researchers feeling today to be more responsive to the public?

KITSON: I think one of the things that they are talking about is that while researchers seem to feel confident in talking about the design of their experiments and the methodology, the challenge some of them are mentioning – it’s not just how to communicate about the science itself, but how to position their research, so reporting in context to help understanding the nature of the work being done – for instance, where they place their level of confidence, especially when we’re seeing research reported at very early stages. Why are they confident? How do they explain preliminary findings? How do they fit with the bigger picture and about the future research directions? So researchers are asking for more help on this. It seems like it’s not about dialing up the communication but getting support with explaining the complexity through to the public, doing that engagement.

Another one of the things that might help there is markers of confidence to be agreed, perhaps, on research output, i.e. papers. And it takes us back again to that whole reproducibility topic. I just want to share one very small example, if I may, which is about STAR methods, something that is close to my heart. We developed it at Cell Press some years ago. This is about supporting authors to produce structured, transparent, accessible reporting, with the intention to promote rigor and robustness with an intuitive and consistent methodological framework that integrates with the scientific workflow.

There’s many, many more initiatives that others are doing – FAIR data, for instance, which is already in play, and which we support, of course. The challenge with the coalition is how to further promote and develop these types of tools and initiatives, and most important, how do we encourage and foster adoption by researchers?

KENNEALLY: So in what way, Anne Kitson, do you think all this attention, public and professional, on research and researchers may change the conduct of science?

KITSON: I think it’s transparency, transparency, transparency. I do think we have a moment, an opportunity, as I said at the start to really garner all the players involved to double down on this. I think there’s a real hunger and will that we can harness there. What we need to do is we need to aim for consistency and cooperation across the board. What we saw in the pandemic, of course, was that the virus does not respect country boundaries.

I also think there’s a point about research culture and incentivizing the communities, particularly early-career researchers, to adopt best practice, to make their research reporting transparent and as accessible as possible. There’s an educative part, if you will, to this, too, with publishers and funders, who should and do play a part. So it basically boils down again to having a renewed collective commitment to focus, to work on this together.

I think there’s just one point that I forgot to mention earlier. There absolutely has been lots of training, for example, around scientific communication for many years. This isn’t new. But it’s going back to that nuance of making sure that we talk about the research, the outputs, in context, and also really publish those markers and almost try and codify that and encourage usage, so that we can all have an increased level of confidence when we’re assessing a particular study.

KENNEALLY: The survey of researchers is an important first step in all of this, Anne Kitson. What kind of activities do you expect to follow it, both online and in person?

KITSON: I think there’s a variety of things. I think there’s going to be many more – I think there’ll be many more tools developed. I think there’ll be a lot of investment in tools. I think the funders are going to be perhaps giving accreditation for use of these things to ensure that researchers feel their recognized.

One of the things that’s been really undervalued as well and came out in the survey is teaching. And what the researchers are saying is there needs to be much more value placed to teaching to support those researchers and what they’re doing. In terms of in person/online, there are conferences – in fact, this September, there is a peer review conference – and I think we will see a growing enthusiasm for those forums to actually make sure that they are driving and working with publishers and funders to try and get traction with those tools. They are there now, but I think they will get actually even stronger. What we heard at the beginning was, for example, peer review and journals are even more critical as part of that credibility story, if you will, for that confidence in research.

KENNEALLY: Anne Kitson, senior vice president and managing director of The Lancet and Cell Press, thank you for telling me about Elsevier’s Confidence in Research collaboration. Good luck with your work.

KITSON: Thanks, Chris, for the opportunity to share with everybody. Thank you very much.

KENNEALLY: That’s all for now. Our producer is Jeremy Brieske of Burst Marketing. You can subscribe to the program wherever you go for podcasts, and please do follow us on Twitter and on Facebook. I’m Christopher Kenneally. Thanks for joining me on Velocity of Content from CCC.