Transcript: Make AI Your Writing Tutor

Interview with Avi Staiman

For podcast release Monday, June 12, 2023

KENNEALLY: In scholarly publishing, researcher and author are often taken as synonymous terms. Writing, however, comes only at the end of a lengthy process of investigation and discovery in a lab or in the field. Many scientists and scholars consider writing drudgery – a necessary evil in the pursuit of tenure and research grants. What if a machine could lighten the burden – or if smart enough, could even do the work entirely?

Welcome to CCC’s podcast series. I’m Christopher Kenneally for Velocity of Content. Such sophisticated writing machines now exist, of course. ChatGPT from OpenAI is the most famous of a growing number of generative AI tools that create narrative responses based on textual input from large language models and that have the appearance of vast scientific knowledge. Publishers and universities alike have expressed concerns that these chatbots facilitate plagiarism and raise questions about the nature of authorship.

Avi Staiman is the founder and CEO of Academic Language Experts, a company dedicated to assisting academic scholars to elevate their research for publication and bring it to the world. While he recognizes the potential fraud and confusion that chatbots can introduce into publishing, Staiman asserts that publishers ought to encourage researchers to use all tools at their disposal in order to make their work as accessible and impactful as possible, especially researchers who are non-native English speakers. Avi Staiman, welcome to Velocity of Content.

STAIMAN: Thanks so much, Chris. It’s a real pleasure to be here. I’m excited to have this chat.

KENNEALLY: It’s an important discussion, and it’s a discussion we’ll be having a lot of on this podcast and throughout the world when it comes to ChatGPT. It’s the hot subject of the year for publishing, for technology. And on March 31st, in a post for The Scholarly Kitchen blog, you wrote that the proliferation of powerful AI tools pushes us to ask fundamental questions about how we perceive the role of scientists in general and the specific role writing plays in their work. In other words, you asked to what degree should we even care that authors write every word of their research in the first place? It’s a provocative question, Avi. So why or why not should anyone care that authors write every word of their research in the first place?

STAIMAN: I think the reason this is a really fascinating question is because it was always a theoretical one, right? Because who else is going to write for you if not yourself? Obviously, over time, people have used the help of scientific writers. That’s nothing new. But the concept that something nonhuman could actually write a text for you is something that I don’t think before November 2022, most folks would think was even a possibility. Now that it is, we are really asking ourselves questions about the basic role of a researcher.

I want to just kind of step into the shoes for a moment of your average taxpayer who wants to make sure that their money is going towards the advance of scientific research and governments that have priorities in terms of where their resources are going. And I want to imagine a researcher who’s working on the COVID vaccine in the months leading up before the pandemic, and I want to ask ourselves, to what degree do we think that that individual writing every single word is really critical in light of other activities or important – whether managerial or scientific activities that they could be doing at that same moment?

So I will stress that, yes, they very much have to take responsibility. We need to make a very clear distinction between folks who are just going to kind of generate whatever comes out and then try to use that to publish. That’s not what I’m talking about. Obviously, I’m very against that. But what I’m for is having a rethink about what the role of a researcher is, keeping in mind all of the myriad of responsibilities they have as well. We’re talking about teaching, advising, researching, reviewing, writing, publishing, speaking engagements. There’s a lot on researchers’ shoulders. And I think we need to ask the question of, well, as a society, what do we want them doing, and where does writing rank up there if we can have tools that can help at least shorten that process?

KENNEALLY: Remind us first, though, Avi Staiman, of the basis for the naysayers’ arguments, specifically that ChatGPT facilitates plagiarism and raises questions about the nature of authorship.

STAIMAN: I would say there are a few main sort of critiques of GPT as a tool and also the use of GPT. First and foremost is that the information is not reliable when you’re using it as a generative tool – and it’s known as generative AI, so that’s what people are using it for. The problem is when we think of it the same way we think of Wikipedia, that’s where we get into trouble, because it is not sourced – although this week, they came out with a WebGPT where you can have some sources. But even so, it’s not necessarily information that is verified. In fact, it’s simply predicting words in a certain sequence or order. That can lead to all sorts of erroneous comments that are made as if they are truisms. So that’s issue number one.

Issue number two that I would bring to the fore is the question – which I don’t think any of us know – of where is GPT and where is OpenAI sourcing their materials from? We know that they’re scraping the internet, but are they giving certain weights to certain websites over other ones? Is Reddit given the same credit as, I don’t know, Elsevier and Wiley? We don’t know. There was a very interesting Washington Post article trying to reverse-engineer and figure out kind of what’s in the black box and how to get to it that I recommend folks looking at. But it’s unclear exactly whether or not the sources that are relied upon should be relied upon – what exactly they are and whether they were given permission. Clearly, no one was asking these sources whether or not they allow their materials to be used.

And the third issue that I would want to bring up is the issue – and I think this is where it’s a little bit more of a gray area, and brings us full circle to the conversation we had before – to what extent do we say a student who goes in and is using it in order to create new materials that it’s not their words, but we’re passing it off as our words – is that an issue? Is that a problem? And to what extent do we see this as an extension of human thinking – and we’re doing prompting, and maybe we’re reviewing and taking responsibility – or to what extent do we say, no, we have a certain expectation of you as a person that when you’ve written something, it means that you’ve written it and not someone else? Those, I would say, are kind of three of the main critiques of GPT.

KENNEALLY: At the Council of Science Editors annual meeting in Toronto in early May, Avi Staiman, you outlined some of the responsible and productive uses of AI tools in research. Tell us about those.

STAIMAN: You mentioned at the outset that I have been kind of pegged as a proponent of trying to see the good, and I think the reason that I’ve taken that approach is simply because researchers are already doing this. They are already using these tools. I remember back to my childhood days when I was a camp counselor, we had a rule, never make a rule that you can’t enforce. That’s my fear, and that was one of the reasons that drove me to write this piece in The Scholarly Kitchen, is if we try – and there have been publishers that have tried to simply outright ban GPT – for example, Science. I understand where they’re coming from, but they basically said you cannot use it in any shape, form, or size, and you’ll be disciplined if you do. I think the issue with that is that it would be almost the equivalent – and will quickly become almost the equivalent – of banning Google. I don’t think any of us are a proponent of trying to ban Google as part of our research process.

I think to turn it on its head and think about, well, what are the possible productive uses, the first and foremost thing that I always emphasize is let’s go back to what GPT is. It’s a large language model, LLM for short. And I like focusing on the second word, language. This is something that I learned from another CCC interview which has really kind of driven my thinking, which is that if we want to use it productively, we should think about its linguistic capabilities and not its informational capabilities or lack of capabilities, because that’s exactly what it is. I like to call it Wordle on steroids. It knows how to predict the next word in a sentence or the next string of words on a very high level. And in that way, it’s using the totality of human language in order to enrich and in order to convey ideas.

So I think it’s a tremendous tool for researchers for whom English is not a native language, which is the vast majority of researchers in the world, who struggle to publish alongside and really get an equal shot at publishing their research alongside their Anglo colleagues, because they don’t have the mode of expression. Anyone who’s ever read a complex academic paper who is a native English speaker – if it’s not your field, you say, I don’t know what the heck is going on here, and I know this language, right? Now, think about the experience for that researcher for whom that’s not their native language, and all of a sudden, there’s the great equalizer. There’s this tool that at quite a cheap price point can actually help your author in Nigeria or your author in really low-income countries be able to express themselves at a high level – again, to make a clear distinction and differentiation from using it as an information source, which I think is severely problematic.

So there are a lot of use cases that I discussed in Toronto and I’ve been discussing to different people. But I think first and foremost, the most important thing is to think about it as language. By the way, that also explains why it’s a good summarizer – because it knows how to take a long text, and it’s giving us an output of I’m feeding you back what you gave me in a form that can be easily digested. I think that’s the power of it. The power is not in creating from scratch. The power is taking our data, taking our information, taking what we have, what we know to be true, or what we know to be reviewed and verified, and then improving on it, tweaking it, changing its form in constructive ways.

You can take an article, and in one GPT prompt, you can turn it into a social media post. You can turn it into a policy paper that you might want to present to a local legislator who’s not going to speak the academic jargon. And at the same time, you can also write a press release that might get picked up by one of the big media news outlets. In my eyes, so long as the prompter – or the researcher, in this case – is taking responsibility over reviewing that material, it has tremendous potential for good.

KENNEALLY: In your work at Academic Language Experts, Avi Staiman, you come in contact with academic scholars around the world. Have they told you that they’re excited about the possibility of ChatGPT?

STAIMAN: Yeah, that’s a really good question, Chris. We’re actually running a survey right now among our researchers to kind of feel their temperature and gauge what their experience is. It’s been a mixed bag, I would say. There is some excitement about potential opportunities, but I think if there’s anything that scientists value, it’s quality of information and veracity of information, and I think that because people started using it as a Wikipedia of sorts, which was problematic, they sort of threw the baby out with the bathwater and said, ooh, I don’t want to touch this. Or a concern that when they saw in the news that – I think it was IBM employees who put some information into GPT, and then it came out in a way that it wasn’t supposed to and became public. So if you’re a researcher and you’re working on highly sensitive and confidential material, well, you’re going to think twice before you’re going to start sharing that with an OpenAI tool.

It has been definitely mixed, and I think there’s hesitation. And I think there should be hesitation. I think that’s a positive thing. I don’t think that everyone should start uploading all their private information – at least to the public GPT servers. On the other hand, I’ve also seen researchers who have been talking about how it’s helped them when they’ve hit writer’s block. Or when they’re trying to come up with counterarguments to their argument, it’s been really effective. I’ve seen use cases where people will use it when they’re trying to come up with a new and novel idea for their research grant, and they’re kind of stuck, and they just say, give me 10 ideas for how I can take my research and what hasn’t been done yet and what I can do. It just gets their juices flowing and head thinking. And in that way, it’s been a real positive.

So it’s really a fascinating – I think we’re at kind of a really interesting point, because it’s really split, and I think that many people themselves are split internally about how they want to kind of relate to it.

KENNEALLY: You’ve already raised the point that so many academic scholars today do not have English as their first language, but they need to communicate and need to publish in English. Your work with your company is to facilitate all of that for them. That really is an important point that we shouldn’t overlook, right – that these tools, not just ChatGPT but other AI tools, can help bring us a greater diversity of researchers who are contributing to journals.

STAIMAN: Yeah, I think that if you look at the publishing landscape, especially among the top-tier journals, it’s still very heavily dominated by American researchers. And it makes sense, right? Some of the top research institutes in the world are in the States, and they’ve worked hard to achieve that, and there’s some really great research going on there. So I don’t want to downplay that. Obviously, China has quickly become a very dominant world force in terms of scientific research and publishing as well. But I do think this is an opportunity to really level the playing field and help some of those researchers express themselves and convey their work in a serious manner. This doesn’t make up for differential resource issues or the fact that they might not have the same – many researchers don’t have the same infrastructure as researchers in the US to the UK or Canada. So it’s not a solve-all for all inequalities in research, but it at least can serve to bridge the gap.

That being said, education around proper use of these tools and best practices becomes so crucial and so critical. One of the reasons I wrote that post on Scholarly Kitchen was because I felt like the publishers hadn’t taken an active enough stance on what people should and should not be using it for. They basically got very excited about this one topic, which was can AI be considered an author – which in theory is a very interesting topic. In practice, I’ve never met a researcher who wants to share credit for an article. So I didn’t really understand why that was a big deal, because I don’t think that most authors, even if they did use GPT, were going to credit GPT.

And I wanted to kind of push the ball forward and say, well, actually, let’s think critically about what proper use and improper use is. I purposely used that term proper and improper, but not necessarily what’s allowed and what’s forbidden, because I think we need to be careful about not making rules that we need to then heavily police or follow up with. But rather, let’s come at it from an educational standpoint, an educational perspective. My background is in education, so maybe I’m biased. But I think in this case, because it’s so hard to detect – I saw a study by an NYU professor the other day that even the best tools are struggling mightily to be able to pick up on AI writing and say, yeah, this is AI or it’s not human. There were some studies done about a peer review written by a human and a peer review written by GPT, and it was really hard for external observers to be able to come in and say this is this, this is that. They didn’t do a very good job of that.

So if that’s going to be the case, I think that the educational perspective and approach is the right one for now. And I’m really excited, actually, to be part of a coalition which is known as Kangaroo (sp?), which is just getting off the ground now. It’s going to include some of the top research institutions in the world. It’s going to include journal editors from the top 10 journals around the world. It’s going to include members from COPE and EASE and WAME and some of the really important institutions that are out there that are overlooking and setting policy when it comes to research integrity. And I think it’s really good that we’re having that conversation, because hopefully we’ll be able to come out of the other side of that with some sort of policy and guidelines about how we should be approaching these issues.

KENNEALLY: An online survey for Nature found that four out of five researchers have tried ChatGPT at least once. AI can be used as a research advisor, even as a peer reviewer. What other ways, Avi Staiman, do you expect to see chatbots emerge as collaborators for researchers?

STAIMAN: I think that’s a really good question, and I think that people are still in experimentation mode and trying to figure it out. I mentioned a few things earlier – brainstorming new topics and finding research gaps that didn’t exist before – again, we have to be really careful that we’re not letting the AI tell us what’s important in society, what topics need to be studied, but rather we’re using it to help us get over those challenges that we might have.

I think a really, really important use case is building an author platform and disseminating research. I think if there was one lesson – there were many lessons that we learned in the scientific world over COVID, but one of the biggest ones was the fact that we can’t expect people to take science at its word. We need to do a better job of communicating in a way that people will understand complex scientific concepts that sometimes we don’t always know all of the answers to. Being able to translate that – and I don’t mean different languages, but I mean translate it in a manner of taking complex, sophisticated ideas in an academic article which are impenetrable to the vast majority of society and turn them into more bite-sized – a lot of things now have TLDR, right – too long, didn’t read. Well, how do we turn research into TLDR without watering it down to the point where it’s not significant anymore? Actually, maybe GPT is a way to do that in a really meaningful way.

And I think it’s really important that GPT is iterative. What I mean by that is if you’re not happy, you tell it you’re not happy, and you tell it why, and it learns and it gets better. Someone made this point the other day. I wish I could give them credit, but I don’t remember who said it. I loved what they said. There were two really interesting observations. Number one, people were saying to think about GPT as a DJ. The DJ doesn’t make the music, but it knows how to mix and synchronize, and it knows what goes before what and what goes after and what order that should be. I really liked that analogy, because I thought it was spot-on in terms of understanding that there’s always a DJ out in the front. In theory, we could go without the DJ, maybe. But no, it’s what brings everything together, which is really kind of important.

And it’s a tool that if used properly can actually be used to really make this research accessible. I’m just thinking about the family, God forbid, who has a sick grandparent, and there’s a clinical trial that’s just been completed, and they’re debating whether or not they should be giving this new drug that just came out on the market. Until now, it’s been like, well, let’s just ask the doctor and hope that we can understand. And maybe if we have a little bit more knowledge, we’ll try to find stuff on WebMD. All of a sudden, maybe we can bring science to a point where, no, I can really feel empowered to make an informed, intelligent decision.

Again, I don’t want to build up GPT to something it’s not, because I’m not saying that if you just go in now, you can do that easily. But I think the potential use cases of it can lead to that in a really meaningful way.

KENNEALLY: Avi Staiman, founder and CEO of Academic Language Experts, thanks for speaking with me today.

STAIMAN: Thanks for having me, Chris. It was a real pleasure, and I’m really excited that you’re taking the opportunity to really do a deep dive into this, because I think it’s going to be an important part of our lives in the upcoming years.

KENNEALLY: Well, that’s all for now. Our producer is Jeremy Brieske of Burst Marketing. You can subscribe to the program wherever you go for podcasts, and please do follow us on Twitter and on Facebook. You can also find Velocity of Content on YouTube as part of the CCC channel. I’m Christopher Kenneally. Thanks for listening.