I've heard many papers are published to never be read by humans. It only makes sense that some portion of those papers aren't written by humans either.
I wonder what the overlap is between AI assisted papers and papers with few to no readers.
this post was submitted on 07 Oct 2023
120 points (96.2% liked)
Science
13163 readers
83 users here now
Subscribe to see new publications and popular science coverage of current research on your homepage
founded 5 years ago
MODERATORS
The whole system should get ready for the 21st century.
Most of the scientists arent great writers. It does not make sense to still force them to be a good writer.
Let be fishes be good at swimming instead of climbing trees.
In a modern world where basically EVERYTHING is specialized and no generalist is alive anymore we should make use of language tools.
Hell Chatgpt writes an introduction which is fun to read instead or my overcomplicated bullshit that I would have brought up
Edit: the comment was not related to the OP but to a general chatgpt discussion.
One important thing is that you have potential. ChatGDP will write something alright-ish, but it's literally impossible for it to move beyond that. It doesn't have the power of creativity.
Writing is painful, but it also helps us think clearer about our work and contribution. I think it's an important part of the process of doing science, no matter which field. And one gets better at it with training.
I dont need it to be beyond that? It does what I told it. And if I am creative enough to get my preferred output its great. I have still to decide if Ill use it.
Its a tool which can be used by people and helps with work.
I think it's an important part of the process of doing science, no matter which field. And one gets better at it with training
Sorry but this expression is probably a similar one when paper writting shifted to digital only format or when the typewriter was introduced.
Boomer tell me the same with printed paper. "oNlY whEn ItS PriNtED yOu cAn rEaD pRoberly"
Thats bullshit its just fear of the something new and convenience of routine.
Nothing personal against you. I welcome any tool that helps me.
So I'm not sure it's helping you.
You would refrain from doing the work of organizing the concept in your head into a clearly communicable explanation of the concept.
I think of it as in another anology.
Compare a screwdriver with a power tool.
Does the convenient solution hinder you from building your house simply because you cant "feel" the strength of the wood while turning the screw in?
i doubt.
The things you mentioned are coming into play when people think of AI as a god mode. As a user you are solely responsible for how to use a tool. If the user overestimates the power of the tool or use it for the wrong things. Its the users fault.
The scientist is still a scientist. Which is the author of the paper. Not gpt because it writes filler text or puts the scientists thoughts into sentences.
The context is still at the scientists plate. If the scientist does a poorly job at reviewing the gpts output. Gpt cant be faulted.
A research paper is not bulk work like a house. It's more like a watch, and a watchmaker using a screw gun is daft.
Thats another point. Fair enough.
But still I dont think that science will stall just because of chatgpt.
Journalism? Will for sure. But scientific publications have a systemic problem (like publisher-polism, pubscores etc) And outsourcing writing work to chargpt is - in my opinion - non of them.
Sure, but using bad tools to do things is going to get you worse results that using the right tools. If we define worse as "less volume" then sure GPT is fine.
it's literally impossible for it to move beyond that. It doesn't have the power of creativity.
a test for creativity seriously that work? also after scraping the entire of internet of course someone could think that, ask any programmer and they gonna explain that the IA don't create anything, it can't even do basic msth because it don't gave logic in that,maybe one day, but not with chatgpt of today
Yes, a test for creativity. If you're going to say something "doesn't have the power of creativity" then it behooves you to accept the notion that creativity is measurable.
@Vilian @FaceDeer I agree. I’m no programmer but do a fair bit of Linux/powershell/bash scripting. Virtually all the code that ChatGPT gives me is wrong. You tell it the errors, and it gives you a modified script with errors, point out those errors and it’s go back to its first answer. The only thing it is useful for is writing lots of basic code, really quickly. I can just copy/paste then start debugging.
I am a programmer and I've found ChatGPT to be able to produce plenty of good, useful code. I haven't encountered the problems you're describing in correcting its errors, perhaps you're not prompting it well.
@FaceDeer @floofloof @henfredemars @PoisonedPrisonPanda @sab @Vilian nah, it was trained in 2021 and parrots 10 year old stack overflow pages. That may have worked a decade ago, but stuff has moved on since then. It still spits out code using AzureAD cmdlets as it doesn’t know MSGraph replace a lot of it in last couple of years. I guess it could be ok if you’re on a legacy tech stack though.
So you're telling me that the code it generated for me wasn't good and useful, and that when I told it to correct errors it actually did introduce new errors and restore old ones, contrary to what I just said? Guess all that stuff I got done using its help didn't actually get done after all and I'm descending ever deeper into a world of delusion, thinking my projects are finished and working when in fact they aren't.
Obviously if you're trying to get it to use APIs from after 2021 that's not going to work. It also won't bake you a cake if you ask it to. Use tools for the tasks they're good for, don't use them for things you know they can't do.
@FaceDeer @floofloof @henfredemars @PoisonedPrisonPanda @sab @Vilian the 500 char limit made me pick the first failure that spring to mind. Maybe you forget, but AI wasn’t trained on “good” data. It was trained “all” data and large amounts of that is plain wrong. People with problems pasting blocks of code and responses correcting a single line. ChatGPT isn’t smart enough to merge those into a single block of working code.
You have to tell ChatGPT that you want good code, then.
I'm actually serious. If you just ask for something generic, it'll assume you want something generic. If you ask it for something that's "high efficiency, well commented and maintainable" then it's going to know you wanted that and give you something more along those lines. Just like if you asked it for something "that looks crappy and sloppy, like an amateur wrote it."
Very often when people complain about ChatGPT's "style" or say they can immediately spot something that "sounds like" ChatGPT it's because they're not giving it good directions. It can't read your mind. Yet.
@FaceDeer @floofloof @henfredemars @PoisonedPrisonPanda @sab @Vilian nope. I ask for highly precise stuff. When I say, “I’m no coder,” it’s coz I use interpreters and don’t compile “real” code, plus it only accounts for 10% of my day job. ChatGPT maybe useful for Hello World, Towers of Babel or other stuff it scraped from udemy, but when you ask it to assist in automating complex production systems, it really falls down.
Maybe ChatGPT just hates you personally, then.
You're saying "it can't work for anyone because it doesn't work for me!" And I'm saying "well, it worked for me, so maybe you're using it wrong."
You can't insist it's not working for me because it did. I'm not disputing that it didn't work for you, all I can suggest is reading up a bit on prompt engineering to see if you can find out what you're doing differently.
@FaceDeer @floofloof @henfredemars @PoisonedPrisonPanda @sab @Vilian I’m not saying it didn’t work for you. I’m saying it’s only good for entry level stuff… It’s not coming for my job yet, but I can seriously see it stealing work from the overseas, underpaid, code shops in China/India.
I am not an "entry level" coder.
I also am not concerned with it "coming for my job." It's a collaborator with humans, not a replacement for them.
This is the key - it does not create, it can only copy. Which is good enough to fool us - there's enough stuff to copy out there that you can spend your whole life copying other people and nobody will ever notice you're not actually creating anything new. What's more, you'll probably come across as pretty clever. But you're not creating anything new.
For me, this poses an existential threat to academia. It might halt development in the field without researchers even noticing: Their words look fine, as if they had thought it through, and they of course read it to make sure it's logically consistent. However, the creative force is gone. Nothing new will come under the sun - the kind of new thoughts that can only be made by creative humans thinking new thoughts that have never been put on paper before.
If we give up that, what's even the point of doing science in the first place.
There's a difference between:
-
Using ChatGPT to help write parts of the text in the same way you'd use a grammar- or spell-checker (e.g. if English isn't your first language) after you've finished the experiments
-
Using ChatGPT to write a paper without even doing any experiments
Clearly the second is academic misconduct. The first one is a lot more defensible.
Yes, absolutely. But I still think it has its dangers.
Using it to write the introduction doesn't change the substance of the paper, yet it does provide the framework for how the reader interprets it, and also often decides whether it'll be read at all.
Maybe worse, I find that it's oftem in the painful writing and rewriting of the introduction and conclusion that I truly understand my own contribution - I've done the analysis and all that, but in forcing myself to think about the relevance for the field and the reader I also bring myself to better understand what the paper means in a deeper sense. I believe this kind of deep thinking at the end of the process is incredibly valuable, and it's what I'm afraid we might be losing with AI.
This is the key - it does not create, it can only copy.
I have asked ChatGPT to write poetry on subjects that I know with great certainty have never had poems written about them.
You can of course shuffle around the meanings of "create" and "copy" to try to accommodate that, but eventually you end up with a "copying" process that's so flexible and malleable that it might as well be creativity. It's not like what comes out of human brains isn't based on stuff that went into them earlier either.
It's interesting that you write this because the last place I worked focused on unspecializing by having almost everyone do every job.
In fact, they relocated across the country to save on building costs, and instead of hiring actual technical writers and office staff, they pushed the extra work down on their engineers because it's more profitable to bill for the engineering time.
I spent much of my job editing papers and I'm not even good at it while getting paid to do embedded design. It was weird. It was basically fraud but walking the fine line of technically legal.
I observed this happening multiple times throughout my career. Sometimes, inefficiency is the point in this case driven by capitalists and market forces.
... Did you read the article? Language tools like grammarly and deepL are in use by scientists today. Copying+pasting the output of chatGPT without ever looking at it, or even using a language tool to publish thoughts that were never in your head to begin with, is the actual concern
Did you read the article?
I for sure didnt.
Thanks for highlighting that.
I was carried away by having the discussions at my university with my peers in mind.
Copying+pasting the output of chatGPT without ever looking at it, or even using a language tool to publish thoughts that were never in your head to begin with, is the actual concern
Nevertheless I dont understand why this is a concern.
The scientific standards existed decades if not already at least a century.
Those discussions are putting chatgpt in a bad light. However the fact that our scientific system was eroded and made a mockery of before the introduction of chatgpt is not highlighted.
There are still plagiarizations around and nobody cares. Mostly because of political sensitivity.
However science has failed to repel "bad actors" (intentional or unintentional) from the scene.
I dont know when. And why. But publisher have for sure something to do with it.
I agree, I have no problem with people guiding chatgpt to help them write something they want and they checked it.
Generating bunch of articles even they didn't read is something else.
ok sorry side note, 40 FUKKEN EUROS TO READ IT? do they want their research read or not
It's worse than that. Authors actually pay (up to several thousand dollars) to publish, the editors who find referees are doing this as a side job, so probably they're not exactly overpaid either. Finally you have the anonymous referee, who not only doesn't get paid, but they get literally zero recognition. Also, papers aren't printed in journals any more, they are online only, so there's no printing fee either, there's only just server hosting costs, paying some people for language editing and final typesetting (in many fields authors must submit LaTeX manuscripts, basically ready for publishing). And profit of course.
Yep, it’s a fucking embarrassment. Clearly science and academia stopped attracting our brightest and best a while ago or their egos are so fragile they’re as easy to manipulate as children. Either way, institutionally, very poor leaders and caretakers of institutions, which truly undermines the faith we can have in the quality of research they are doing.
I can understand why it seems the way. But the people doing academic research by and large could make a lot more money working less hard at some company, but choose instead to try to advance human knowledge.
The incentives are just terrible. When I was a PhD student, I railed against this system, but when it came time to publish, I was overruled by my PI. And I know now that he was right - success is built off publication, and the best journals have this shitty model.
I used to think that when I became boss, I wouldn't participate in the bullshit, but if any of my trainees want a career in academia, that stance would be screwing them over. The rules need to come from the top, but the people at the top, almost by definition, are the ones that have prospered with the current system.
I used to think that when I became boss, I wouldn’t participate in the bullshit,
You can't change the system single-handedly overnight, but you can be active in your research community, e.g. you can suggest that conference proceedings be available for free online.
Also, if your trainees publish in journals, just make sure to put your pre-prints on arxiv or somewhere similar for free.
I do these things. I also refuse to review for-profit journals and paper mills, post all of my code in open source repositories, and advocate for these practices whenever I get the chance. When I had a popular science blog over 10 years ago, writing about this stuff a bunch.
But as long as hiring committees are scanning CVs for the number of Nature/Science/Cell journals, and granting agencies aren't insisting on different practices, this shit will continue.
The rules need to come from the top, but the people at the top, almost by definition, are the ones that have prospered with the current system.
And if these smart academically inclined people can’t reason about the merits of the system beyond whether it has worked for them, then they are as I accused them … unintelligent or childish.
You speak of higher salaries outside of academia, but from what I’ve seen (where you shouldn’t presume I haven’t worked in academia) success in academia is its own reward with prestige that should not be underestimated.
And if these smart academically inclined people can’t reason about the merits of the system beyond whether it has worked for them, then they are as I accused them … unintelligent or childish.
Nah, it's really hard to notice things that are against your incentives to notice. And if all of the people around you are prospering in the same system, extra hard. The myth of meritocracy is extremely compelling, possibly to an even greater extent in academia than elsewhere.
success in academia is its own reward with prestige that should not be underestimated.
No doubt. And listen, I'm on the tenure track job market at this very moment, having said that last year was definitely going to be my last attempt. There's some kind of cultish nature, all the more inextricable in that I can see it, and it doesn't stop me.
I guess my point is that it's obvious to most of us that that success is extremely rare, and getting rarer. The thing that keeps me in it is the sense that I can do more good pursuing knowledge for knowledge's sake than work that is easier and more remunerative but less fulfilling. Call that stupid or childish? Maybe 🤷.
Yea we’re not disagreeing here.
The myth of meritocracy is extremely compelling, possibly to an even greater extent in academia than elsewhere.
This of essentially what I’m targeting in my dumb/childish accusation, especially in those with some job security or tenure and so have more power and ideally wisdom.
I can do more good pursuing knowledge for knowledge’s sake than work that is easier and more remunerative but less fulfilling.
That’s idealistic I’d say, and all the luck to you my friend.
In the end though, my accusation is intentionally provocative and intentionally aimed at what academics take pride in because at some point, IMO, academia needs to see that they’re often embarrassing themselves and letting themselves down, maybe not individually, but at some level. And, beyond that, maybe not pursuing as much of the greater good as we would all like to think.
I suspect academia might be pretty central to civilisation and the more corrupt it gets the more corruption leaks into the civilisation.
Anyway. All the best to you. Hope you get what you’re looking for and don’t burn out from the system or anything like that. Cheers for the chat!!
Okay but who gets the 40 euros then? All goes for server maintenance costs?
It goes to the publisher's profits.
Publishing houses who control the entire industry and whom you have to go through because they have the professional networks and publishing somewhere without "prestige" is literally worse than not publishing at all.
It's pretty fucked.
That's the publishers fee, the authors typically don't get paid for their work to be published. It costs a couple grand to get your paper published and free for the general public.
Also a fee for the universities, who need to subscribe to the journals for the authors to be able to read their own fucking work.
Which btw is most often paid for by the respective countries.
@floofloof They charge €40 for access, yet one is left wondering what sort of peer review this paper has undergone when obvious signs of generative AI has slipped in. What about less obvious signs? If the "authors" had simply used the copy-to-clipboard icon in chatgpt, they would have been all good and this would never have been uncovered.
If anything, this is an argument for free public access to scientific papers. Any experts on AI could scan and detect this, even when it's more subtle.