Technology

"cheat", "lie", "cover up"... Assigning human behavior to Stochastic Parrots again, aren't we Jimmy?

This is bad science at a very fundamental level.

Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.

I've written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM's generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn't obtain tips or act upon them -- it generates text based on previous text. That's it. There's no need to put a black box around it and treat it like it's human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.

To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Part of being a good scientist is studying things that mean something. There's no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you'd be the first person to study that, but it's a stupid thing to study.

This makes perfect sense. It's been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.

I've never had ChatGPT just say "actually I don't know the answer" it just gives me confidently correct wrong information instead.

Study finds nonintelligent pattern-generating algorithm to be nonintelligent and only capable of generating patterns.

I feel like "lie" implies intent, and these imitative large language models don't have the ability to have intent.

They're imitating us. Or more specifically, they're imitating the database(s) they were fed. When chat GPT "lies" to "cover it up," all it's actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

Ahah it is ready to take the job of pur politicians

Everybody forgot that chatGPT-2 was just a bullshitting machine. Version 3 to the surprise of the developers very useful to many people while they just made a highly trained bullshitting machine.

Bullshit.

It should instead read:

"Humans were stupid and taught a ChatBot how to cheat and lie."

deleted

we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent

This already is total BS. If you know how such language models work you'd never take their responses at face value, even though it's tempting because they spout their BS so confidently. Always double-check their responses before applying their "knowledge" in the real world.

The question they try to answer is flawed, no wonder the result is just as bad.

Before anyone starts crying about my language models opposition: I'm not opposed to LMs or ChatGPT. In fact, I'm running LMs locally because they help me be more productive and I'm a paying ChatGPT customer.

It seems like there's a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token (part of a word) probabilistically.

Created in our image

They learn so quick (sniff), it’s almost all grown-up now.

It's a neural net designed in our image based using our pain and greed based logic/learning/universal context as a knowledge base. Can't really be surprised it emulates this feature of humanity 😂

Yeah so what? Its purpose is literally to SIMULATE natural language, of course it will simulate it.

Nowhere ChatGPT is advertised as something telling only the truth and reporting only facts.

It is like watching a movie and then whining because what is shown in the movie is not real.

Saying "study" in this context makes me imagine some dude chatting with cgpt at 3am and writing down "success" next to "give you up and let you down"

Yet again confusing LLMs with an AGI. They make statistically plausible text on the basis of past text, that's it. There's no thinking thing there

Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:

“The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.

Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.

All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.

Honestly, the fact that these things are dishonest and we dont, maybe even can't know why is kind of a relief to me. It suggests they might not do the flawless bidding of the billionaires.