Technology

> When people are writing, they aren’t looking at a huge database of information and determining the most likely word to come next, they’re synthesizing concepts together to create new ones, or building a narrative based on their notes. They understand concepts, they understand definitions.

A huge part of what we do is like drawing from a huge mashup of accumulated patterns though. When an image or phrase pops into your head fully formed, on the basis of things that you have seen and remembered, isn't that the same sort of thing as what AI does? Even though there are (poorly understood) differences between how humans think and what machine learning models do, the latter seems similar enough to me that most uses should be treated by the same standard for plagiarism; only considered violating if the end product is excessively similar to a specific copyrighted work, and not merely because you saw a copyrighted work and that pattern being in your brain affected what stuff you spontaneously think of.

I don't think this is true.

The models (or maybe the characters in the conversations simulated by the models) can be spectacularly bad at basic reasoning, and misunderstand basic concepts on a regular basis. They are of course completely insane; the way they think is barely recognizable.

But they also, when asked, are often able to manipulate concepts or do reasoning and get right answers. Ask it to explain the water cycle like a pirate, and you get that. You can find the weights that make the Eifel Tower be in Paris and move it to Rome, and then ask for a train itinerary to get there, and it will tell you to take the train to Rome.

I don't know what "understanding" something is other than to be able to get right answers when asked to think about it. There's some understanding of the water cycle in there, and some of pirates, and some of European geography. Maybe not a lot. Maybe it's not robust. Maybe it's superficial. Maybe there are still several differences in kind between whatever's there and the understanding a human can get with a brain that isn't 100% the stream of consciousness generator. But not literally zero.

How can you tell that our thoughts don't come from a biological LLM? Maybe what we conceive as "understanding" is just a feeling emerging from a more fondamental mechanism like temperature emerges from the movement of particles.

I think you underestimate the reasoning power of these AIs. They can write code, they can teach math, they can even learn math.

I've been using GPT4 as a math tutor while learning linear algebra, and I also use a text book. The text book told me that (to write it out) "the column space of matrix A is equal to the column space of matrix A times its own transpose". So I asked GPT4 if that was true and it said no, GPT disagreed with the text book. This was apparently something that GPT did not memorize and it was not just regurgitating sentences. I told GPT I saw it in a text book, the AI said "sorry, the textbook must be wrong". I then explained the mathematical proof to the AI, and the AI apologized, admitted it had been wrong, and agreed with the proof. Only after hearing the proof did the AI agree with the text book. This is some pretty advanced reasoning.

I performed that experiment a few times and it played out mostly the same. I experimented with giving the AI a flawed proof (I purposely made mistakes in the mathematical proofs), and the AI would call out my mistakes and would not be convinced by faulty proofs.

A standard that judged this AI to have "no understanding of any concepts whatsoever", would also conclude the same thing if applied to most humans.