I'd have to imagine that this PII was made publicly-available in order for GPT to have scraped it.

Get it to recite pieces of a few books, then let publishers shred them.

Accountability? For tech giants? AHAHAHAAHAHAHAHAHAHAHAAHAHAHAA

I'm curious how accurate the PII is. I can generate strings of text and numbers and say that it's a person's name and phone number. But that doesn't mean it's PII. LLMs like to hallucinate a lot.

I bet these are instances of over training where the data has been input too many times and the phrases stick.

Models can do some really obscure behavior after overtraining. Like I have one model that has been heavily trained on some roleplaying scenarios that will full on convince the user there is an entire hidden system context with amazing persistence of bot names and story line props. It can totally override system context in very unusual ways too.

I've seen models that almost always error into The Great Gatsby too.

Yup, with 50B parameters or whatever it is these days there is a lot of room for encoding latent linguistic space where it starts to just look like attention-based compression. Which is itself an incredibly fascinating premise. Universal Approximation Theorem, via dynamic, contextual manifold quantization. Absolutely bonkers, but it also feels so obvious.

In a way it makes perfect sense. Human cognition is clearly doing more than just storing and recalling information. "Memory" is imperfect, as if it is sampling some latent space, and then reconstructing some approximate perception. LLMs genuinely seem to be doing something similar.

They are probably publishing this because they've recently made bard immune to such attack. This is google PR.

Always has been. Just yesterday I was explaining AI image generation to a coworker. I said the program looks at a ton of images and uses that info to blend them together. Like it knows what a soviet propaganda poster looks like, and it knows what artwork of Santa looks like so it can make a Santa themed propaganda poster.

Same with text I assume. It knows the Mario wiki and fanfics, and it knows a bunch of books about zombies so it blends it to make a gritty story about Mario fending off zombies. But yeah it's all other works just melded together.

My question is would a human author be any different? We absorb ideas and stories we read and hear and blend them into new or reimagined ideas. AI just knows it's original sources

It isn’t, but the GDPR requires companies to scrub PII when requested by the individual. OpenAI obviously can’t do that so in theory they would be liable for essentially unlimited fines unless they deleted the offending models.

In practice it remains to be seen how courts would interpret this though, and I expect unless the problem is really egregious there will be some kind of exception. Nobody wants to be the one to say these models are illegal.

Now delete your posts from ChatGPTs memory.

I think the point is that it doesn’t matter how you got it, you still have an ethical responsibility to protect PII/PHI.

Don't doubt that they're doing this for evil reasons

google is probably trying to take out competing ai

Model collapse is likely to kill them in the medium term future. We're rapidly reaching the point where an increasingly large majority of text on the internet, i.e. the training data of future LLMs, is itself generated by LLMs for content farms. For complicated reasons that I don't fully understand, this kind of training data poisons the model.