TechTakes

1499 readers

60 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago

MODERATORS

dgerard@awful.systems

Get your Torment Nexus news from reputable journalistic outlet Know Your Meme (Character.AI Suicide Lawsuit) (knowyourmeme.com)

submitted 2 months ago* (last edited 2 months ago) by jaschop@awful.systems to c/techtakes@awful.systems

18 comments fedilink hide all child comments

archive of the mentioned NYT article

you are viewing a single comment's thread
view the rest of the comments

[–] BlueMonday1984@awful.systems 9 points 2 months ago (4 children)

Okay, quick prediction time:

Even if character.ai manages to win the lawsuit, this is probably gonna be the company's death knell. Even if the fallout of this incident doesn't lead to heavy regulation coming down on them, the death of one of their users is gonna cause horrific damage to their (already pretty poor AFAIK) reputation.
On a larger scale, chatbot apps like Replika and character.ai (if not chatbots in general) are probably gonna go into a serious decline thanks to this - the idea that "our chatbot can potentially kill you" is now firmly planted in the public's mind, and I suspect their userbase is gonna blow their lid from how heavily the major apps are gonna lock their shit down.

[–] gerikson@awful.systems 8 points 2 months ago (3 children)

I'm not familiar with Character.ai. Is their business model that they take famous fictional characters, force feed their dialog into LLMs, then offer these LLMs as personalized chatbots?

[–] BlueMonday1984@awful.systems 6 points 2 months ago (2 children)

From the looks of things, that's how their business model works.

[–] gerikson@awful.systems 7 points 2 months ago (1 children)

I found an Ars piece:

https://arstechnica.com/tech-policy/2024/10/chatbots-posed-as-therapist-and-adult-lover-in-teen-suicide-case-lawsuit-says/

Asked for comment, Google noted that Character.AI is a separate company in which Google has no ownership stake and denied involvement in developing the chatbots.

However, according to the lawsuit, former Google engineers at Character Technologies "never succeeded in distinguishing themselves from Google in a meaningful way." Allegedly, the plan all along was to let Shazeer and De Freitas run wild with Character.AI—allegedly at an operating cost of $30 million per month despite low subscriber rates while profiting barely more than a million per month—without impacting the Google brand or sparking antitrust scrutiny.

[–] Soyweiser@awful.systems 11 points 2 months ago

Updated 'do no evil' into 'if you are going to convince children to kill themselves while doing massive copyright infringement, at least dont hurt the brand'