Technology
This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.
Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.
Rules:
1: All Lemmy rules apply
2: Do not post low effort posts
3: NEVER post naziped*gore stuff
4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.
5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)
6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist
7: crypto related posts, unless essential, are disallowed
view the rest of the comments
I 100% can see it easily selling for that much.
You want to know why it's worth that much?
Petabytes of raw training Data for LLMs. Arguably atm reddit us one of the better gold mines of LLM training data on the internet, bazillion of posts already formatted as post-response chains, which is the exact type if format an LLM wants to train on.
Can you imagine how valuable those servers loaded with posts are to a company like OpenAI, Google, or Microsoft?
5 billion is quite reasonable to harvest every reddit post that has ever been made ever and cut it off from your competitors.
Has reddit not already been scraped? With all of that information exposed bare on the public Internet for decades, and apparently so valuable, I find it hard to believe that everybody's just been sitting there twiddling their thumbs, saying "boy I sure hope they decide to sell us that data one day so that we don't have to force an intern to scrape it for us".
I asked Bard whether they could make a bastardization of rogersimon10 post and it made this:
Scraped data isn't legal to resell, scraping isn't even legal in the first place.
Just because you can scrape the data doesn't mean it's worth anything.
Companies like MS, Google, OpenAI, FB they make money by selling the usage of their LLM services to other companies who then they use that service to make their own products.
If it came to light that MS/Google/OAI/FB were using illegal training data for their LLMs, it would get all those other companies hit in the crossfire.
So these companies have to do a shit tonne of diligence to assure their investors and clients that their LLMs are purely trained on legally obtained data and are safe to use.
And you know what is a super easy way to assure them of that?
If they literally own the original data themselves
This data was out in the open for a decade and still is. People could train their llm without problems.
Not legally / free.
And yes, that very very much matters if you intend to actually sell the service to companies that they themselves dont want to get hit in the crossfire of potential lawsuits for building their products on top of stolen info.
So if you can own the data itself (via buying reddit), you now have an ENORMOUS quantity of prime training data that you're investors and potential customers know is legally clean, because you literally own it.
They just started charging for API usage so isn't it already not open?
And then everyone started deleting accounts, comments and even rewriting and poisoning their comments. The data was way better before the API change.
The worst part is that ai chatbots will start responding like redditers. I can't wait for chatgpt to regale me with a story about his dad beating him with jumper cables, or jolly ranchers, or hell in a cell.
Nonstop "this"
Thanks for the gold
And then everyone clapped
And then asked "AITA?"