this post was submitted on 24 Jul 2024
436 points (97.2% liked)
Technology
58131 readers
4401 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This isn't really accurate either. At the moment of generation, an LLM only has context for the input string and the network of text tokens it's been assigned. It pulls from a "pool" of these tokens based on what it's already output and the input context, nothing more.
Most LLMs have what are called "Top P", "Top K" etc, these are the number of tokens that it ends up selecting from based on the previous token, alongside the input tokens. It then randomly chooses one based on temperature settings.
It's why if you turn these models' temperature settings really high they output pure nonsense both conceptually and grammatically, because the tenuous thread linking the previous token's context to the next token has been widened enough that it completely loses any semblance of cohesiveness.