Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
I feel like I don't want it to happen, but maybe artificial general intelligence?
I think we're still a bit far off from that. No doubt the models will be quite good, but they won't be anywhere near general intelligence.
Cross modality is what is missing. We have models that can produce text, hear things, and see things really really well. Facilitating communication between these individual models, and probably creating some executive model to utilize them, is what's missing. We're probably still a few years from beginning to touch general intelligence
It probably won't happen until we move to new hardware architectures.
I do think LLMs are a great springboard for AGI, but I don't think the current hardware allows for models to cross the hump to AGI.
There's not enough recursive self-interaction in the network to encode nonlinear representations. So we saw this past year a number of impressive papers exhibiting linear representations of world models extrapolated from training data, but there hasn't yet been any nonlinear representations discovered and I don't think there will be.
But when we switch to either optoelectronics or colocating processing with memory at a node basis, that next generation of algorithms taking advantage of the hardware may allow for the final missing piece of modern LLMs in extrapolating data from the training set, pulling nonlinear representations of world models from the data (things like saying "I don't know" will be more prominent in that next generation of models).
From there, we'll quickly get to AGI, but until then I'm skeptical that classical/traditional hardware will get us there.