Technology

I’m not sure I’d say it’s plateaued today but I definitely think machine learning is going to hit a wall soon. Some tech keeps improving until physical limits stop progress but I see generative AI as being more like self-driving cars where the “easy” parts end up solved but the last 10% is insanely hard.

There’s also the economic reality of scaling. Maybe the “hard” problems could, in theory, be easily solved with enough compute power. We’ll eventually solve those problems but it’s going to be on Nvidia’s timeline, not OpenAI’s.

Let me save you a click: he doesn’t say anything interesting about why he thinks this.

What does Bill Gates know about GenAI? Is he an expert on the subject?

Cool, Bill Gates has opinions. I think he's being hasty and speaking out of turn and only partially correct. From my understanding, the "big innovation" of GPT-4 was adding more parameters and scaling up compute. The core algorithms are generally agreed to be mostly the same from earlier versions (not that we know for sure since OpenAI has only released a technical report). Based on that, the real limit on this technology is compute and number of parameters (as boring as that is), and so he's right that the algorithm design may have plateaued. However, we really don't know what will happen if truly monster rigs with tens-of-trillions of parameters are used when trained on the entirety of human written knowledge (morality of that notwithstanding), and that's where he's wrong.

bill is a wanker. dont be like bill.

I hope so. Theyve already got scary implications for creative parts of the economy.

That said, we're in the Cambrian explosion of the tech. As it plateaus, the next step will be enhanced tooling and convenience around it. Better inputs than just text, better, more applications in new spaces, etc.

"GPT-4 should be enough for anyone." -Bill Gates

But we have more areas to apply this to. I still can't ask my PC to do some work, like Unistall OneDrive or change a setting in the OS. Send a message on Teams. Where is Jarvis?

Not a single comment yet stating how Gates is a great human being because of his foundation, and how all you haters should fuck the fuck off? sigh, let me the first one.

On the one hand, I don't really know enough about AI to comment. What I do remember is that, Bill Gates said the Internet was just a fad in the 90s. This comment caused myself and others problems promoting the Internet in workplaces because those in charge for some reason put some weight to his words. :p

I've been saying this for years!

Bill Gates views on AI are about as insightful as Gordon Ramsey’s on orbital mechanics.

I think he could be right about generative AI, but that's not a serious problem given we're moving beyond generative AI and into virtual intelligence territory.

Generative ai right now requires someone (or something) to initiate it with a prompt, but according to some of the latest research papers in OpenAI as well as the drama that happened recently surrounding the leadership, it appears that we're moving beyond the 'generative' phase into the 'virtual intelligence' phase.

It's not going to be 'smart' it will be knowledgeable (and accurate, hopefully). That is to say VI's will be useful as data retrieval or organization but not necessarily data creation (although IIRC the way to get around this would be to develop a VI that specifically only works on creating ideas but we'd be moving into AGI territory and I don't expect we'll have serious contenders for AGI for another decade at least).

The rumours abound surrounding the OpenAI drama, the key one being the potential for accidentally developing AGI internally (I doubt this heavily). The more likely reason is that the board of directors had a financial stake in Nvidia and when they found out altman was working on chips specifically for AI that were faster, lower cost, and lower power consumption than current nvidia trash (by literally tens of thousands of dollars), they fired him to try and force the company onto their preferred track (and profit in the process, which IMO, kind of ironic that a non-profit board of directors has so many 'closed door' discussions with nvidia staff...)

This is just the thoughts of a comp-sci student with a focus on artificial intelligence systems.

If interested in further reading:

https://www.ibm.com/blog/understanding-the-different-types-of-artificial-intelligence/

https://digitalreality.ieee.org/publications/virtual-intelligence-vs-artificial-intelligence

https://www.psychologytoday.com/us/blog/what-we-really-want-in-a-leader/202204/why-you-need-to-focus-on-virtual-intelligence

Keep in mind that because it's still early days in this field that a lot of terms haven't reached an established consensus across academia yet, so you'll notice variations in how each organization explains what "x" type of intelligence is.

Maybe, but I am sure the tools the AIs can use will improve making the AIs jobs easier and thus the AI more efficient. I hope he is right tbh.

Eww, as a long time Linux user I need to take a shower now. I feel dirty.