GPT3: An Even Bigger Language Model - Computerphile

What people said

MichaelMoser123 · 2023-02 · Reference

maybe ask your favorite language model for advice. They say it's the answer to all questions, sort of like 42. My point is here: [VIDEO] - they explain that gpt3 (almost) learned how to add and multiply, but they don't know how. To me that's the thing. (the insight comes at 12:29, 15:23) in a sense that is kind of an admission that we don't know the heck what is going on.

Read the original HN comment ↗

CSMastermind · 2023-01 · Reference

> No one in 2018 suspected that large language models would smoothly scale up by simply increasing the number of parameters. [LINK] That was kind of the entire point of GPT-2. Computerphile summed it up pretty well on GPT-3's release: [VIDEO] Here's some quotes from that video: "The thing about gpt2 is just that it was much bigger than anything that came before. It was more parameters and was kind of the point of that paper." ... "They made gpt2 because the curve wasn't leveling off. We've gone 117 times bigger than gpt2 and they're still not leveling off."

Read the original HN comment ↗

stcredzero · 2021-08 · Reference

The industry collectively sank untold billions into the blind belief that neural algorithms will somehow turn into "AI." The huge qualitative differences between GPT-2 and GPT-3 seem to suggest that they will, if you just keep adding orders of magnitude more connections and more data. [VIDEO]

Read the original HN comment ↗