Intelligence and Stupidity: The Orthogonality Thesis

What people said

simonh · 2023-05 · Strong recommendation

That’s the alignment problem. We don’t know what the actual goals of an AI trained neural net are. We know what criteria we trained it against, but it turns out that’s not at all the same thing. I highly recommend Rob Miles channel on YouTube. Here’s a good one, but they’re all fascinating. It turns out training an AI to have the actual goals we want it to have is fiendishly difficult. [VIDEO]

Read the original HN comment ↗

pegasus · 2023-03 · Strong recommendation

I've seen that paper, and it's amazing indeed what GPT-4 is capable of. But none of that supports that closing quote, which to me points to a worrying philosophical naïveté. How can one equip an entity with "intrinsic motivation"? By definition, if we have to equip it with it, that motivation is extrinsic. It belongs to the one who puts it there. A software engineer might decide to prompt his creation with something along the lines of "Be fruitful and increase in number; fill the earth and subdue it. Rule over the fish in the sea and the birds in the sky, and over every living creature that moves on the ground." and the bot will duly and with limitless enthusiasm put those wise orders into practice. If that ends up causing some "minor" problems, should we confine the software for one thousand years in an air-gapped enclosure, until it learns a bitter lesson? Or should we take to task the careless Maker of the bot? To me, the answer is obvious. PS: this video to me states the problem in a nutshell (that whole channel is gold) [VIDEO]

Read the original HN comment ↗

pegasus · 2023-03 · Strong recommendation

Yes, I believe some careful and informed pondering leads one towards that conclusion. Hume's guillotine is an implacable philosophical device: [VIDEO] (I warmly recommend all the content on that delightfully frightening channel)

Read the original HN comment ↗