We Were Right! Real Inner Misalignment

What people said

kmeisthax · 2023-03 · Recommendation

I recommend actually watching some videos on the topic before insisting that the problem does not exist: [VIDEO]

Read the original HN comment ↗

DennisP · 2023-07 · Reference

That's an accurate assessment of the situation, according to every AI alignment researcher I've seen talk about it, including the relatively optimistic ones. This includes people who are mainly focused on AI capabilities but have real knowledge of alignment. This part in particular caught my eye: "Other assumptions could also break down in the future, like favorable generalization properties during deployment". There have been actual experiments in which AIs appeared to successfully learn their objective in training, and then did something unexpected when released into a broader environment.[1] I've seen some leading AI researchers dismiss alignment concerns, but without actually engaging with the arguments at all. I've seen no serious rebuttals that actually address the things the alignment people are concerned about. [1] [VIDEO]

Read the original HN comment ↗

airgapstopgap · 2023-05 · Criticism

Not really a characterization of this work (I like mechanistic interpretability even though I consider it a detour in the long run), but of the submission title. It's bad. I suspect it's informed by Rob Miles' characterization of this algorithm as "bizarre and inhuman"[1] AI safety/alignment discourse as a whole is so incredibly bad. It's autodidactic and weird in the bad sense, weird like people who think they are brave iconoclasts but just haven't the curiosity and humility to learn the basics of the field are weird. Add some money from Dustin Moskovitz to Open Philanthropy to Robert Miles' propaganda firehose on youtube, add policy lobbying and buying bankrupt crypto podcasters, and you have the worst that could come from the culture of public intellectualism. The notions they have inserted into this potentially vital field, like inner/outer misalignment and "mesaoptimization" [2] (where simple overfitting is the real and more productive explanation) are deeply misleading, technically illiterate and stoking public fears in directions we should care about the least, siphoning resources and attention from real issues informed by specifics of contemporary ML (chiefly data and curriculum engineering). They're close to putting a lid on LLM development – because they're getting spooked by some vague sense of growing "capabilities"; even though LLMs are evidently a vastly better approach to safe-yet-capable-general-AI than anything any of their thought leaders have ever come up w

Read the original HN comment ↗