Transformers, the tech behind LLMs | Deep Learning Chapter 5

What people said

robrenaud · 2026-06 · Strong recommendation

3 blue 1 brown has a great visual introduction to transformers, the heart of LLMs. It's chapter 5. Start at chapter 1 if you want more background on neural nets and backprop. [VIDEO]

Read the original HN comment ↗

gf000 · 2026-01 · Strong recommendation

I really recommend watching this section of this video. Embeddings do encode plenty of "human knowledge" into the vector values and their relations to each other. [VIDEO] This even opens up a more data-based approach to linguistics, where it is also heavily used.

Read the original HN comment ↗

manmal · 2025-05 · Strong recommendation

This video explains the direction-encodes-trait topic very well IMO: [VIDEO] It’s the first in a series of three that I can very highly recommend. > there's no way that there's a 1-to-1 correspondence between concepts and dimensions. I don’t know about that! Once you go very high dimensional, there is a lot of direction vectors that are almost perfectly perpendicular to each other (meaning they can cleanly encode a trait). Maybe they don’t even need to be perfectly perpendicular, the dot product just needs to be very close to zero.

Read the original HN comment ↗