HackerTube.dev

LSTM is dead. Long Live Transformers!

Seattle Applied Deep Learning · 540.6K views · 2019-12-03 · 28:48

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Leo Dirac explains how transformer models have replaced LSTMs in NLP. He gives background from bag of words through RNNs and LSTMs, then a technical deep dive into multi-headed self-attention and positional encoding, with sample code.

On Hacker News

Recommended as a clear explanation of why RNNs fall short and how transformers solve it.

Recommended by 4 HN people across 4 months · HN mentions 2020–2023

Topics

LLMs & generative AIMachine learning & deep learningComputer vision, speech & NLP
Search by tags
transformerslstmself-attentionpositional encodingnlp

What people said

leopd · 2023-05 · Strong recommendation

I don't think you really want a 5yo's version, because that's "computers can be smart". But for an SDE1-level explanation I made this video which has gotten tons of compliments and keeps people watching year after year: [VIDEO] It's in the context of NLP, which is where transformers started of course.

Read the original HN comment ↗
qigtyofhp · 2023-02 · Recommendation

The attention mechanism started as a simple trick to not use recurrent neural networks. Read the intro in the original paper "Attention is all you need paper" ( [LINK] ) This video explains the drawbacks to RNNs and how transformers solve that: [VIDEO] Andrej Karpathy explains attention here: [LINK] He explains how attention is seen as a communication network: [LINK]

Read the original HN comment ↗
peterfirefly · 2023-01 · Recommendation

Yannic Kilcher is great but this video worked better for me: "LSTM is dead. Long live transformers!" (Leo Dirac): [VIDEO]

Read the original HN comment ↗

Sources

Video by Seattle Applied Deep Learning, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Matches the configured criteria
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources