HackerTube.dev

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy · 8.2M views · 2023-01-17 · 1:56:20

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

On Hacker News

It is the go-to recommendation for learning how transformers and LLMs actually work, praised for its clarity and Karpathy's teaching.

Recommended by 14 HN people across 8 months · HN mentions 2023–2025

Topics

LLMs & generative AIMachine learning & deep learning
Search by tags
gpttransformerpytorchnanogptattentionfrom scratch

What people said

villedespommes · 2024-01 · Strong recommendation

+1, Andrey is an amazing educator! I'd also recommend his [VIDEO] and checking out his github repos. MinGPT, for example, implements a small gpt model that's compatible with HF API, whereas more modern nanoGPT shows how to use newer features such as flash attention. The quality of every video, every blog post is just so high.

Read the original HN comment ↗
wesammikhail · 2023-05 · Strong recommendation

Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on the topic. Here is the link: [VIDEO] If you conclude, after watching this, that this is all intelligence is - namely cascading optimization and data structure problems chained in a row, then you and a fair number of people here wont ever find common ground. That is not to say that you're wrong, it just seems as if there is a missing component here that we haven't yet discovered.

Read the original HN comment ↗
durdn · 2023-02 · Strong recommendation

Very impressive. Recently I watched this really amazing lecture on building GPT from scratch from Karpathy, I was blown away: [VIDEO]

Read the original HN comment ↗

Sources

Video by Andrej Karpathy, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Matches the configured criteria
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources