HackerTube.dev

The spelled-out intro to language modeling: building makemore

Andrej Karpathy · 1.3M views · 2022-09-07 · 1:57:45

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.

On Hacker News

HN recommends it as a long but worthwhile way to demystify how language models work technically.

Recommended by 2 HN people across 2 months · HN mentions 2024–2024

Topics

Machine learning & deep learningLLMs & generative AI
Search by tags
makemorebigrampytorchneural networkskarpathynext-token prediction

What people said

georgehill · 2024-04 · Strong recommendation

You've made something interesting. Many people have already answered this here [1], so there's no point in me explaining it again. Dropping some pointers that will be helpful for you: - [LINK] - Highly recommend this one [VIDEO] - [LINK] - [LINK] 1: ex: [LINK]

Read the original HN comment ↗
islewis · 2024-02 · Strong recommendation

I resource I found invaluable in demystifying LLM's was Andrej Karpathy's Youtube series. In particular, his "Makemore" video built a basic LLM in the same vein as OP's. It's long (2h), but well worth the time if anyone is curious about the technical workings of LLMs. [VIDEO]

Read the original HN comment ↗
caprock · 2024-07 · Reference

For reference, this project is from Eureka Labs, which is the new education startup from Andrej Karpathy. * [LINK] It appears to be a generalized version of the bi-gram project, which he links to here: * [VIDEO] It's cool to see him putting in even more effort on the edu work. It's a real gift he has, and was visible all the way back in his grad student projects.

Read the original HN comment ↗

Sources

Video by Andrej Karpathy, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Matches the configured criteria
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources