HackerTube.dev

The Simple Essence of Automatic Differentiation - Conal Elliott

Microsoft Research · 17K views · 2018-07-26 · 1:30:45

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Conal Elliott presents his paper on automatic differentiation, deriving a simple, generalized AD algorithm from a natural specification. Varying the representation of derivatives yields forward and reverse mode algorithms that avoid the complexity and statefulness of backpropagation.

On Hacker News

Cited as the more accessible companion to the paper, and as the most interesting treatment of AD via compiling to categories.

Recommended by 3 HN people across 4 months · HN mentions 2018–2023

Topics

Machine learning & deep learningFunctional programmingPure mathematics
Search by tags
automatic differentiationcategory theorybackpropagationhaskell

What people said

infogulch · 2022-04 · Strong recommendation

The most interesting thing I've seen on AD is "The simple essence of automatic differentiation" (2018) [1]. See past discussion [2], and talk [3]. I think the main idea is that by compiling to categories and pairing up a function with its derivative, the pair becomes trivially composable in forward mode, and the whole structure is easily converted to reverse mode afterwards. [1]: [LINK] [2]: [LINK] [3]: Talk at Microsoft Research: [VIDEO] Other presentations listed here: [LINK]

Read the original HN comment ↗
sriku · 2022-09 · Recommendation

I'm yet to give reverse mode a second try, but my attempts at forward mode (linked in parent post) were on my own without the paper's help. There's a talk that Conal gives that makes the paper a bit more accessible - [VIDEO] - but only a bit more. I know it'll take effort to wrap my head around the implementation details ... but the paper certainly helped clarify things I was stumbling around about.

Read the original HN comment ↗
noblethrasher · 2019-11 · Recommendation

He also gave nice presentation on AD: [VIDEO] (surprised that you didn't already mention it, considering the venue ;-))

Read the original HN comment ↗

Sources

Video by Microsoft Research, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Topic keywords do not match
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources