HackerTube.dev

Zebras All the Way Down - Bryan Cantrill, Uptime 2017

Code & Supply · 49.4K views · 2017-10-23 · 46:37

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Bryan Cantrill's Uptime 2017 talk on the engineering challenges of the stateful data path, including storage, firmware and hardware bugs. It uses the 'zebras vs. horses' diagnostic heuristic to explain why rare failures keep appearing.

On Hacker News

A perennial HN favorite whenever firmware, disks, ECC or opaque hardware come up, valued as both entertaining and sobering.

Recommended by 6 HN people across 6 months · HN mentions 2018–2025

Topics

Databases & storageTesting, debugging & developer toolsCloud, DevOps & reliability
Search by tags
bryan cantrillfirmwarestoragedata pathdebugging

What people said

thomasjudge · 2025-04 · Recommendation

For an entertaining/terrifying perspective on firmware, obligatory Bryan Cantrill talk "Zebras All the Way Down" [VIDEO]

Read the original HN comment ↗
Bayart · 2021-01 · Recommendation

>Why does anyone really care about non-free firmware blobs? Aside from the philosophical considerations, stuff breaking in a black-box firmware that you're lead into believing "just works" is particularly aggravating. I'll link Bryan Cantrill's talk on the subject, he's more entertaining than I am. [1] [1] [VIDEO]

Read the original HN comment ↗
ignoramous · 2020-02 · Recommendation

In my time writing databases, I found that not only the mount-flags could result in data-loss despite fsync, given you use fsync as it is supposed to be used [0], but even enterprise-grade hardware would at times (at that scale [1]) drop and/or corrupt a few bits here and there. That reminds me of the case where AWS famously revealed that a single NIC in their ginaromous S3 fleet flipped a single bit once in a while and that caused an outage because their gossip-daemon responsible for fleet health-checks failed spectacularly [2]. Bryan Cantrill's talk on the realted topic of hardware/firmware bugs is pretty good [3]. [0] [LINK] [1] [LINK] [2] [LINK] [3] [VIDEO]

Read the original HN comment ↗

Sources

Video by Code & Supply, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Topic keywords do not match
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources