Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017

GOTO Conferences · 100.9K views · 2017-05-10 · 52:35

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Bryan Cantrill's GOTO Chicago 2017 talk on debugging production systems under pressure, centered on the incident where an operator's mistyped command rebooted an entire Joyent datacenter. It covers human factors, automation and architecture.

On Hacker News

A perennial HN recommendation for outage and postmortem threads, praised as funny, educational and one of the best postmortems anyone has seen.

Recommended by 8 HN people across 11 months · HN mentions 2018–2025

Topics

Cloud, DevOps & reliability Testing, debugging & developer tools

What people said

KronisLV · 2023-03 · Strong recommendation
The best postmortem I've seen was actually a conference talk: "Debugging Under Fire: Keep your Head when Systems have Lost their Mind" by Bryan Cantrill (in the GOTO conference, 2017) Here's a YouTube video of it: [VIDEO] Here's the slides: [LINK] It goes into detail about a pretty bad outage (when an entire data center was brought down), the human aspects, automation, how they handled it, the various risks, architectures, how things fail and about software development in general.
Read the original HN comment ↗

int0x2e · 2023-01 · Strong recommendation
Time to share one of my favorite talks (and speakers) ever - "Debugging Under Fire: Keep your Head when Systems have Lost their Mind" (Bryan Cantrill, GOTO 2017) [VIDEO]
Read the original HN comment ↗

OmarAssadi · 2022-01 · Strong recommendation
Perhaps not famous, but Bryan Cantrill, who gives my favorite talks, has an interesting and funny talk on one of the Joyent outages: [VIDEO]
Read the original HN comment ↗

Sources

Video by GOTO Conferences, played through YouTube.

HN Library · Source license

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks

SaaS & AI candidates: Topic keywords do not match
Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources