HackerTube.dev

Thesis: Partial State in Dataflow-Based Materialized Views

Jon Gjengset · 17K views · 2020-10-23 · 45:50

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

Jon Gjengset's 2020 PhD dissertation presentation at MIT on partial state in dataflow-based materialized views, the idea behind the Noria database. The Q&A is cut out.

On Hacker News

HN points to it, including Gjengset himself, as a more digestible way into partially-stateful dataflow than the thesis text, mostly in caching and database threads.

Recommended by 1 HN people across 1 months · HN mentions 2020–2024

Topics

Databases & storageUniversity lectures & courses
Search by tags
noriapartial statedataflowdatabasescaching

What people said

Jonhoo · 2022-11 · Recommendation

:wave: Author of the paper this work is based on here. I'm so excited to see dynamic, partially-stateful data-flow for incremental materialized view maintenance becoming more wide-spread! I continue to think it's a _great_ idea, and the speed-ups (and complexity reduction) it can yield are pretty immense, so seeing more folks building on the idea makes me very happy. The PlanetScale blog post references my original "Noria" OSDI paper ( [LINK] ), but I'd actually recommend my PhD thesis instead ( [LINK] ), as it goes much deeper about some of the technical challenges and solutions involved. It also has a chapter (Appendix A) that covers how it all works by analogy, which the less-technical among the audience may appreciate :) A recording of my thesis defense on this, which may be more digestible than the thesis itself, is also online at [VIDEO] , as well as a shorter talk from a few years earlier at [LINK] . And the Noria research prototype (written in Rust) is on GitHub: [LINK] . As others have already mentioned in the comments, I co-founded ReadySet ( [LINK] ) shortly after graduating specifically to build off of Noria, and they're doing amazing work to provide these kinds of speed-ups for general-purpose relational databases. If you're using one of those, it's worth giving ReadySet a look to get these kinds of speedups there! It's also source-available @ [LINK] if you're curious.

Read the original HN comment ↗
dpcx · 2024-02 · Reference

Pretty sure that this is the database that Jon Gjengset[0] was working on as part of his thesis project. There have been several videos shared by him during talks about the system. It's a really interesting concept. edit: Here's[1] a video where he talks about the concept [0]: [LINK] [1]: [VIDEO]

Read the original HN comment ↗
kondu · 2020-10 · Reference

this is also one of the main ideas behind Noria[1] (built by @jonhoo, who also gave his PhD dissertation[2] on this recently) [1] [LINK] [2] [VIDEO]

Read the original HN comment ↗

Sources

Video by Jon Gjengset, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Topic keywords do not match
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources