HackerTube.dev

AI can't cross this line and we don't know why.

Welch Labs · 2.5M views · 2024-09-13 · 24:07

Open on YouTube ↗

If playback is restricted in your region, try opening the original video. Use Mark watched to add it to your marked videos.

Overview

A Welch Labs explainer on neural scaling laws and the compute-efficient frontier, asking why model loss seems to hit a line it cannot cross. It draws an analogy to an ideal gas law for AI.

On Hacker News

Recommended as a well-made, digestible introduction to scaling laws and why models cluster along a similar efficiency frontier.

Recommended by 2 HN people across 2 months · HN mentions 2024–2025

Topics

Machine learning & deep learningLLMs & generative AI
Search by tags
scaling lawscompute frontierneural networksloss curvesdeep learning

What people said

mdaniel · 2025-08 · Recommendation

I'm not deep enough in the ecosystem to ingest that paper but it's title reminded me of an interesting video I saw recently: "AI can't cross this line and we don't know why" [VIDEO]

Read the original HN comment ↗
kla-s · 2024-10 · Recommendation

Please someone correct me if I’m wrong, but my understanding of ML/LLMs is that this kind of hand crafting has been tried, but it is easier to train/less finicky to let behavior like this emerge from more data, see [1] “Bitter Lesson” and [2] “Scaling Laws”. MAMBA as an architecture claims to have some significant gains performance wise, but to my knowledge there haven't been any really large models (>~100B params) with open weights/leaked MAMBA architecture been disclosed other than this (7B). As mentioned by other comments, another dimension not to forget is the training data quality. Not only quantity but also quality really matters, is what we are learning more and more with LLMs.. [1] [LINK] [2] see eg [VIDEO] for a well made/easily digestable intro

Read the original HN comment ↗
general1465 · 2025-09 · Reference

Yep we do. There is a 1 year old video on YouTube, which describes this limitation [VIDEO] Called efficient compute frontier

Read the original HN comment ↗

Sources

Video by Welch Labs, played through YouTube.

Data and selection notes

HN summaries and labels may contain AI errors. HN dates refer to discussions, not video publication.

HN Library: HackerTube export (2026-10-31) and open-index/hacker-news. Upstream summaries and classifications may contain AI errors.

Views, duration and playback status refresh separately from the published catalog. When an update fails, existing data remains visible.

Selection checks
  • SaaS & AI candidates: Matches the configured criteria
  • Editor-approved picks: Editorial review is required

Rule matches identify candidates. They do not guarantee video quality or financial results.

About the library and its sources