Scalability at YouTube

What people said

gitarr · 2012-09 · Recommendation

YouTube (owned by Google) is mainly Python. Here a talk from Pycon 2012, the technical part starts around 10 min. in: [VIDEO]

Read the original HN comment ↗

teitoklien · 2024-05 · Reference

They improve their stack on multiple fronts 1 - They use their own custom hardware encoders now optimised for their scale (previously they’d use a ton of intel cpu for encoding) This is often not possible for smaller startups to have their own custom hardware, but one could setup their own Xilinx FPGA based hardware accelerated encoding servers, Xillinx Alveo Boards are popular for media acceleration, they support both ffmpeg and gstreamer for acceleration needs. Nvidia Nvenc encoding is decent too but mostly for the most popular encoding types, and honestly an fpga based accelerator is probably better after cpu unless you have gpus lying around. 2 - Much lower bandwith costs, a lot of startups build their streaming applications on top of AWS, but that’s insanely expensive due to the insanely expensive price of AWS itself As a streaming service you could immediately cut down costs by shifting to ovhcloud, hetzner etc, they often have dedicated gameserver like offerings which are beneficial for streaming usecases too. That itself should allow you to bring down costs by 10x or even more (you have to choose the plan correctly on other tho, ovh, hetzner etc say you have unlimited bandwith on some packages but there is always a catch or they throttle you later, need to be careful about the offering you choose but otherwise a lot better) 3 - Having colocation hosts with your own servers setup and each video encoded at different sizes for different media devices also help only sendi

Read the original HN comment ↗

Leftium · 2024-02 · Reference

For reference, YouTube runs on Python[1,2,3]: > 1. Python and Django: YouTube’s backend is predominantly written in Python, offering a balance of performance and readability. > 2. Google Cloud Platform... > 3. Java and C++: YouTube also utilizes Java and C++ for specific backend services, as they provide better performance for certain tasks. --- A long time ago, I looked into these Python frameworks: - [LINK] - [LINK] [1]: [LINK] [2]: [LINK] [3]: [VIDEO]

Read the original HN comment ↗