https://www.youtube.com/watch?v=YX6QdIYVSgY

无作者

DeepSeek just dropped the crown jewels. For the past year, the biggest mystery, in tech wasn’t just how DeepSeek trained their models. It was how they managed to, sell access to them so ridiculously cheap. While companies in Silicon Valley, were charging premium prices just to pay off their massive cloud bills, DeepSeek, was practically giving intelligence away. Most people in the industry, assumed it had to be a bluff.

bluff

[U, C] an attempt to trick sb by making them believe that you will do sth when you really have no intention of doing it, or that you know sth when you do not, in fact, know it 虚张声势的做法;唬人
It was just a game of bluff.
那只不过是唬人的把戏。
He said he would resign if he didn’t get more money, but it was only a bluff.
他说如果不给他加薪他就辞职,但那不过是虚张声势而已

The common theory was that DeepSeek was simply, burning venture capital

venture
a business project or activity, especially one that involves taking risks (尤指有风险的)企业,商业,投机活动,经营项目 SYN undertaking
A disastrous business venture lost him thousands of dollars.
一个彻底失败的经营项目使他损失成千上万美元

capital

[sing.] a large amount of money that is invested or is used to start a business 资本;资金;启动资金
to set up a business with a starting capital of £100 000
以 10 万英镑为启动资金创办一个企

and taking heavy financial losses just to undercut, Western models. Well, hours ago, DeepSeek published the actual numbers, from inside their data centers, and they released the secret software behind it.The numbers are genuinely staggering. In a single cluster, this system processes, 155 trillion tokens every single day. That comes out to roughly 1.8 billion, words generated every second around the clock. And even during their discounted, off-peak hours, a single GPU running their setup generates about $38 of.

revenue per hour. When an enterprise AI chip typically costs around 5 to $10 an, hour to run, DeepSeek isn’t losing money at all. They are operating at an, estimated 80% gross profit margin.

(business 商业) = profit margin
What are your average operating margins?
你的平均营业利润率是多少?
a gross margin of 45%
45% 的毛利

And to back it up, they officially, open-sourced the exact engine powering it, Deep Infer, along with the companion, gateway called DeepSeek Recipe. If you follow AI infrastructure, this is a, massive moment. A year and a half ago, DeepSeek told the community that, open-sourcing their production serving code was impossible because it was too, tightly tangled with their private data center setup. Now, they fulfilled that, promise and put the code on GitHub. In just its first 24 hours, the repository, racked up over 9,000 stars, and it’s already sitting near 20,000 as, developers scramble to see what’s inside. So, how did they actually pull, this off? Normally, when you send a massive prompt to an AI, like an entire, software code base, the model has to process every single word through every.

single layer of its network before it can write the very first letter of the, response. That reading phase eats up an enormous amount of computing power. To, picture how this works, think of an AI’s layers like stages on a factory assembly, line. A typical large model has around 40 stages and your text normally has to, pass through every single one before the AI can even begin to think about an, answer. DeepSeek solved this at the architectural level using what’s called, a causal encoder-decoder design. They took that 40-stage assembly line and, cleanly split it in half. The first 20 layers act as the dedicated reader and, the top 20 layers act as the writer. When your prompt comes in, it only runs, through the first 20 layers. The top 20 layers don’t even touch your text. Their, work is projected mathematically from the bottom half. Right out of the gate,that cuts the heavy reading work in half before the server even breaks a sweat.

On top of that, the model is built as a mixture of experts. While it has 748, billion total parameters, it only wakes up 8 billion parameters to read your, prompt and 16 billion to write the answer. Now, this is where the DeepInfra, software steps in with four clever engineering moves. The first move is how, the software divides the workload across graphics cards. In AI, reading your, prompt and writing your answer are two completely different problems. Reading, text takes pure math power, but writing the answer one word at a time is a, memory bottleneck. Every time the AI generates a single word, it has to fetch, billions of numbers, what engineers call weights, out of the GPUs memory chips, and load them into its processor. Think of it like this. If you force one person, to carry 100 heavy books across a room, they can barely walk. That’s what, happens when you run a giant AI on a small server. The chips spend all their, time waiting to fetch heavy data from memory. DeepSeek software fixes this.

with what they call wide expert parallelism. For reading prompts, the, software uses normal affordable eight GPU servers, but for writing the answer,it links together massive supernodes of 64 to 128 GPUs. Now, for anyone, wondering what an expert actually means here, think of it as dividing one giant, brain into 384 specialized mini departments, where each department, handles different topics. The software spreads those 384 experts across 128, chips, meaning each GPU only holds about seven of them. Instead of one person, carrying 100 books, you have 128 people holding just one or two books each.Memory traffic drops by nearly 88% allowing a single GPU to pump out over, 26,000 words per second. The second move is a software traffic cop called EPLB or, the expert parallelism load balancer. When you split an AI across 128 GPUs,you run into a very human problem. If thousands of people are asking the AI, coding questions, the coding experts get hammered with traffic while the creative.

writing experts sit completely idle. If one GPU is drowning in work and takes 50, milliseconds while the other 127 GPUs finish in 5 milliseconds, the entire, system has to sit there and wait for the slow one. To stop this, the software, monitors traffic constantly. The moment it detects an expert getting, overwhelmed, it dynamically spins up duplicate copies of that popular expert, onto idle GPUs. It’s like a grocery store instantly opening four new, checkout lanes the moment a line starts forming. Because the load is balanced in, real time, the cluster runs with virtually zero wasted idle time.

A cluster is a group or number of similar things, people, or units that occur, grow, or are gathered closely together
Common Definitions by Field
• General / Everyday Use: A small group of people or objects gathered in one spot (such as a cluster of houses, a cluster of trees, or people huddling near a door).
• Astronomy: An aggregation of stars, nebulae, or galaxies held together by gravity (such as an open star cluster or globular cluster like the Pleiades).
• Computing & Networks: Multiple individual computers or servers (nodes) linked together via a high-speed network to work as a single, unified system for load balancing and high availability

Next up, is raw speed. To handle this, Deep Infer uses a system called D-Spark. Now, while, D-Spark itself isn’t completely new, Deep Seek actually published research on, it earlier. Having it baked directly into the engine makes a massive, difference. What it basically does is handle speculative decoding. Instead of, writing one word at a time, a small ultra-fast helper model drafts the next.

five words in advance and the big AI verifies them all at once. The problem, with traditional speculative decoding is that the system always verifies all five, guessed words. If word number three is wrong, words four and five get thrown in, the trash, meaning the computer power spent checking them was completely, wasted. Instead of doing that blindly, Deep Seek added what they call a, confidence head to the drafter. As it guesses words, the software calculates, the odds that the big model will actually accept them. And if it notices, that the fourth word looks sketchy, it cuts the line right then and there,telling the big model to only check the first three words. So, by acting like a, disciplined gambler and folding bad bets early,

In “folding bad bets early,” fold means to give up or abandon a losing strategy early, especially when continuing would waste more time or money.

It’s a metaphor from poker. 🃏

  1. The original meaning of fold
    In poker, if you think your cards are too weak to win, you can fold—put your cards down and withdraw from the current hand.
  • Fold your cards = 弃牌
  • Fold early = 早点弃

it boosts overall cluster speed, by another 4 to 7% without dropping accuracy.The fourth move is how it handles conversation memory, and this solves a, huge real-world problem. Say you’re having a long conversation with an AI.

about a complex topic, or you’re working on a big weekend coding project. Keeping, track of that massive conversation history eats up an insane amount of, server storage. AI companies often have to clear that, memory to save space, because this is happening with millions of users at the, same time. The bottleneck comes when you step away for 2 hours and come back with, a follow-up question. The server usually has to reread that entire massive chat, from scratch just to answer you, which burns a ton of computing time. To fix, this, DC came up with a method called bounded replay. What they did is keep, that active conversation memory only in super-fast local RAM for about 3, minutes, which handles quick back-and-forth chatting. If you walk, away and come back hours later after that RAM has cleared, the server doesn’t, reread your whole massive project. Because of how the model is designed, it, only has to reread strictly the last 140 words. It’s like having a 500-page book.

where you lose your train of thought. You don’t start over from page one. You, just glance at a sticky note on the final page. Whether your prompt is 1,000, words or 1 million words, the time to catch up on a catch miss is basically, instant. Now, alongside the Deep Infra Engine, Deep Seek released a second, program called Deep Seek RSP. And because it feels like every piece of, software trying to go fast lately has to be written in Rust, they built this one, in Rust, too. Before a prompt ever reaches a GPU, your computer’s processor, has to turn raw text into numbers, a process called tokenization. For massive, prompts, this usually creates a noticeable lag before the AI even starts, thinking. DeepSeek’s Rust program splits the prompt along structural boundaries, and tokenizes multiple chunks in parallel across all CPU cores, simultaneously. At the same time, it acts as a security shield against prompt, injection attack. If a user tries to inject malicious system commands.

directly into the chat to hijack the AI, the Rust program forces that input to be, encoded strictly as plain text, completely neutralizing the attack.Then, there is the hardware story, and this is where the geopolitics come in.DeepMind for explicitly supports two hardware platforms, Nvidia’s brand new, Blackwell architecture and Huawei’s domestic Ascend 950 chips. Notice who is, missing? Nvidia’s Hopper chips, like the H100. DeepSeek bypassed the previous, generation entirely. Instead, they built this software to run natively on, domestic Chinese silicon with first-class performance. Amidst, tightening US trade restrictions, DeepSeek is showing the world that their, 155 trillion token operation can run at frontier scale on domestic hardware,completely independent of Western silicon if necessary. Now, as exciting, as all of this sounds, there is a catch you need to know about. DeepSeek’s own, documentation is very upfront about this, with a section literally listing.

the parts they left out. DeepInfer gives you the engine, but DeepSeek definitely, didn’t hand over their entire factory. They kept three massive pieces behind, closed doors. Their 100 petabyte storage cluster that pushes 10 terabytes of data, per second, their specialized network routing system, and their global, internet load balancer. To put it simply, they gave the world a, high-performance commercial oven, but they kept the automated industrial, bakery line to themselves. To actually get the kind of speed we’re talking, about here, you still need a serious enterprise setup of at least 128, graphics cards running their specific flagship model DeepSeek V4.1 flash.Looking at the the picture, this is a massive win for all of us. For the last, 2 years, the narrative has been that whoever spends the most money buying the, most GPUs wins. But Deep Seek has been showing us repeatedly that smart systems, engineering beats brute force spending every single time. This puts serious.

pressure on cloud providers that charge high markups just to host open-source, models. And developers working on popular community engines like vLLM are, definitely already digging through this code to bring these speed tricks into, tools everyone can use. The competition isn’t just about who can build the, smartest model anymore. It’s about who can deliver that intelligence at the, lowest possible cost. And for sure, right now, Deep Seek is writing the, playbook. Let me know in the comments, do you think Western AI companies will, finally be forced to lower their prices after this? Personally, I really hope, they do. Either way, if you enjoyed breaking this down with me today, a like, and subscribe always means a lot and keeps the channel rolling. Thanks for, watching all the way to the end. Stay safe, and I’ll catch you on the next, gem.