Thought I was going to do a "Happy Lunar New Year" post instead here is some observations for #DeepSeek  :

DeepSeek-V2 introduced two major architectural innovations:
1. DeepSeekMoE (Mixture of Experts) - Instead of activating the entire model for every token, MoE architectures “route” inputs to specialized “experts.”
(a) lower inference costs
(b) increases training overhead due to the routing and load balancing for each expert.

2. DeepSeekMLA (Multi-head Latent Attention)
Traditional large language models require large key-value stores for each token in the context window; memory requirements grow quickly.

MLA compresses that key-value store, dramatically lowering memory usage.

This makes it easier (and cheaper) to perform inference, because you don’t need as much GPU memory to store all those tokens.

DeepSeek-V3
- demonstrated a dramatically cheaper approach to training state-of-the-art large language models:
- 671 billion parameters overall, but only ~37B parameters get activated (“expert” chosen) for each token.

Uses DeepSeek’s MoE and MLA optimizations (from V2), plus additional load-balancing and multi-token training strategies that squeeze more work out of each GPU hour.

Headline stat
The “final” training run took 2.788M H800 GPU hours, which DeepSeek priced at $2/GPU hour = $5.576 million total.

Cost shock:
The V3 training cost (a rumored $5.6M for the final run) is far lower than many would have guessed, suggesting the speed of commoditization of frontier models outpaces the the depreciation of heavy capex

Implications for Big Tech:
- OpenAI’s “o1” line was a major differentiator; now R1 is free for anyone to use (the weights are open).

Suddenly, the leading edge advantage looks ephemeral if other labs can incorporate these techniques and skip the huge training bills.

- Nvidia: Market is unsure how these new optimizations (which reduce total GPU spend) affect Nvidia’s long-term growth assumptions.

- Anthropic: Far less consumer traction than ChatGPT, and without a big product angle, is more vulnerable to open-source competition.

- Google: Might see a relative advantage (TPUs, synergy with search) erode as AI costs drop and new entrants proliferate.

- AWS/Meta/Apple: Potentially big winners; cheaper training/inference helps them meet their own AI needs more easily (Meta in ads and Reels ranking, Apple in on-device inference, AWS in hosting new open-source models, etc.).

Final thoughts :
- DeepSeek’s V3 proves that with MoE approach and memory compression (MLA), you can train huge models cheaply — especially if you’re forced (by sanctions) to optimize for limited GPU bandwidth.

R1 (and R1-Zero) shows that advanced “reasoning” capabilities aren’t uniquely locked behind closed labs like OpenAI.

- It's not far-fetched to project near-free inference where intelligence is infused into all applications in ways like how electricity is empowering all electronics.