What should we make of DeepSeek v3?

DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37B active parameters, or that cost under $6 million.

According to the benchmarks, it can play with GPT-4o and Claude Sonnet.

Anecdotal reports and alternative benchmarks tells us it's not as good as Claude Sonnet, but it is plausibly on the level of GPT-4o.

So what do we have here? And what are the implications?

Table of Contents

What is DeepSeek v3 Techncially?.
Our Price Cheap.
Run Model Run.
Talent Search.
The Amazing Incredible Benchmarks.
Underperformance on AidanBench.
Model in the Arena.
Other Private Benchmarks.
Anecdata.
Implications and Policy.

What is DeepSeek v3 Techncially?

I’ve now had a [...]

---

Outline:

(00:39) What is DeepSeek v3 Techncially?

(01:56) Our Price Cheap

(02:33) Run Model Run

(04:57) Talent Search

(05:22) The Amazing Incredible Benchmarks

(07:23) Underperformance on AidanBench

(12:59) Model in the Arena

(13:27) Other Private Benchmarks

(15:05) Anecdata

(23:57) Implications and Policy

The original text contained 9 images which were described by AI.

---

First published:
December 31st, 2024

Source:
https://www.lesswrong.com/posts/NmauyiPBXcGwoArhJ/deekseek-v3-the-six-million-dollar-model

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“DeekSeek v3: The Six Million Dollar Model” by Zvi