← AI News & Strategy Daily with Nate B. Jones
AI News & Strategy Daily with Nate B. Jones
Stop guessing whether a cheaper model can do the job.
00:24:00
Which Chinese AI models are worth using? The test I run on DeepSeek, Qwen, GLM, Kimi, and MiniMax, and how I count what an accepted result really costs.
Loading summary
Transcript1 lines
- [00:00]
A
It's something that everyone has on their lips right now. We've had this moment since Deep Seq where it feels like China has been coming on. There's been a lot of narrative recently with the Release of Kimi K3 and Quinn 3.8. Basically everyone's asking, are these models caught up? Do we see the model frontier actually closing? I want to take this video and just talk about the Chinese model phenomenon. What it looks like when you're deep in in testing and you test Chinese models, how they differ from US models, how you should approach thinking about Chinese models as an individual, maybe as a company. So we're going to get into all of that detail in this video. I'm really excited for it. I don't think Chinese models get the attention they should get, and I wanted to make a dedicated video just to jump into the whole Chinese model world. Last week Moonshot AI released Kimike 3, a Chinese model with 2.8 trillion parameters and a million token context. Windows is API charges a fair bit, right? $15 per million output tokens. Meanwhile, Deepseek v4 Pro charges just $0.87 for the same unit of volume. Moonshot also calls K3 open weight. Now, as I record this, the weights are not downloadable, but apparently very, very soon, maybe by the time this video is out, you will have the full weights released. Both Deepseek and Kimi and also Quinn are Chinese frontier models. The they don't share a price, they don't share a deployment path or a hardware burden or even a best use. They just share a country of origin. And that's one of the things I want to emphasize at the top here. We say Chinese models and we make this big simplifying assumption. That's the problem with the Chinese model conversation. We use Chinese model sometimes as a shorthand for cheap or for open or for local. Sometimes all three are true, and sometimes none of them are. This video is going to dive deeper. It's going to give you a practical way to choose whether to include a Chinese model in your stack. Which work should you give them? When do you use an API? When is self hosting justified? I'm going to get into the distillation fight because it could change how quickly capability spreads. I'm going to talk about which models remain and are likely to remain available regardless. Now, before we go too far, my answer up front is yes. So serious AI users should test Chinese models, but you need to plan to make separate decisions about the task, the model and the deployment path for your model. So let's not Conflate those and I'm going to get into all three of them here. If you are tackling high volume, price sensitive API work, DeepSeek absolutely belongs in your test set. It has an output price of, I think, yeah, $0.87 per million tokens. And it can change the economics of document processing, of code generation, of research pipelines, anything at volume. When you can cut the price down that much 15x20x30x from whatever you're using now, it's a big deal. So if you're looking at extraction, if you're looking at classification, if you're looking at first pass research at test generation, any other reviewable work in that line, the low price matters a ton because a machine can afford to check and recheck and check again and you can get really good quality at a really affordable price. Now, I would be more cautious about using a tool like Deep Seq when one ambiguous answer drives an unrecoverable action, right? If there's something you can't easily verify, then you want to think about whether you are trading down quality in a way you can't accept. If you're looking at local work, if you're on personal hardware like this laptop right here, you want to look at some of the smaller Quinn models. Some of the models Deep seq distilled from R1. These are releases that are behind the real local AI story. They're not the same artifact as full deep seq R1, which had 671 billion total parameters, or deep seq V4 Pro, which has 1.6 trillion local models make sense for private notes, for offline work, for predictable availability and workflows where you would rather trade some capability for a high, high degree of control and privacy. A useful comparison might be maybe not a small Koen model against Kimike3. It's more whether the small model is good enough for for the exact job you want to keep on your machine. For long horizon coding research agent work, I would test a different set of Chinese models. I would look at the glm, Kimi, Quinn and Minimax against the best American frontier models using the same tools in the same acceptance standard. Don't change your bar. Some Chinese models are close to the frontier on selected tasks. GLM 5.2 on coding is particularly strong, but that doesn't mean that they are automatically universal replacements in that domain. I would not use GLM 5.2 for every task task in the coding universe just because it happens to be good at coding. A model can write an excellent patch, right? It can still lose a repository level task because it used the wrong tool or missed a test or failed after 20 minutes. And I'm asking you to test because these things matter, right? A model might be strong at research in your area and weak at visual reasoning for you. Frontier becomes a useful phrase only after you understand what you're actually testing, so you can measure where the frontier is for you. This is why benchmarks are notoriously easy to game and hard to trust for sensitive work. I would decide where the data may go before I choose the model. A Chinese model running on your own server creates a completely different risk profile from the same model accessed through a first party chat service in China. Those are very different decisions, right? Let's start with Deepseek at a slightly deeper level. Pun intended. Deepseek is pursuing stream inference economics. Quen is a broad family that runs from genuinely small open models to a separate hosted Macs line whose weights are closed. Z AI's GLM 5.2 is MIT licensed and very strong on long horizon coding, but the released BF16 checkpoint is really large. It's like one and a half terabytes. Moonshot is pricing Kimik 3 as a premium frontier product. Minimax M3 combines coding and computer use and image and video understanding under a custom license that requires attribution and notice and requires prior written authorization above $20 million in annual revenue and prohibits military use. Sound different? Because they are. They're entirely different products and different strategies. Open is not one property either. You need to ask a bunch of separate questions to understand what open really means. Can I call the model through an API? Can I download the weights? Does the license permit my use? Can my hardware run it at the context length that I need at the speed I need? A model can be open on one dimension enclosed, or simply impractical on a bunch of the rest. Now, one technical reason enormous models can still be economical is this concept of a mixture of experts. Deepseek V4 Pro has 1.6 trillion total parameters. Minimax M3 has about 427 billion total and just 23 billion active parameters per token. The router does not use the whole network for every token, and part of the magic is in picking the right parameters to activate for a given token. And active parameters help explain how we handle compute per token. Total parameters still determine much of the storage, the memory, the networking, and the serving burden. And that's why a model can be inexpensive through an optimized API and absolutely absurd to run on your laptop. So don't turn the active count into a simple hardware estimate. It's more complicated than that. You have to ask about checkpoint size, about precision, about context requirements, and about serving topology. And if all of that sounds really complicated, well, it kind of is. And that's exactly the advantage that frontier models have in the states is that you don't have to think about that stuff if you're just signing up for OpenAI. And that gives us one of our first hosting rules. If you want to run a score smaller model locally, when the checkpoint fits your hardware and the lower capability is enough for the task, absolutely you can do that. If you want to host a giant open weight model, make sure that you have hardware and data control and portability and you're set up to handle the scale you're talking about. Downloading the weights does not magically solve the hardware and team requirements. You actually need to serve what is effectively an enterprise model from China. Hardware and licensing can tell you what you have to deploy to get a model up, right? So that's a part of it. But even then they don't tell you whether the output is going to be good enough for that. The gap has to be measured against the work that you are doing. Chinese labs can report near frontier or leading scores on coding and math and tool use and long context tasks, but that doesn't mean they work for you, right? You have to think about your harness, about your reasoning budget and about your tools. And if you're not ready to go to that level of detail, you're probably not ready for a Chinese model conversation. In May, the US government's CAISI evaluation called DeepSeq V4 Pro, the most capable Chinese model it had tested, while estimating that it remained about eight months behind the leading US Frontier. That capability gap was smaller on individual tasks, which tells us the model is spiky and reinforcement learned in specific areas. And of course, the economics are super different, right? Deep Seek would cost less, which is what everybody expects, while in other tasks the cheaper tokens didn't end up producing the cheaper solution because either it had to spend so much tokens to get there or it never got there at all. And so what CAISI found is that Deep seq ranged from 53% cheaper to 41% more expensive per correctly solved task across seven benchmarks. Have I said testing enough? You gotta test, otherwise you won't realize savings here. Deepseek has changed their prices since as well. So those percentages are not a permanent verdict. They're an indicator of where you need to test. They're proof that token price and finished work cost can often point you in opposite directions. And almost everybody cares about the finished work cost. Those results remind us why cost per accepted result is the gold standard. And you have to define your task clearly enough so you can do that. A cheap model can become expensive when it produces longer reasoning traces, which is literally what we see with Kim and K3 versus Fable 5 when it makes unnecessary tool calls, when it fails late or hands a human more cleanup. A premium model can end up being cheaper like Fable 5 if it succeeds in one pass and actually Opus 5 has just come out. And it has the same token efficiency focus, which is how frontier models are starting to compete right now. So if we back up a bit for bounded, repeatable, high volume jobs, Chinese models can be an extraordinary value for ambiguous work where the model has to infer intent or recover from surprises or make a high stakes judgment call. The strongest American frontier systems remain the baseline that I would assume you have to beat to justify. Leading American services often come with more mature tool integrations with enterprise controls, with operational evidence. Some Chinese offerings will counter, and they'll often counter with things that we feel like we hear a lot in the narrative right in the media. Lower prices, more open weight options, more deployment choice. And neither bundle is going to work for everybody. The relative value changes when your job values raw judgment or values control or values portability or values cost. Now there are two more complexifiers I want to mention here. How does capability move into the model and where does your data go? And that takes us into policy and distillation. And it begins with those restrict specified advanced chips, manufacturing equipment, software, end users and end uses different from users without telling us which hardware trained every model. I would not say trolls invented Chinese efficiency. I think that Chinese efficiency is something that we see across a wide range of Chinese technologies over the last two or three decades. Mixture of experts, low precision training, sparse attention, quantization and better hardware utilization are techniques that we do see in Chinese models that reflect that broader tendency. And the controls simply increased the strategic incentives to get more capability. The Americans don't have to worry about chips, they have to worry about power. China is pushing at the opposite side of the system in most of these incentive sets versus what American AI companies are doing. Its AI plus policy supports open source communities in the sharing of models, tools and data sets. That doesn't prove that Beijing is ordering a lab to release weights. It's just a part of the community there. The labs have commercial reasons to build an ecosystem. While official policy can support Broad adoption Open weights can also reduce users dependence on American APIs, although the policy doesn't state that motive explicitly. Now Distillation, which has been a very hotly debated topic. In fact, Kimmy K3 was publicly accused of distilling Fable 5 weights just last week. Distillation connects those stories. In the normal version, a powerful teacher model generates examples, answers, reasoning, traces or preferences. Another model, often smaller and more specialized, trains on those outputs and inherits some of the teacher's behavior without reproducing the teacher's full training program. OpenAI actually sells distillation as a product. Frontier Labs use it internally. There is a Mythos teacher model that teaches the released versions of Mythos and Fable. Deep seq openly used R1 generated samples to train smaller models based on QEN and on llama. Its repository says that Qin derived student models were fine tuned on 800,000 samples curated from R1. That is super ordinary disclosed teacher to student model training for AI. A distilled student is not the full teacher squeezed into a smaller file. It can inherit useful reasoning patterns on the kinds of examples it saw, but it loses breadth, reliability and safeguards. This is why we talk about the idea of a big model smell for models that are teacher models. That is another reason the phrase Deep SEQ runs locally. The local student and the full frontier teacher are very different, so the systems the controversy about distillation begins when the teacher's outputs were never authorized for that use. Case Anthropic says that Deepseek, Moonshot and Minimax used about 24,000 fraudulent accounts to generate more than 16 million Claude exchanges while evading access restrictions. A later White House memo described industrial scale extraction by China based entities but did not name those labs or validate Anthropics numbers. Now that's since changed because Kimik3 was named by the White House just last week. And these are very serious allegations. Not court findings, right, but allegations. And the public record doesn't tell us how much release capability came from the alleged campaigns that Anthropic is describing. I think a useful distinction here is authorization and conduct. A lab can distill its own or a licensed teacher. It can train on API outputs contrary to contractual restrictions or use fake accounts and access control evasion at scale. The technical method may overlap, but the contractual and potential legal claims really differ in this situation. And typically Chinese entities are not authorized to distill American models. And this is why the fight matters even if you want to reserve judgment on the companies. Chips are physical and model outputs are information restricting A teacher model's GPU doesn't control everything that that model can teach through an API, right? This is reminding me as a graybeard of back in the 90s when we talked about Napster and music wanting to be free. Capability can move very easily across the global Internet. It can move through synthetic data, through fine tuning, through reinforcement, learning, through distillation. The transfer may not be perfect, but it may not need to be right. It can reproduce enough valuable behavior fast enough and cheaply enough that it's worth doing. And that means today's capability gap can move faster than hardware policy alone would suggest. And it gives frontier providers strong incentives to tighten their access, their monitoring and their account controls, which is exactly what we see with Anthropic and the Moonshot program, for example. Now, for a customer, that absolutely increases the value of owning your own evals and preserving an exit path. If you think about it as this creates uncertainty, a workflow is much less exposed to one provider when the prompts, the tests and the tools can easily be lifted and shifted to another model. Now let's talk about the hosting question directly. There are three basic deployment choices that you have, right? You have a first party API that's the easiest. But then the provider controls the service, the provider controls the the price, the logging, the availability. Now a third party host can run open waits in a region and contract any way you prefer. Self hosting gives you the most control over prompts and runtime behavior, but it makes you responsible for hardware, for security, for updates, for monitoring, for everything that goes with having a model and self host only makes sense I think when four things are true. The weights need to be available and the license needs to be permit your use. Like that's fairly intuitive. I think the data or sovereignty requirement has to be real enough that you need to have the model on your premises. The workload should justify the economic infrastructure. You need the AI infrastructure to run, and you need to have a team that can operate the stack safely. If those conditions aren't present, you're going to have to look at either an API or a managed third party deployment, and that's usually going to be a better solution for you. Self hosting also needs a named owner for some of the high liability stuff. And so that gets at the team site, right? For authentication, for model server patches, for capacity and observability. Without a team that has that kind of accountability, you're just sort of converting your vendor risk into an operating problem that you may only discover when something goes wrong. So don't assume that Chinese model means the data path or the governing regime is set. China's interim rules govern generative AI services offered to the public inside mainland China. They don't automatically govern raw model weights privately operated outside China. So Deep Seq's first Party privacy policy says personal data is processed and stored in China and inputs may be used to improve its technology. That's not surprising. Alibaba says Model Studio customer data is not used for model training and offers deployment scopes that can exclude mainland China. And that's something not a lot of people know. And you can go even farther here, right? An American third party host running Quinn weights creates yet another path, right where that would be under American jurisdiction. A properly configured self hosted model can keep prompts away from the original lab and remove provider side runtime filters, but it does not remove refusal behavior or political bias learned in the weights. You may have to fine tune, right? You may have to check inference calls, tools, telemetry and logs to figure that out. And then a lot of people run a fine tune on a Chinese model to adjust it for that American sensibility. Moving the weights onto your server changes who can see the data. It doesn't make the rest of the software supply chain automatically trustworthy by magic. Just to be clear. First, I name the job. Is it bounded volume work? Is it local assistance, Is it sensitive internal work, or is it frontier judgment? I do not choose a model before I choose what I can tolerate from a failure perspective. Second, I name the artifact in the deployment that I'm expecting. Is it an API? Is it weights I can download? Is some kind of promised release that's coming in the future? What license applies to all of that? What. What does the full model require at a realistic context length for my work? Third, I measure cost per accepted result, which we talked about earlier. I include input and output and reasoning and tool calls and retries and latency, the entire chain and the infrastructure. When I measure that, I don't want hidden costs here. Fourth, I trace the data and the exit path really clearly. I identify which company receives the prompt, where it's stored and processed, the governing law and contract there, and the retention and training terms. I then ask whether I can move the workflow and keep my prompts, my tools, my evaluations, if the provider changes its price, whether I can keep my model and access policy. Those are all things that you have to think about if you're thinking about taking a serious bet on Chinese models. Now, if you're just an individual, you can run this test without building a Laboratory, right? You can take 20 examples of real work. You can include ugly edge cases, you can pick a Chinese model and you can run a test through it and find out whether it works. And I did this, right? I did this when I was building the Ringer multi agent that I launched a week or two ago, where you can actually have multi agent swarms that deliver frontier level intelligence for much less than frontier cost, where fable orchestrates and much cheaper models actually do the work. And what I found was quite quen was really helpful as part of an agent swarm. And so by testing I determined that for a lot of use cases, QEN was a helpful swarm agent participant. But of course I configured Ringer so you can use it however you want. Similarly, you need to do your own testing. You need to decide where you need frontier model intelligence, where you would put GLM or Kimi or Qin or Minimax on your particular work. And absolutely, you need to think about sensitive workloads. You want to think about where is the data architecture that allows you to preserve your liability in a situation where people may expect that you processed your data using an intelligence based in your jurisdiction. So should you use Chinese models? My answer is yes, selectively. I mean, I do and sometimes aggressively. Use them as specialists, use them as challengers, use them in some workflows as the default. But don't be sloppy in your analysis. Don't use Chinese as shorthand for a particular capability. Score for a particular security verdict for a typical deployment plan. Start with the work. Check what you're actually doing. Make sure you understand the full hardware burden of what you're going after. Understand the data path, understand the real cost for the work you want done. Country of origin is just the very beginning of having a conversation about due diligence. It doesn't substitute for the full testing regime that you need to do. So if you're serious about doing Chinese model work, you need to take the responsibility to do that testing. Now, if you want to see how I tested how I think about these models, I have way more detail in the article for today. I know it's a hot topic. So I have a deep dive on these models and I will show you, if you're interested, how to put them into that multi agent swarm approach, that Ringer approach that I used and introduced a couple of videos ago. I hope you had fun with this one. And I will be back soon with whatever's next in the world of AI.