deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Chinese LLM tool calling beats GPT-5.5 on benchmarks but migration is no config swap

A dev.to comparison of ten models from DeepSeek, GLM, Qwen, Kimi and MiniMax finds OpenAI-compatible tool calling everywhere, but streaming, parallel-call and error-handling differences force per-model adapters.

Chinese LLM tool calling beats GPT-5.5 on benchmarks but migration is no config swap

A systematic comparison of ten production models from five Chinese LLM families concludes that OpenAI-style tool calling has become table stakes among Chinese providers — and that this surface compatibility hides enough divergence to trip up production agents.

According to the comparison published on dev.to, every tested variant — DeepSeek-V4-Pro; GLM-5.1 and GLM-5.3; Qwen3.5, Qwen3.6-Plus and Qwen3.7-Max; Kimi K2.5, K2.6 and K3; and MiniMax M3 — accepts OpenAI-format functions and tools schemas, including tool_choice and multi-turn tool-result messages, with GPT-5.5 used as the compatibility baseline. All ten were run through a unified harness with identical tool schemas, prompts and mock downstream services, scored across six dimensions: protocol compatibility, JSON payload schema fidelity, parallel tool-call support, streaming tool-call parsing, error recovery and overall integration effort.

Three deployment tiers

The comparison splits the field into three groups. DeepSeek-V4-Pro, GLM-5.3 and the Qwen3.x line are hosted APIs that take standard function schemas directly; GLM-5.3 and DeepSeek-V4-Pro also offer one-million-token context windows with adjustable reasoning effort. The Kimi K2 family is the open-weight option — a one-trillion-parameter mixture-of-experts model with 32 billion active parameters, with deployment paths documented for vLLM, SGLang, KTransformers and TensorRT-LLM. Qwen3.7-Max sits in a tier of its own: closed and reachable only through Alibaba Cloud Model Studio in mainland China, which the author flags as a data-residency concern for EU and US enterprises.

Where compatibility breaks

Protocol acceptance is not the problem. The comparison pinpoints three areas of real divergence. First, parallel tool calls: some models emit only one call per assistant turn, which changes orchestration assumptions. Second, streaming deltas: providers differ on whether tool calls arrive as incremental token fragments or as whole JSON objects per chunk, a distinction that directly shapes parser design. Third, MCP adapter quality varies significantly between providers, so teams using MCP as their tool layer should test each implementation individually.

Model-specific quirks add friction. MiniMax M3 handles basic OpenAI-format calls but appends custom extension fields that need extra parsing. Kimi K2.6 accepts the standard payload yet has historically emitted non-standard streaming tool-call deltas in some versions, pushing teams toward version-specific parsers.

Router layers do not resolve this. As tested in the comparison, OpenRouter, Requesty and Eden AI smooth over the wire protocol but not the meaning of tool-call payloads — they will not repair schema or streaming incompatibilities for you.

Benchmarks and long-horizon behaviour

On the BenchLM tool-use suite cited by the comparison (updated August 21, 2026), GLM-5.1 and MiniMax M3 lead at roughly 70.1, ahead of GPT-5.5's 67.8, with Qwen3.7-Max close behind at 68.8. Kimi K2.6 scores 60.5 and legacy GLM-5 trails at 58.3. The author also cites an Alibaba demonstration in which Qwen3.7-Max ran a single task unattended for 35 hours across 1,158 cumulative tool calls, suggesting strong context retention over long agent loops — with the same mainland-China hosting caveat attached.

Practical guidance

For teams migrating agent stacks, the comparison recommends writing a per-model adapter for tool-call parsing rather than attempting a base-URL swap in production; picking GLM-5.1 or MiniMax M3 when benchmark accuracy is the priority; choosing Kimi K2 (with a pinned, known-good version) for self-hosting or strict data control; and placing schema validation and retry logic in the application layer to handle malformed JSON, downstream exceptions and empty results. Routers should be used for endpoint routing and failover only. A closing migration checklist covers validating tool schemas against each target model, testing parallel-call behaviour, capturing raw streaming chunks for per-provider comparison and verifying MCP adapters.

Why it matters

The article frames a decision now facing many agent developers: Chinese models reportedly match or exceed GPT-5.5-level tool-use accuracy at a fraction of the cost, but only for teams that budget real engineering work for compatibility instead of assuming a configuration change. The hidden expense is not the API — it is parser stability, streaming handling and error recovery around each model, and routers will not absorb that cost for you. Since these findings come from a single comparison, the benchmark figures are best treated as a starting point, with teams running their own harness tests before committing to a provider switch.

  • #llm
  • #tool-calling
  • #ai-agents
  • #openai-api
  • #qwen

Related posts