fix(openclaw): configure 64k completion headroom for deepseek-v4.1-flash, promote cloudflare, and onboard fireworks
This commit is contained in:
@@ -90,8 +90,11 @@ How do we architect the full-spectrum model selection across **all modalities**
|
|||||||
* **Primary (Fleet-Wide Standard as of 2026-10-17)**: `openrouter/deepseek/deepseek-v4.1-flash-1017`
|
* **Primary (Fleet-Wide Standard as of 2026-10-17)**: `openrouter/deepseek/deepseek-v4.1-flash-1017`
|
||||||
- **1017 Snapshot Upgrade (Pareto Jump in Quality & Price)**: Released on October 17, 2026, `deepseek/deepseek-v4.1-flash-1017` boosts the Artificial Analysis Intelligence Index from **24.7 to 39.5** (+60%) via Reinforcement Learning with Self-Feedback (RLSF) while cutting DeepInfra `fp8` pricing by an additional **30%** (`$0.098/M` prompt, `$0.294/M` completion, and **`$0.00294/M` `cacheRead`** vs. `$0.140/$0.420/$0.0042` on the 0910 baseline).
|
- **1017 Snapshot Upgrade (Pareto Jump in Quality & Price)**: Released on October 17, 2026, `deepseek/deepseek-v4.1-flash-1017` boosts the Artificial Analysis Intelligence Index from **24.7 to 39.5** (+60%) via Reinforcement Learning with Self-Feedback (RLSF) while cutting DeepInfra `fp8` pricing by an additional **30%** (`$0.098/M` prompt, `$0.294/M` completion, and **`$0.00294/M` `cacheRead`** vs. `$0.140/$0.420/$0.0042` on the 0910 baseline).
|
||||||
- **Provider Pinning & Tool-Compatibility Verification**:
|
- **Provider Pinning & Tool-Compatibility Verification**:
|
||||||
- Pinned `order: ["deepinfra", "inference-net", "parasail", "cloudflare"]`, `quantizations: ["fp8", "bf16", "unknown"]`, and `ignore: ["morph", "baseten", "deepseek", "open-inference", "decart", "sail-research", "wafer"]`.
|
- Pinned `order: ["deepinfra", "cloudflare", "inference-net", "fireworks", "parasail"]`, `quantizations: ["fp8", "bf16", "unknown"]`, and `ignore: ["morph", "baseten", "deepseek", "open-inference", "decart", "sail-research", "wafer"]`.
|
||||||
- Live OpenRouter routing forensics verified that `morph/fp8` is filtered out by OpenRouter's `Filter by Tool Compatibility` when `tools` are passed, `deepseek` (official) is filtered by account privacy guardrails (`Paid model training violation`), and `baseten` returns `HTTP 429`. Meanwhile, `deepinfra`, `inference-net`, `parasail`, and `cloudflare` deliver 100% tool-calling reliability and 2–5s latency.
|
- **Cloudflare (#2 Promotion)**: Promoted ahead of InferenceNet and Parasail because Cloudflare delivers a 97% cache discount (`$0.003/M` `cacheRead`), 3× lower completion costs (`$0.60/M` vs. `$1.20/M` on Parasail), and a massive **943,718** completion token limit.
|
||||||
|
- **Fireworks (#4 Addition)**: Onboarded as a high-reliability enterprise tier-1 provider (`$0.30/M` prompt, `$1.20/M` completion, `$0.006/M` `cacheRead`, 100% uptime, 943k max completion tokens, full tool choice and structured outputs support).
|
||||||
|
- **Completion Token Headroom Guardrail (`maxTokens: 65536`)**: Because reasoning models share the completion token budget between internal chain-of-thought (`reasoningTokens`, typically 2k–4k tokens) and the actual tool-call payload, omitting `maxTokens` previously allowed OpenClaw/OpenRouter to default to an 8,192 token ceiling. During large multi-kilobyte code or website generation, hitting 8,192 tokens abruptly truncated tool-call JSON mid-payload (`Provider returned an incomplete or malformed tool call`). Explicitly configuring `maxTokens: 65536` and `params.max_tokens: 65536` provides ample headroom for complex reasoning and large outputs.
|
||||||
|
- Live OpenRouter routing forensics verified that `morph/fp8` is filtered out by OpenRouter's `Filter by Tool Compatibility` when `tools` are passed, `deepseek` (official) is filtered by account privacy guardrails (`Paid model training violation`), and `baseten` returns `HTTP 429`. Meanwhile, `deepinfra`, `cloudflare`, `inference-net`, `fireworks`, and `parasail` deliver 100% tool-calling reliability and sub-second to 3s latency.
|
||||||
- **Post-Trial Evaluation of `z-ai/glm-5.3-flash` (2026-09-13 to 2026-10-17)**:
|
- **Post-Trial Evaluation of `z-ai/glm-5.3-flash` (2026-09-13 to 2026-10-17)**:
|
||||||
- While `z-ai/glm-5.3-flash` was trialed as primary conductor, 17-day production telemetry (Oct 1–17, 2026) revealed **12 mid-stream tool-call aborts** (`stopReason: "error"`, `errorMessage: "Provider returned an incomplete or malformed tool call"`) across 5 agents (`main`, `grabengasse_media`, `koch`, `hebamme`, `doktor`) when generating nested `tool_call({ id, args })` payloads after `tool_describe`, as well as occasional thinking-only whitespace stalls (Issue 23) and long-context attention drift (Issues 22 & 25).
|
- While `z-ai/glm-5.3-flash` was trialed as primary conductor, 17-day production telemetry (Oct 1–17, 2026) revealed **12 mid-stream tool-call aborts** (`stopReason: "error"`, `errorMessage: "Provider returned an incomplete or malformed tool call"`) across 5 agents (`main`, `grabengasse_media`, `koch`, `hebamme`, `doktor`) when generating nested `tool_call({ id, args })` payloads after `tool_describe`, as well as occasional thinking-only whitespace stalls (Issue 23) and long-context attention drift (Issues 22 & 25).
|
||||||
- Promoting `deepseek/deepseek-v4.1-flash-1017` to primary across all agents eliminates malformed tool-call stream aborts AND lowers total monthly cost by >50% compared to GLM 5.3 Flash.
|
- Promoting `deepseek/deepseek-v4.1-flash-1017` to primary across all agents eliminates malformed tool-call stream aborts AND lowers total monthly cost by >50% compared to GLM 5.3 Flash.
|
||||||
|
|||||||
@@ -1012,3 +1012,42 @@ This document captures fixed issues, operational quirks, and incident root-cause
|
|||||||
- Always enforce `quantizations: ["fp8", "bf16"]` in OpenRouter `extra_body.provider` for tool-calling agents so OpenRouter never routes to degraded `fp4` (`nvfp4`) endpoints.
|
- Always enforce `quantizations: ["fp8", "bf16"]` in OpenRouter `extra_body.provider` for tool-calling agents so OpenRouter never routes to degraded `fp4` (`nvfp4`) endpoints.
|
||||||
- Whenever updating `entry_json` in `session_nodes` via SQLite, always follow up with `UPDATE session_nodes SET entry_valid = 1` in a separate statement (after the `AFTER UPDATE OF entry_json` trigger fires), and always set `PRAGMA foreign_keys = ON` before deleting from `transcript_events`.
|
- Whenever updating `entry_json` in `session_nodes` via SQLite, always follow up with `UPDATE session_nodes SET entry_valid = 1` in a separate statement (after the `AFTER UPDATE OF entry_json` trigger fires), and always set `PRAGMA foreign_keys = ON` before deleting from `transcript_events`.
|
||||||
- Keep Knowledge Wiki and workspace Markdown files in the nightly `prewarm_page_cache()` pass so morning turns hit RAM (`<50ms`) instead of cold rotational disks.
|
- Keep Knowledge Wiki and workspace Markdown files in the nightly `prewarm_page_cache()` pass so morning turns hit RAM (`<50ms`) instead of cold rotational disks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Issue 31: DeepSeek V4.1 Flash Tool-Call Truncation (8,192 Default Token Limit), Provider Re-Ranking, and Fireworks Addition
|
||||||
|
|
||||||
|
- **Date**: 2026-10-18
|
||||||
|
- **Severity**: High (Topic 116 `#Regensburg`, agent `regensburg`, failed with `Provider returned an incomplete or malformed tool call` and wedged in status `failed`)
|
||||||
|
- **Symptom**:
|
||||||
|
1. Moritz sent an audio message to Topic 116 (`#Regensburg`): *"Ach ja, auf der Website soll dann quasi, also die Karte soll so aussehen, dass quasi die Anzahl an Sternen da steht, direkt auf der Karte... wo sind die besten Dönerladen in der City..."*
|
||||||
|
2. The agent prepared the dataset `/tmp/doner_web.json` (45 places, 21 KB) and then invoked `openrouter/deepseek/deepseek-v4.1-flash-1017` to construct the single-page interactive Leaflet map application.
|
||||||
|
3. In Turn 160 (`responseId: gen-1792313455-dX7teZlPrKNOMq1BeMCe`), after 3,376 reasoning tokens (`reasoningTokens`), the model streamed the tool call (`exec`) with the HTML/JS website.
|
||||||
|
4. At exactly **8,192 output tokens** (`output: 8192`), the stream was abruptly cut off by OpenRouter/Cloudflare (`finish_reason: "length"`).
|
||||||
|
5. Because the truncation occurred mid-stream inside the tool-call argument JSON payload, OpenClaw threw:
|
||||||
|
`stopReason: "error"`, `errorMessage: "Provider returned an incomplete or malformed tool call"`.
|
||||||
|
6. The agent sent `⚠️ openrouter/deepseek/deepseek-v4.1-flash-1017 request failed.` to Telegram and remained stuck in `status: "failed"`.
|
||||||
|
- **Root Cause**:
|
||||||
|
1. **Missing `maxTokens` / `max_tokens` Configuration**:
|
||||||
|
- `models.providers.openrouter.models` registered `deepseek/deepseek-v4.1-flash-1017` without `maxTokens`.
|
||||||
|
- When `options.maxTokens` is unset, OpenClaw omits `max_tokens` from `/chat/completions`.
|
||||||
|
- OpenRouter / Cloudflare defaults to **8,192 tokens** when `max_tokens` is omitted.
|
||||||
|
2. **Reasoning Token Competition**:
|
||||||
|
- Unlike standard chat models, DeepSeek V4.1 Flash emits 2,000–3,500 reasoning tokens per complex turn, which count directly against the 8,192 completion limit. This left only ~4,800 tokens for the generated code, causing truncation on large single-page web applications.
|
||||||
|
3. **Suboptimal Provider Ordering in ADR 0002**:
|
||||||
|
- `order: ["deepinfra", "inference-net", "parasail", "cloudflare"]` had Cloudflare in 4th place behind Parasail.
|
||||||
|
- Cloudflare offers `$0.10/M` prompt, `$0.60/M` completion, and `$0.003/M` `cacheRead` (3× cheaper than Parasail), with a **943,718** completion token limit.
|
||||||
|
- **Resolution**:
|
||||||
|
1. **Configured 64k Completion Headroom (`maxTokens: 65536`)**:
|
||||||
|
- Added `maxTokens: 65536` and `contextWindow: 1048576` under `models.providers.openrouter.models` in `openclaw-instance.secret.yaml`.
|
||||||
|
- Added `max_tokens: 65536` to both `params` and `params.extra_body` for `deepseek/deepseek-v4.1-flash-1017` and `deepseek/deepseek-v4.1-flash`.
|
||||||
|
- Applied the change immediately to `/home/openclaw/.openclaw/openclaw.json` on `openclaw-0`.
|
||||||
|
2. **Promoted Cloudflare (#2) and Onboarded Fireworks (#4)**:
|
||||||
|
- Re-ranked provider order to:
|
||||||
|
`order: ["deepinfra", "cloudflare", "inference-net", "fireworks", "parasail"]`
|
||||||
|
- Promoted Cloudflare to #2 for its 97% cache discount and 943k token ceiling.
|
||||||
|
- Added Fireworks as #4 for enterprise tier-1 stability (100% uptime, 943k completion ceiling).
|
||||||
|
3. **Un-wedged Regensburg Session**:
|
||||||
|
- Reset `agent:regensburg:telegram:group:-1004326921510:topic:116` in SQLite from `failed` to `done` and updated `entry_valid = 1`.
|
||||||
|
- **How to Avoid Recurrence**:
|
||||||
|
- For all reasoning-enabled models (DeepSeek V4.1, Gemini 3, etc.), always explicitly configure `maxTokens: 65536` so chain-of-thought tokens never starve downstream tool call payloads.
|
||||||
|
|||||||
Binary file not shown.
Reference in New Issue
Block a user