L1nSn0w 3bf92323b2 refactor(llm): measure first-token timeout via httpx read timeout ...

The thread-based FirstTokenWatchdog could never fire in production: the
plugin daemon withholds the SSE response headers until the model's first
token, so the whole time-to-first-token is spent inside httpx
Client.stream() waiting for headers -- before the watchdog is armed. It
only ever measured headers->first-frame (~0s) and was disarmed by the
first frame.

Replace the watchdog with httpx's per-request read timeout set to the
first-token budget. httpcore applies read to both the header wait and each
body read, and the daemon sends no heartbeat before the first token, so it
measures time-to-first-token directly. A pre-first-token ReadTimeout with
the gate enabled becomes FirstTokenTimeoutError; anything else (inter-token
stall, gate off, other transport errors) stays a normal transport error.

Deletes the entire watchdog subsystem (timer, lock, socket-abort,
content-detection); the ContextVar plumbing and producer side are unchanged.

2026-07-10 13:58:48 +08:00

..

backwards_invocation

fix(api): resolve plugin external user ids in backwards invocation (#38098)

2026-07-06 08:25:26 +00:00

endpoint

fix: handle EndpointSetupFailedError in BasePluginClient (#19613)

2025-05-15 13:59:43 +08:00

entities

fix: resolve 36288 mypy errors (#37850)

2026-07-07 06:37:20 +00:00

impl

refactor(llm): measure first-token timeout via httpx read timeout

2026-07-10 13:58:48 +08:00

utils

refactor: replace isinstance chains with match-case (#24487) (#37482)

2026-06-16 04:41:31 +00:00

plugin_service.py

fix: can't debug model plugins (#38500)

2026-07-07 13:03:32 +00:00