The thread-based FirstTokenWatchdog could never fire in production: the
plugin daemon withholds the SSE response headers until the model's first
token, so the whole time-to-first-token is spent inside httpx
Client.stream() waiting for headers -- before the watchdog is armed. It
only ever measured headers->first-frame (~0s) and was disarmed by the
first frame.
Replace the watchdog with httpx's per-request read timeout set to the
first-token budget. httpcore applies read to both the header wait and each
body read, and the daemon sends no heartbeat before the first token, so it
measures time-to-first-token directly. A pre-first-token ReadTimeout with
the gate enabled becomes FirstTokenTimeoutError; anything else (inter-token
stall, gate off, other transport errors) stays a normal transport error.
Deletes the entire watchdog subsystem (timer, lock, socket-abort,
content-detection); the ContextVar plumbing and producer side are unchanged.
Follow-ups from code review:
- Transport: treat a non-positive timeout (0/negative) as disabled so a 0-second
watchdog can never arm; add a post-stream backstop that raises
FirstTokenTimeoutError when aborting the socket ends the stream with a clean EOF
instead of a transport error; only raise when no token was delivered.
- Watchdog: make fire/disarm mutually exclusive with a lock, so a token arriving on
the deadline is never both yielded and aborted and `fired` reliably means
"timed out before the first token".
- Adapter: extract the shared set/reset-ContextVar logic into one helper.
- i18n: fix nodes.common.retry.ms, mistranslated as the "Ms./Mrs." honorific in
9 locales (de/es/hi/th/pl/tr/vi/id/ro; uk lowercased).
- Tests: real-socket test proving socket.shutdown unblocks a blocked httpx
iter_lines and surfaces a real RequestError; a clean-EOF backstop test; a
0=disabled test; and NaN/negative-clamp + preserve-other-fields assertions.
Adds transport-layer enforcement for the LLM node's first_token_timeout, which
graphon carries down the LLM invoke path. While a stream waits for its first
token, a wall-clock watchdog fires after the configured timeout and shuts the
underlying socket down to abort the in-flight read; the caller raises
FirstTokenTimeoutError, which bubbles up as a node failure and is handled by the
node's existing error strategy (retry / fail-branch / default).
The value is tunneled from the graphon-facing adapter (DifyPreparedLLM) to
_stream_request through a ContextVar, so the intermediate model-runtime layers
that don't take the parameter stay untouched. It is disarmed on the first token,
so later tokens are never gated. Default 0 (disabled) keeps today's behavior.
response.close() does not interrupt a blocked sync iter_lines(); the cut is
socket.shutdown(SHUT_RDWR) reached via httpx' network_stream extension. The
daemon emits no pre-first-token heartbeats, so first-token timing is clean.
Requires graphon with langgenius/graphon#212 (FirstTokenTimeoutError + the
protocol first_token_timeout parameter); the graphon pin bump is release-gated.
