mirror of
https://github.com/langgenius/dify.git
synced 2026-07-16 01:48:40 +08:00
bbd017189a
Adds transport-layer enforcement for the LLM node's first_token_timeout, which graphon carries down the LLM invoke path. While a stream waits for its first token, a wall-clock watchdog fires after the configured timeout and shuts the underlying socket down to abort the in-flight read; the caller raises FirstTokenTimeoutError, which bubbles up as a node failure and is handled by the node's existing error strategy (retry / fail-branch / default). The value is tunneled from the graphon-facing adapter (DifyPreparedLLM) to _stream_request through a ContextVar, so the intermediate model-runtime layers that don't take the parameter stay untouched. It is disarmed on the first token, so later tokens are never gated. Default 0 (disabled) keeps today's behavior. response.close() does not interrupt a blocked sync iter_lines(); the cut is socket.shutdown(SHUT_RDWR) reached via httpx' network_stream extension. The daemon emits no pre-first-token heartbeats, so first-token timing is clean. Requires graphon with langgenius/graphon#212 (FirstTokenTimeoutError + the protocol first_token_timeout parameter); the graphon pin bump is release-gated.