Currently, the celery worker executing workflows / chatflows uses redis pubsub to publish events to api.
(See \_topic\_msg\_generator and \_publish\_streaming\_response)
The current implementation uses the default redis client.
For large scale deployment, we need to use a dedicated redis cluster to ensure performance.
To achieve this, you should:
1. introduce a dedicated configuration class to control
the redis address used for pubsub. (Ideally, there should only be one configuration item such as `pubsub_redis_url`, and its default value should be the original redis confugration.)
2. Add an option to switch between pubsub and sharded pubsub. When shared pubsub is specified, the ShardedRedisBroadcastChannel should be used instead.
COmplete the task above, add some unit tests.
Reason should be an enumeration with only one member `resumption` currently.
Please update these part of events:
- Graph / Engine Event (GraphRunStartedEvent)
- Queue event (QueueWorkflowStartedEvent)
- SSE response event (WorkflowStartStreamResponse)
Besides, you should remove the `is_resumption` flag for `node_started` events; including:
- Queue Event (`QueueNodeStartedEvent`)
- SSE Event (`NodeStartStreamResponse`)
- Node event (`NodeRunStartedEvent`)
After finishing the changes above, adjust related tests.
You should run the affected tests and ensure they can pass. (You should use `uv run pytest` to run tests)
While running workflow / chatflow from "Installed Apps" / "Web App"
pages, the `node_started` SSE event is manually serialized from the
pydantic model. This causes the lack of `is_resumption` flag in SSE
events.
This PR addresses the problem by adding a `is_resumption` field to
the serialized dict.
This commit:
1. Convert `pause_reason` to `pause_reasons` in `GraphExecution` and relevant classes. Change the field from a scalar value to a list that can contain multiple `PauseReason` objects, ensuring all pause events are properly captured.
2. Introduce a new `WorkflowPauseReason` model to record reasons associated with a specific `WorkflowPause`.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Certain metadata (including but not limited to `InvokeFrom`, `call_depth`, and `streaming`) is required when resuming a paused workflow. However, these fields are not part of `GraphRuntimeState` and were not saved in the previous
implementation of `PauseStatePersistenceLayer`.
This commit addresses this limitation by introducing a `WorkflowResumptionContext` model that wraps both the `*GenerateEntity` and `GraphRuntimeState`. This approach provides:
- A structured container for all necessary resumption data
- Better separation of concerns between execution state and persistence
- Enhanced extensibility for future metadata additions
- Clearer naming that distinguishes from `GraphRuntimeState`
The `WorkflowResumptionContext` model makes extending the pause state easier while maintaining backward compatibility and proper version management for the entire execution state ecosystem.
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
