2db638b992
Add a configuration for controlling the redis instance / type used for streaming events between celery worker and api (vibe-kanban 08e07904)
...
Currently, the celery worker executing workflows / chatflows uses redis pubsub to publish events to api.
(See \_topic\_msg\_generator and \_publish\_streaming\_response)
The current implementation uses the default redis client.
For large scale deployment, we need to use a dedicated redis cluster to ensure performance.
To achieve this, you should:
1. introduce a dedicated configuration class to control
the redis address used for pubsub. (Ideally, there should only be one configuration item such as `pubsub_redis_url`, and its default value should be the original redis confugration.)
2. Add an option to switch between pubsub and sharded pubsub. When shared pubsub is specified, the ShardedRedisBroadcastChannel should be used instead.
COmplete the task above, add some unit tests.
2026-01-19 07:40:44 +08:00
4cc6652424
feat: VECTOR_STORE supports seekdb ( #29658 )
2025-12-16 12:35:04 +09:00
569c593240
feat: Add InterSystems IRIS vector database support ( #29480 )
...
Co-authored-by: Tomo Okuyama <tomo.okuyama@intersystems.com >
2025-12-15 10:20:43 +08:00
c0b7ffd5d0
feat:mysql adaptation for metadb ( #28188 )
2025-11-20 09:44:39 +08:00
32c715c4d0
rm type ignore ( #25715 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com >
2025-10-21 11:26:58 +08:00
cbf2ba6cec
Feature integrate alibabacloud mysql vector ( #25994 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
2025-10-11 10:47:28 +08:00
5a0bf8e028
feat: make SQLALCHEMY_POOL_TIMEOUT configurable ( #25468 )
...
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-18 16:13:56 +08:00
bab4975809
chore: add ast-grep rule to convert Optional[T] to T | None ( #25560 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 13:06:33 +08:00
b05245eab0
fix: resolve typing errors in configs module ( #25268 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-06 16:08:14 +08:00
b5c2756261
fix(api):safe reset in db pool, avoid rollback in gevent callback ( #24556 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-26 18:14:06 +08:00
aaf9fc1562
fix: add @property decorator to pydantic computed_field for compatibility ( #23728 )
2025-08-11 15:34:19 +08:00
e01510e2a6
feat: Add Clickzetta Lakehouse vector database integration ( #22551 )
...
Co-authored-by: Claude <noreply@anthropic.com >
2025-08-07 14:21:46 +08:00
1821726d4f
fix celery backend when use rabbitmq ( #23238 )
...
Signed-off-by: kenwoodjw <blackxin55+@gmail.com >
2025-08-01 09:26:05 +08:00
a189d293f8
make logging not use f-str, change others to f-str ( #22882 )
2025-07-25 10:32:48 +08:00
19c09d6111
enabling vector index prefix name via configuration files ( #22661 )
2025-07-20 11:10:08 +08:00
61a5741c05
fix celery config ( #22566 )
...
Signed-off-by: kenwoodjw <blackxin55+@gmail.com >
2025-07-18 09:41:09 +08:00
da53bf511f
chore: add SQLALCHEMY_POOL_USE_LIFO option and missing SQLALCHEMY_POOL_PRE_PING env default value. ( #22371 )
2025-07-15 19:46:48 +08:00
28f5c37211
Add Env 'CELERY_SENTINEL_PASSWORD' for celery connect redis sentinel. ( #21198 )
2025-06-27 17:37:11 +08:00
17fe62cf91
feat: add support for Matrixone database ( #20714 )
2025-06-19 10:20:12 +08:00
92614765ff
Feat/queue monitor ( #20647 )
2025-06-04 19:56:34 +08:00
0fed5c1193
fix(config): Allow DB_EXTRAS to set search_path via options ( #19560 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-05-13 10:10:18 +08:00
cbc8ebd8f5
chore: bump pydantic to 2.11 and pydantic-settings to 2.9 ( #15049 )
2025-05-08 17:39:51 +08:00
0babdffe3e
feat: support vastbase vector database ( #16308 )
2025-04-24 18:04:57 +08:00
eb1ce3dd6b
feat: support huawei cloud vector database ( #16141 )
2025-04-22 13:03:35 +08:00
7f70cadacb
feat: support Tablestore vector database ( #16601 )
...
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com >
2025-03-27 15:53:33 +08:00
223ab5a38f
feat: support openGauss vector database ( #15865 )
2025-03-17 19:42:54 +08:00
dfdd6dfa20
fix: change the config name and fix typo in description of the number of retrieval executors ( #13856 )
2025-02-19 09:13:36 +08:00
222df44d21
Retrieval Service efficiency optimization ( #13543 )
2025-02-17 14:09:57 +08:00
56e15d09a9
feat: mypy for all type check ( #10921 )
2024-12-24 18:38:51 +08:00
8d4bb9b40d
feat: integrate opendal storage ( #11508 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2024-12-11 14:50:54 +08:00
967b7d89e3
feat:add apollo configuration to load env file ( #11210 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: huanshare <liuhuan101@longfor.com >
Co-authored-by: -LAN- <laipz8200@outlook.com >
2024-12-10 02:51:20 +08:00
0c9e79cd67
Add Lindorm as a VDB choice ( #10202 )
...
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com >
2024-11-04 09:10:26 +08:00
b61baa87ec
fix: avoid unexpected error when create knowledge base with baidu vector database and wenxin embedding model ( #10130 )
2024-10-31 21:34:23 +08:00
878d13ef42
Added OceanBase as an option for the vector store in Dify ( #10010 )
2024-10-29 21:10:18 +08:00
c8ef9223e5
feat: couchbase integration ( #6165 )
...
Co-authored-by: crazywoola <427733928@qq.com >
Co-authored-by: Elliot Scribner <elliot.scribner@couchbase.com >
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
Co-authored-by: Bowen Liang <bowenliang@apache.org >
2024-10-29 15:00:23 +08:00
18106a4fc6
add tidb on qdrant type ( #9831 )
...
Co-authored-by: Zhaofeng Miao <522856232@qq.com >
2024-10-25 13:57:03 +08:00
8e7a752b2a
feat: add upstash as a new vector database provider ( #9644 )
2024-10-23 09:16:35 +08:00
a9fc85027d
chore: update the description for storage_type ( #9492 )
2024-10-18 20:22:57 +08:00
d15ba3939d
Add Volcengine VikingDB as new vector provider ( #9287 )
2024-10-13 21:26:05 +08:00
7a6970e570
feat: add supabase object storage ( #9229 )
2024-10-11 22:48:57 +08:00
42b02b3a5f
Fix/agent external knowledge retrieval ( #9241 )
2024-10-11 19:21:03 +08:00
2571b0c4e3
feat: add baidu obs storage ( #9024 )
2024-10-07 11:09:27 +08:00
ddf6569dc5
chore: enhance configuration descriptions ( #8624 )
2024-09-22 13:38:41 +08:00
c8df92d0eb
add volcengine tos storage ( #8164 )
2024-09-10 09:19:47 +08:00
d542b15cc0
feat: support redis sentinel mode ( #7756 )
2024-09-08 13:23:51 +08:00
e0d3cd91c6
support huawei cloud obs storage ( #7980 ) ( #7981 )
2024-09-06 14:00:47 +08:00
122ce41020
feat: rewrite Elasticsearch index and search code to achieve Elasticsearch vector and full-text search ( #7641 )
...
Co-authored-by: haokai <haokai@shuwen.com >
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
Co-authored-by: Bowen Liang <bowenliang@apache.org >
Co-authored-by: wellCh4n <wellCh4n@foxmail.com >
2024-08-27 11:43:44 +08:00
2da63654e5
chore(api/configs): apply ruff reformat ( #7590 )
2024-08-23 23:46:01 +08:00
0b4c26578e
Enhance database URI security and add URL encoding ( #6668 )
2024-07-25 16:48:00 +08:00
63e34e5227
feat: support MyScale vector database ( #6092 )
2024-07-11 15:21:59 +08:00
