559ab46ee1
fix: Removes redundant token calculations and updates dependencies
...
Eliminates unnecessary pre-calculation of token limits and recalculation of max tokens
across multiple app runners, simplifying the logic for prompt handling.
Updates tiktoken library from version 0.8.0 to 0.9.0 for improved tokenization performance.
Increases default token limit in TokenBufferMemory to accommodate larger prompt messages.
These changes streamline the token management process and leverage the latest
improvements in the tiktoken library.
Fixes potential token overflow issues and prepares the system for handling larger
inputs more efficiently.
Relates to internal optimization tasks.
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-04-28 15:39:12 +08:00
87763fc234
feat(model_providers): Support deepseek for Azure AI Foundry ( #13267 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-06 12:45:48 +08:00
7203991032
feat: add parameter "reasoning_effort" and Openai o3-mini ( #13243 )
2025-02-06 09:29:48 +08:00
03243cb422
Modify params for bedrock retrieve generate ( #13182 )
2025-02-05 12:17:42 +08:00
d4c916b496
chore(pyproject): Add type stubs into pyproject.toml ( #13145 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-02-04 12:01:28 +08:00
b2bbc28580
support bedrock kb: retrieve and generate ( #13027 )
2025-01-25 17:28:06 +08:00
c62b7cc679
chore(build): bump poetry from 1.x to 2.x ( #12369 )
2025-01-22 13:38:24 +08:00
166221d784
chore(lint): fix quotes for f-string formatting by bumping ruff to 0.9.x ( #12702 )
2025-01-21 10:12:29 +08:00
69d58fbb50
Add new integration with Opik Tracking tool ( #11501 )
2025-01-13 17:41:44 +08:00
040a3b782c
FEAT: support milvus to full text search ( #11430 )
...
Signed-off-by: YoungLH <974840768@qq.com >
2025-01-08 17:39:53 +08:00
baeddd4d15
feat:Add support for stop parameter in hunyuan model #12313 ( #12315 )
...
Co-authored-by: xander-art <xander-art@gmail.com >
2025-01-03 09:15:04 +08:00
15f43dd326
chore(deps): update yarl version from 1.9.4 to 1.18.3 ( #12302 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2025-01-02 16:26:47 +08:00
63a0b8ba79
feat: integrate psycogreen for gevent compatibility in PostgreSQL ( #12253 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2024-12-31 14:45:59 +08:00
56e15d09a9
feat: mypy for all type check ( #10921 )
2024-12-24 18:38:51 +08:00
efa8eb379f
fix: memory leak by pypdfium2 close(maybe) #11510 ( #11700 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2024-12-17 00:42:01 +08:00
430ca3322b
chore(dependency): bump gunicorn to 23.0 ( #11560 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2024-12-13 16:16:58 +08:00
8d4bb9b40d
feat: integrate opendal storage ( #11508 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2024-12-11 14:50:54 +08:00
376726cf90
[feat] Add AWS Bedrock rerank ( #11349 )
...
Co-authored-by: crazywoola <427733928@qq.com >
2024-12-05 11:31:43 +08:00
a67b0d4771
chore(lint): extract ruff configs into .ruff.toml file keeping pyproject.toml clean ( #11222 )
2024-12-01 12:51:28 +08:00
2feb44e2c5
chore(dep): bump flask from 3.0.1 to 3.1.0 and flask-compress to 1.17 ( #11195 )
2024-11-29 09:28:53 +08:00
b791a80b75
chore: update chromadb version to 0.5.20 ( #11038 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2024-11-25 11:14:04 +08:00
6c8e208ef3
chore: bump minimum supported Python version to 3.11 ( #10386 )
2024-11-24 13:28:46 +08:00
d3051eed48
chore (dep): bump gevent from v23 to v24 for better support for Python 3.11 and 3.12 ( #10387 )
2024-11-23 00:07:07 +08:00
c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. ( #10679 )
...
Co-authored-by: Joel <iamjoel007@gmail.com >
2024-11-22 16:30:22 +08:00
90d6ebc879
Add youtube-transcript-api as tool ( #10772 )
...
Co-authored-by: crazywoola <427733928@qq.com >
2024-11-18 10:58:16 +08:00
51db59622c
chore(lint): cleanup repeated cause exception in logging.exception replaced by helpful message ( #10425 )
2024-11-15 15:41:40 +08:00
365cb4b368
chore(lint): bump ruff from 0.6.9 to 0.7.3 ( #10714 )
2024-11-15 09:19:41 +08:00
ad16180b1a
feat(tool): fal ai wizper ASR built-in tool ( #10716 )
2024-11-15 09:01:07 +08:00
7c2a9b0744
celery worker log format following LOG_FORMAT env#9404 ( #10016 )
...
Signed-off-by: root <root@localhost.localdomain >
Co-authored-by: root <root@localhost.localdomain >
2024-11-08 17:12:09 +08:00
574c4a264f
chore(lint): Use logging.exception instead of logging.error ( #10415 )
2024-11-07 21:13:02 +08:00
d45d90e8ae
chore: lazy import sagemaker ( #10342 )
2024-11-06 12:45:22 +08:00
acb22f0fde
Updates: Add mplfonts library for customizing matplotlib fonts and Va… ( #9903 )
2024-11-05 10:34:28 +08:00
878d13ef42
Added OceanBase as an option for the vector store in Dify ( #10010 )
2024-10-29 21:10:18 +08:00
c8ef9223e5
feat: couchbase integration ( #6165 )
...
Co-authored-by: crazywoola <427733928@qq.com >
Co-authored-by: Elliot Scribner <elliot.scribner@couchbase.com >
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com >
Co-authored-by: Bowen Liang <bowenliang@apache.org >
2024-10-29 15:00:23 +08:00
caf7bc8569
upgrade nltk, unstructured and starlette ( #9860 )
2024-10-25 17:15:44 +08:00
121bb99cc2
downgrade unstructured nltk version ( #9726 )
2024-10-23 19:02:27 +08:00
cac1ef7ade
remove ppt import ( #9721 )
2024-10-23 18:22:30 +08:00
3e9d271b52
nltk security issue and upgrade unstructured ( #9558 )
2024-10-23 16:23:55 +08:00
8e7a752b2a
feat: add upstash as a new vector database provider ( #9644 )
2024-10-23 09:16:35 +08:00
3f1c84f65a
chore: cleanup ineffective linter rules exclusions ( #9580 )
2024-10-22 09:18:31 +08:00
495cf58014
dep: bump pydantic to 2.9 ( #9077 )
2024-10-21 23:32:09 +08:00
e61752bd3a
feat/enhance the multi-modal support ( #8818 )
2024-10-21 10:43:49 +08:00
b597a0d31c
fix: Azure OpenAI o1 max_completion_token and get_num_token_from_messages error ( #9326 )
...
Co-authored-by: wwwc <wwwc@outlook.com >
2024-10-15 16:26:44 +08:00
ffc3f33670
chore: remove the copied zhipu_ai sdk ( #9270 )
2024-10-14 10:53:45 +08:00
d15ba3939d
Add Volcengine VikingDB as new vector provider ( #9287 )
2024-10-13 21:26:05 +08:00
2ec6ffe478
feat:support baidu vector db ( #9185 )
2024-10-12 23:24:17 +08:00
d9773c963f
chore: fix the misclassification of the opensearch-py package ( #9266 )
2024-10-12 17:37:01 +08:00
7a6970e570
feat: add supabase object storage ( #9229 )
2024-10-11 22:48:57 +08:00
05c1ef75c4
feat: add allow_llm_to_see_data flag for vanna ( #9156 )
2024-10-10 17:39:26 +08:00
68107fe355
downgrade elastic to 8.14 ( #9171 )
2024-10-10 16:43:11 +08:00
