Commit Graph

164 Commits

Author SHA1 Message Date
265dc54c51 fix(watercrawl): don't disable request timeouts with timeout=None (#37685)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-22 03:08:58 +00:00
adfd820220 fix(watercrawl): bound client request timeouts (#37515) 2026-06-21 00:47:25 +00:00
431d6bb983 fix(watercrawl): accept content type parameters (#37500) 2026-06-16 08:16:24 +00:00
fe64c5d4a8 fix(watercrawl): bound result download timeout (#37495) 2026-06-16 08:16:16 +00:00
b61d39ae2b chore(api): Fix several typing errors (#37237)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-12 01:36:48 +00:00
813bfea730 feat(api): support embedded Excel images in knowledge import (#37104) 2026-06-08 01:26:07 +00:00
e14cb209a4 chore: add missing @override decorator to api/core/rag/extractor (#37013)
Co-authored-by: mac <mac@1234.local>
2026-06-03 12:34:10 +00:00
71ffaacb58 fix(api): centralize remote file retrieval (#36399)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-01 09:25:08 +00:00
372a2e3e9c refactor: convert isinstance chains to match/case (part 7) (#35902) (#36826) 2026-05-29 09:40:33 +00:00
af4b9bfa8f chore: Remove pyright in favor of pyrefly (#36154) 2026-05-14 05:49:08 +00:00
6facd9360c chore: some match case (#36080)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-12 14:45:58 +00:00
f720a3bed2 fix: Image rendering in the knowledge base failed. (#35914)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-09 10:06:01 +00:00
77f8f2babb refactor(api): tighten core rag typing batch 1 (#35210)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-21 04:32:43 +00:00
99
3e876e173a chore(api): adapt Graphon 0.2.2 upgrade (#35377)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-18 11:16:24 +00:00
ef28a63ad3 refactor(api): add null safety to extractor_processor and firecrawl (#35209)
Co-authored-by: tmimmanuel <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>
2026-04-14 18:23:20 +00:00
06b63d65d1 refactor: replace bare dict with dict[str, Any] in rag extractors (#35068) 2026-04-13 14:05:21 +00:00
1b935a367f refactor: replace bare dict with dict[str, Any] in watercrawl client (#35063) 2026-04-13 13:01:32 +00:00
1a4eb47e1d refactor(api): tighten types in trivial lint and config fixes (#34773)
Co-authored-by: tmimmanuel <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>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-08 23:14:44 +00:00
99
8f9dbf269e chore(api): align Python support with 3.12 (#34419)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-02 05:07:32 +00:00
99
f27d669f87 chore: normalize frozenset literals and myscale typing (#34327) 2026-03-31 08:21:22 +00:00
456684dfc3 refactor: core/rag docstore, datasource, embedding, rerank, retrieval (#34203)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-03-30 08:09:49 +00:00
30deeb6f1c feat(firecrawl): follow pagination when crawl status is completed (#33864)
Co-authored-by: Crazywoola <100913391+crazywoola@users.noreply.github.com>
2026-03-23 21:19:32 +08:00
f40f6547b4 refactor(api): type bare dict/list annotations in remaining rag folder (#33775) 2026-03-20 03:31:06 +09:00
e4f1d3c63a refactor: use EnumText(StorageType) for UploadFile.storage_type (#33728) 2026-03-19 15:15:32 +09:00
9ff0d9df88 refactor(api): type WaterCrawl API responses with TypedDict (#33700) 2026-03-19 10:35:44 +09:00
b2a388b7bf refactor(api): type Firecrawl API responses with TypedDict (#33691) 2026-03-19 04:00:06 +09:00
54637144c5 fix: fix parse mailto / http link in table cell (#33224)
Co-authored-by: Oz <oz-agent@warp.dev>
2026-03-11 10:02:33 +08:00
27f9cdedad fix: fix failed test (#33241) 2026-03-11 09:37:19 +08:00
dbfc47e8b0 fix: SSRF in WordExtractor URL download (credit to @EaEa0001 ) (#31678)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-29 14:01:21 +08:00
114a34e008 fix: correct docx hyperlink extraction (#30360) 2026-01-06 11:24:26 +08:00
cad7101534 feat: support image extraction in PDF RAG extractor (#30399)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-12-31 15:49:06 +08:00
a5309bee25 fix: handle missing credential_id (#30051)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-24 11:21:51 +08:00
111a39b549 fix: fix firecrawl url concat (#30008) 2025-12-24 09:40:32 +08:00
32605181bd feat: first use INTERNAL_FILES_URL first, then FILES_URL (#29962) 2025-12-21 16:53:37 +08:00
8d1e36540a fix: detect_file_encodings TypeError: tuple indices must be integers or slices, not str (#29595)
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-12-17 13:58:05 +08:00
ae4a9040df Feat/update notion preview (#29345)
Co-authored-by: twwu <twwu@dify.ai>
2025-12-16 16:43:45 +08:00
8f3fd9a728 perf: commit once (#29590) 2025-12-15 11:40:26 +08:00
db42f467c8 fix: docx extractor external image failed (#29558) 2025-12-12 13:41:51 +08:00
12e39365fa perf(core/rag): optimize Excel extractor performance and memory usage (#29551)
Co-authored-by: 01393547 <nieronghua@sf-express.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-12-12 12:15:03 +08:00
45911ab0af feat: using charset_normalizer instead of chardet (#29022) 2025-12-05 11:19:19 +08:00
81832c14ee Fix: Correctly handle merged cells in DOCX tables to prevent content duplication and loss (#27871)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-11-13 15:56:24 +08:00
a16ef7e73c refactor: Update Firecrawl to use v2 API (#24734)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-10-15 10:48:54 +08:00
d299e75e1b refactor: use dynamic max characters for chunking in extractors (#26782) 2025-10-13 10:22:59 +08:00
1bd621f819 remove .value (#26633)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-10-11 09:08:29 +08:00
bb6a331490 change all to httpx (#26119)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-10-10 23:41:16 +08:00
ab2eacb6c1 use model_validate (#26182)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-10-10 17:30:13 +09:00
85cda47c70 feat: knowledge pipeline (#25360)
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: jyong <718720800@qq.com>
Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: lyzno1 <yuanyouhuilyz@gmail.com>
Co-authored-by: quicksand <quicksandzn@gmail.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: lyzno1 <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yongtao Huang <yongtaoh2022@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: Hanqing Zhao <sherry9277@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry <xh001x@hotmail.com>
2025-09-18 12:49:10 +08:00
bab4975809 chore: add ast-grep rule to convert Optional[T] to T | None (#25560)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 13:06:33 +08:00
67a686cf98 [Chore/Refactor] use __all__ to specify export member. (#25681) 2025-09-15 09:45:35 +08:00
a13d7987e0 chore: adopt StrEnum and auto() for some string-typed enums (#25129)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-09-12 21:14:26 +08:00