Magicbook1108 158503a1aa Feat: optimize ingestion pipeline with preprocess (#13211) ...

### What problem does this PR solve?

Feat: optimize ingestion pipeline with preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

2026-02-26 10:24:13 +08:00

..

extractor

Fix: tokenizer issue. (#11902)

2025-12-11 17:38:17 +08:00

hierarchical_merger

Feat: optimize ingestion pipeline with preprocess (#13211)

2026-02-26 10:24:13 +08:00

parser

Feat: optimize ingestion pipeline with preprocess (#13211)

2026-02-26 10:24:13 +08:00

splitter

Fix: overlap cannot be properly applied (#12828)

2026-01-27 12:43:01 +08:00

tests

Refa:replace trio with asyncio (#11831)

2025-12-09 19:23:14 +08:00

tokenizer

Refa: asyncio.to_thread to ThreadPoolExecutor to break thread limitat… (#12716)

2026-01-20 13:29:37 +08:00

__init__.py

Feat: refine dataflow and initialize dataflow app (#9952)

2025-09-05 18:50:46 +08:00

base.py

Refa:replace trio with asyncio (#11831)

2025-12-09 19:23:14 +08:00

file.py

Fix: ingestion pipeline (#13012)

2026-02-05 15:55:41 +08:00

pipeline.py

Refa:replace trio with asyncio (#11831)

2025-12-09 19:23:14 +08:00