mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-05-24 01:47:36 +08:00
a69e0c73c7
### What problem does this PR solve? Partially addresses #14362. This PR enables syncing deleted files for RSS data sources. Previously, RSS incremental sync only returned feed entries whose timestamps were inside the poll window. If an entry was removed from the RSS feed, RAGFlow had no full current RSS snapshot to pass into the shared stale-document cleanup path, so the deleted remote entry could remain in the knowledge base. This PR: - adds `retrieve_all_slim_docs_perm_sync()` to `RSSConnector` - reuses the same `rss:<md5(stable_key)>` document ID derivation used by normal RSS ingest - returns `(document_generator, file_list)` for incremental RSS sync when `sync_deleted_files` is enabled - captures the poll end timestamp before snapshot/poll so cleanup does not race against the same sync window - adds start/end logs around RSS slim snapshot collection - exposes the deleted-file sync toggle for RSS in the data source UI Per maintainer request on related datasource PRs, this PR contains no test-case changes. Local verification was run with an external script. Validation: - `uv run ruff check common/data_source/rss_connector.py rag/svr/sync_data_source.py` - `uv run pytest test/unit_test/rag/test_sync_data_source.py -q` - `./node_modules/.bin/eslint src/pages/user-setting/data-source/constant/index.tsx` - `git diff --check` - `uv run python /tmp/verify_rss_deleted_sync.py --repo /root/74/ragflow` ### Type of change - [x] New Feature (non-breaking change which adds functionality)
Install front-end dependencies
npm install
Launch front-end
npm run dev
The following output confirms a successful launch of the system:
Login to RAGFlow web UI
Open your browser and navigate to:
http://localhost:9222 or http://[YOUR_MACHINE_IP]:9222
Replace [YOUR_MACHINE_IP] with your actual machine IP address (e.g., http://192.168.1.49:9222).
Login to RAGFlow web admin UI
Open your browser and navigate to:
http://localhost:9222/admin or http://[YOUR_MACHINE_IP]:9222/admin
Replace [YOUR_MACHINE_IP] with your actual machine IP address (e.g., http://192.168.1.49:9222/admin).
Shutdown front-end
Ctrl + C or
kill -f "umi dev"