mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-05-24 01:47:36 +08:00
653b00b94c
Fixes #14360 ## Problem When the same blob storage bucket is connected to multiple knowledge bases (each through a different data source connector), the sync pipeline hashes only the blob path (`bucket_type:bucket_name:object_key`) to derive the document ID. Every connector pointing at the same bucket therefore produces **identical IDs** for the same object. The collision guard in `FileService.upload_document` then fires for the second knowledge base: ``` Existing document id collision with another knowledge base; skipping update. ``` This makes it impossible to index the same bucket into more than one KB simultaneously. ## Solution Include `connector_id` in the hash input so that each connector produces a distinct document ID even when the underlying blob path is identical: ```python # Before "id": hash128(doc.id), # After "id": hash128(f"{task['connector_id']}:{doc.id}"), ``` Because each KB connection uses its own connector (with a unique `connector_id`), documents are now namespaced per connector and no collision occurs. **Note:** This is a breaking change for existing synced data sources. After upgrading, a re-sync will create new documents with the updated ID format. Old documents (indexed under the previous format) will remain in the database but can be manually deleted or cleaned up via a re-sync with reindex enabled. ## Testing - Verified that the one-line change produces unique IDs for two connectors pointing at the same S3 path. - Existing unit test `test_upload_document_skips_cross_kb_document_id_collision` continues to pass — the collision guard in `FileService` is still valid for genuinely colliding IDs from other sources. --------- Co-authored-by: octo-patch <octo-patch@github.com>