fix(collections): Use correct file batch ID#1073
Conversation
📝 WalkthroughWalkthrough
ChangesVector batch ingestion
Document upload validation
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
changed the title
OpenAPI changes ⚪ No API surface changesNote This PR does not modify the API contract.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
backend/app/tests/crud/rag/test_open_ai.py (2)
72-72: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRemove the self-evident comment.
assert_not_called()already expresses this expectation; retain comments only when they explain rationale. As per coding guidelines, “Write comments and docstrings to explain why something is done, not what the code already shows.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/tests/crud/rag/test_open_ai.py` at line 72, Remove the self-evident “list_files should not have been called on the happy path” comment near the assert_not_called() expectation in the test, leaving the assertion unchanged.Source: Coding guidelines
44-44: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd required type annotations to changed test signatures.
Annotate non-
selffixture parameters and return values consistently.
backend/app/tests/crud/rag/test_open_ai.py#L44-L44: annotatemock_client: MagicMock.backend/app/tests/crud/rag/test_open_ai.py#L75-L75: annotate injected fixtures and add-> None.backend/app/tests/crud/rag/test_open_ai.py#L106-L108: annotate injected fixtures and add-> None.backend/app/tests/crud/rag/test_open_ai.py#L122-L124: annotate injected fixtures and add-> None.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/tests/crud/rag/test_open_ai.py` at line 44, Add the requested annotations in backend/app/tests/crud/rag/test_open_ai.py: type _wire_batch’s mock_client parameter as MagicMock, and annotate all non-self injected fixture parameters plus return values with -> None in the test functions at lines 75, 106-108, and 122-124. Apply the annotations consistently across all four affected sites.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@backend/app/tests/crud/rag/test_open_ai.py`:
- Line 72: Remove the self-evident “list_files should not have been called on
the happy path” comment near the assert_not_called() expectation in the test,
leaving the assertion unchanged.
- Line 44: Add the requested annotations in
backend/app/tests/crud/rag/test_open_ai.py: type _wire_batch’s mock_client
parameter as MagicMock, and annotate all non-self injected fixture parameters
plus return values with -> None in the test functions at lines 75, 106-108, and
122-124. Apply the annotations consistently across all four affected sites.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 79f5c90e-9ea2-43f6-8dc9-ff715cb54c65
📒 Files selected for processing (6)
backend/app/crud/rag/open_ai.pybackend/app/tests/crud/rag/test_open_ai.pybackend/app/tests/services/collections/providers/test_openai_provider.pybackend/app/tests/utils/llm_provider.pybackend/app/tests/utils/openai.pydocs/wiki/modules/knowledge-base.md
…in document upload process
There was a problem hiding this comment.
🧹 Nitpick comments (2)
backend/app/services/documents/helpers.py (2)
1-6: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winModernize deprecated
typingimports.
Callable/Iterableshould come fromcollections.abc, andTupleis deprecated in favor of builtintuple(PEP 585). Ruff flags both (UP035).♻️ Proposed fix
-from typing import Callable, Optional, Tuple, Iterable, Union +from collections.abc import Callable, Iterable +from typing import Optional, UnionNote: other
Tuple[...]usages later in the file (e.g._read_edges,validate_document_content... wait, that one already uses lowercaseTuplereturn) should be updated totuple[...]for consistency once the import changes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/services/documents/helpers.py` around lines 1 - 6, Update the imports in this module so Callable and Iterable come from collections.abc, remove Tuple from typing, and replace every Tuple[...] annotation in the file with the builtin tuple[...] syntax; preserve Optional and Union unless independently required.Source: Linters/SAST tools
53-57: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winDuplicated magic byte signatures between
MISLABELLED_BINARY_SIGNATURESandFORMAT_SPECS;"doc"has noFormatSpec.
b"PK\x03\x04"andb"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1"are inlined in both structures. Extracting them (and the zip-signature triple used for both docx and xlsx) into named constants would remove the duplication.Separately,
"doc"is absent fromFORMAT_SPECSeven though it shares the OLE2 signature documented for"xls"(comment at line 246 explicitly calls out ".doc"), andregistry.pymaps.doc→"doc". As written,.docuploads get zero content sniffing while.xlsuploads (same binary container format) do.♻️ Proposed fix
+OLE2_SIGNATURE = b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1" +ZIP_SIGNATURES = (b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08") + MISLABELLED_BINARY_SIGNATURES: dict[bytes, str] = { - b"PK\x03\x04": "an Office/zip file (xlsx, docx)", - b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1": "a legacy Office file (xls, doc)", + ZIP_SIGNATURES[0]: "an Office/zip file (xlsx, docx)", + OLE2_SIGNATURE: "a legacy Office file (xls, doc)", b"%PDF-": "a PDF", } ... "docx": FormatSpec( - signatures=(b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08"), checker=_check_docx + signatures=ZIP_SIGNATURES, checker=_check_docx ), "xlsx": FormatSpec( - signatures=(b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08"), checker=_check_xlsx + signatures=ZIP_SIGNATURES, checker=_check_xlsx ), - "xls": FormatSpec(signatures=(b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1",)), + "xls": FormatSpec(signatures=(OLE2_SIGNATURE,)), + "doc": FormatSpec(signatures=(OLE2_SIGNATURE,)),Also applies to: 237-250
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/app/services/documents/helpers.py` around lines 53 - 57, Extract the shared PK/zip and OLE2 byte signatures into named constants, then reuse them in MISLABELLED_BINARY_SIGNATURES and the relevant FORMAT_SPECS entries, including the existing zip-signature triple for docx and xlsx. Add a “doc” FORMAT_SPECS entry matching the “xls” OLE2 signature and appropriate shared sniffing behavior so .doc uploads are handled like .xls.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@backend/app/services/documents/helpers.py`:
- Around line 1-6: Update the imports in this module so Callable and Iterable
come from collections.abc, remove Tuple from typing, and replace every
Tuple[...] annotation in the file with the builtin tuple[...] syntax; preserve
Optional and Union unless independently required.
- Around line 53-57: Extract the shared PK/zip and OLE2 byte signatures into
named constants, then reuse them in MISLABELLED_BINARY_SIGNATURES and the
relevant FORMAT_SPECS entries, including the existing zip-signature triple for
docx and xlsx. Add a “doc” FORMAT_SPECS entry matching the “xls” OLE2 signature
and appropriate shared sniffing behavior so .doc uploads are handled like .xls.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 6e6e3f5f-23a5-4f47-96b3-cfc57b2d0caa
📒 Files selected for processing (3)
backend/app/api/routes/documents.pybackend/app/services/doctransform/registry.pybackend/app/services/documents/helpers.py
1 participant
Issue
Closes #1076
Summary
create()function when listing failed files.Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Notes
Please add here if any other information is required for the reviewer.
Original PR description
Issue
Closes #PLEASE_TYPE_ISSUE_NUMBER
Summary
Explain the motivation for making this change. What existing problem does the pull request solve?
Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Summary by CodeRabbit
New Features
.xlsand.xlsxspreadsheets.Bug Fixes
Documentation