refactor(vector-stores): vector store interface #151

micpst · 2024-10-27T22:56:04Z

No description provided.

github-actions · 2024-10-27T22:57:15Z

Code Coverage Summary

Filename                                                                                                     Stmts    Miss  Cover    Missing
---------------------------------------------------------------------------------------------------------  -------  ------  -------  ----------------------------------------
packages/ragbits-core/src/ragbits/core/__init__.py                                                               0       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                 6       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                   11       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                         14       8  42.86%   27-39
packages/ragbits-core/src/ragbits/core/llms/base.py                                                             31       2  93.55%   33, 52
packages/ragbits-core/src/ragbits/core/llms/factory.py                                                          18       2  88.89%   47, 60
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                          24       4  83.33%   7-8, 53, 84
packages/ragbits-core/src/ragbits/core/llms/types.py                                                             9       2  77.78%   25, 29
packages/ragbits-core/src/ragbits/core/llms/clients/__init__.py                                                  4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/base.py                                                     23       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/exceptions.py                                               14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                                  49      10  79.59%   9-10, 69, 107, 121-126
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                                    36      12  66.67%   10-11, 61-69, 90-101
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                        2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                           18       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                        35       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                        110       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                              2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                     33       2  93.94%   54-55
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                      28       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                 16       4  75.00%   32-33, 37-38
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                      29       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/__init__.py                                                12       1  91.67%   27
packages/ragbits-core/src/ragbits/core/vector_stores/base.py                                                    20       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/chroma.py                                                  40       2  95.00%   65-66
packages/ragbits-core/src/ragbits/core/vector_stores/in_memory.py                                               23       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                     0       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                           64       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/__init__.py                                                        3       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                            9       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_llm_from_factory.py                                       8       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_has_default_llm.py                                            9       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                             0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                        65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                        143       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                   0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                  30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                     18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                    2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py            3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py       14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py       14       0  100.00%
packages/ragbits-core/tests/unit/utils/test_decorators.py                                                       26       2  92.31%   17, 39
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                   13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                              9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                            27       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chroma.py                                                   37       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_in_memory.py                                                74       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                         2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                           70       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                              60       3  95.00%   55, 98, 145
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                               37       2  94.59%   34, 107
packages/ragbits-document-search/src/ragbits/document_search/documents/exceptions.py                            11       5  54.55%   7-8, 17, 26-27
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                               93      16  82.80%   13-14, 63, 76, 160-165, 203-206, 210-211
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                    28       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                    10       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                        14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                       11       1  90.91%   33
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/__init__.py        4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py        44       4  90.91%   97, 102-103, 135
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py         38      15  60.53%   61-68, 75-83, 95, 108
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py            19       6  68.42%   23, 35-43
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py          32      10  68.75%   49, 60-61, 76-79, 103-118
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                   15       4  73.33%   39-44
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                        7       1  85.71%   32
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/llm.py                        20       9  55.00%   27-28, 45-48, 65-67
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                        4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/prompts.py                    16       4  75.00%   49-54
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                    12       1  91.67%   25
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                         6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                         6       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                               0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                   0       0  100.00%
packages/ragbits-document-search/tests/integration/test_sources.py                                              23      10  56.52%   22-32, 40-45
packages/ragbits-document-search/tests/integration/test_unstructured.py                                         47      10  78.72%   46-52, 65-71
packages/ragbits-document-search/tests/unit/__init__.py                                                          0       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                          17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                             75       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                                   13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                    15       0  100.00%
packages/ragbits-document-search/tests/unit/test_local_file_source.py                                           13       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                   31       0  100.00%
packages/ragbits-document-search/tests/unit/test_sources.py                                                     25       0  100.00%
TOTAL                                                                                                         2004     161  91.97%

Diff against main

Filename                                                                 Stmts    Miss  Cover
---------------------------------------------------------------------  -------  ------  --------
packages/ragbits-core/src/ragbits/core/vector_stores/__init__.py           +12      +1  +91.67%
packages/ragbits-core/src/ragbits/core/vector_stores/base.py               +20       0  +100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/chroma.py             +40      +2  +95.00%
packages/ragbits-core/src/ragbits/core/vector_stores/in_memory.py          +23       0  +100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chroma.py              +37       0  +100.00%
packages/ragbits-core/tests/unit/vector_stores/test_in_memory.py           +74       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py       +1       0  +100.00%
TOTAL                                                                     +207      +3  +0.16%

Results for commit: ba69f30

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

github-actions · 2024-10-28T01:45:46Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1)

CRITICAL: HuggingFace (hugging-face-access-token)
════════════════════════════════════════
Hugging Face Access Token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt:36162
────────────────────────────────────────
36160 mncai/Llama2-7B-Active_3rd-floor-LoRA-dim64_epoch4
36161 ajcdp/CM
36162 [ Nagharjun17/*************************************
36163 BigSalmon/InformalToFormalLincoln114Paraphrase
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1288
────────────────────────────────────────
1286 alert_to_webhook_url: Optional[Dict] = Field(
1287 None,
1288 [ bhook_url: {'budget_alerts': '*****************************************************************************'}`",
1289 )
────────────────────────────────────────

akonarski-ds

lgtm

packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py

refactor chroma client

8487460

fix tests

fc6d636

micpst linked an issue Oct 28, 2024 that may be closed by this pull request

refactor(vector-stores): vector store interface #152

Closed

4 tasks

micpst marked this pull request as ready for review October 28, 2024 01:54

micpst changed the title ~~refactor: chroma client~~ refactor(vector_stores): vector store interface Oct 28, 2024

micpst changed the title ~~refactor(vector_stores): vector store interface~~ refactor(vector-stores): vector store interface Oct 28, 2024

micpst added 2 commits October 28, 2024 03:03

small fix

792e528

merge main

9f767fc

akonarski-ds approved these changes Oct 28, 2024

View reviewed changes

packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py Outdated Show resolved Hide resolved

micpst added 4 commits October 28, 2024 12:51

renames + small fixes from main

1c8407d

fix docs

7d52b47

examples formatting

7df04db

examples formatting

ba69f30

micpst merged commit 55b8c4e into main Oct 28, 2024
6 checks passed

micpst deleted the mp/refactor-chroma-store branch October 28, 2024 12:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(vector-stores): vector store interface #151

refactor(vector-stores): vector store interface #151

micpst commented Oct 27, 2024

github-actions bot commented Oct 27, 2024 •

edited

Loading

github-actions bot commented Oct 28, 2024 •

edited

Loading

akonarski-ds left a comment

refactor(vector-stores): vector store interface #151

refactor(vector-stores): vector store interface #151

Conversation

micpst commented Oct 27, 2024

github-actions bot commented Oct 27, 2024 • edited Loading

Code Coverage Summary

Diff against main

github-actions bot commented Oct 28, 2024 • edited Loading

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

akonarski-ds left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 27, 2024 •

edited

Loading

github-actions bot commented Oct 28, 2024 •

edited

Loading