feat(document-search): Allow to create DocumentSearch instances from config #62

akotyla · 2024-10-03T12:16:20Z

No description provided.

github-actions · 2024-10-03T12:17:48Z

Code Coverage Summary

Filename                                                                                                     Stmts    Miss  Cover    Missing
---------------------------------------------------------------------------------------------------------  -------  ------  -------  --------------------------------
packages/ragbits-core/src/ragbits/core/__init__.py                                                               0       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                 5       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                   11       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/exceptions.py                                                 14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                    28      19  32.14%   7-8, 43-51, 69-85
packages/ragbits-core/src/ragbits/core/embeddings/local.py                                                      39      25  35.90%   9-10, 35-45, 58-70, 74-76, 80-81
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                        4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                          4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                             33       4  87.88%   33, 52, 88, 97
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                          25       4  84.00%   8-9, 54, 85
packages/ragbits-core/src/ragbits/core/llms/local.py                                                            24      10  58.33%   8-9, 42-47, 57, 70-71
packages/ragbits-core/src/ragbits/core/llms/types.py                                                             8       2  75.00%   24, 28
packages/ragbits-core/src/ragbits/core/llms/clients/__init__.py                                                  4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/base.py                                                     23       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/clients/exceptions.py                                               14       6  57.14%   7-8, 17, 26-27, 36
packages/ragbits-core/src/ragbits/core/llms/clients/litellm.py                                                  50      10  80.00%   10-11, 70, 108, 122-127
packages/ragbits-core/src/ragbits/core/llms/clients/local.py                                                    37      12  67.57%   11-12, 62-70, 92-103
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                        2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                           18       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                        34       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                        110       2  98.18%   107, 111
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                              2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                     33       2  93.94%   55-56
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                      26       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                  9       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/__init__.py                                                 13       1  92.31%   29
packages/ragbits-core/src/ragbits/core/vector_store/base.py                                                     12       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py                                           56       7  87.50%   9-10, 60-67, 84, 126
packages/ragbits-core/src/ragbits/core/vector_store/in_memory.py                                                18       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                     0       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                           63       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                             0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                        65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                        143       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                   0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                  30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                     18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                    2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py            3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py       14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py       14       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                   13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                              9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                            27       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chromadb_store.py                                           69       4  94.20%   31, 34, 39, 44
packages/ragbits-core/tests/unit/vector_stores/test_simple_vector_store.py                                      16       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                         2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                           57       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                              55       3  94.55%   52, 95, 124
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                               30       1  96.67%   34
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                               46       9  80.43%   13-14, 58, 87, 106, 109, 120-122
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                    27       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                    11       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                        14       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                       11       1  90.91%   29
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured.py                49       5  89.80%   93, 96-97, 127, 151
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                               0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                   13       3  76.92%   29-32
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                        5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                        5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                    13       1  92.31%   27
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                         6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                         7       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                               0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                   0       0  100.00%
packages/ragbits-document-search/tests/integration/test_unstructured.py                                         38       5  86.84%   46-52
packages/ragbits-document-search/tests/unit/__init__.py                                                          0       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                          17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                             50       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                                   13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                    15       0  100.00%
packages/ragbits-document-search/tests/unit/test_gcs_source.py                                                  14       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                   23       0  100.00%
TOTAL                                                                                                         1670     145  91.32%

Diff against main

Filename                                                                                         Stmts    Miss  Cover
---------------------------------------------------------------------------------------------  -------  ------  --------
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                       +7       0  +100.00%
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                           +4       0  +100.00%
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                     +9       0  +100.00%
packages/ragbits-core/src/ragbits/core/vector_store/__init__.py                                     +9      +1  -7.69%
packages/ragbits-core/src/ragbits/core/vector_store/chromadb_store.py                               +6      +3  -4.50%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                              +12      -1  +2.22%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py        +7       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py       +11       0  +100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py      +13      +3  -23.08%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py       +13      +1  -7.69%
packages/ragbits-document-search/tests/unit/test_document_search.py                                 +9       0  +100.00%
TOTAL                                                                                             +100      +7  +0.11%

Results for commit: b268df1

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

github-actions · 2024-10-03T12:20:24Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1)

CRITICAL: HuggingFace (hugging-face-access-token)
════════════════════════════════════════
Hugging Face Access Token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt:36162
────────────────────────────────────────
36160 mncai/Llama2-7B-Active_3rd-floor-LoRA-dim64_epoch4
36161 ajcdp/CM
36162 [ Nagharjun17/*************************************
36163 BigSalmon/InformalToFormalLincoln114Paraphrase
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1288
────────────────────────────────────────
1286 alert_to_webhook_url: Optional[Dict] = Field(
1287 None,
1288 [ bhook_url: {'budget_alerts': '*****************************************************************************'}`",
1289 )
────────────────────────────────────────

packages/ragbits-core/src/ragbits/core/embeddings/__init__.py

packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py

packages/ragbits-document-search/src/ragbits/document_search/utils.py

akonarski-ds · 2024-10-04T08:17:42Z

Could you provide an example in /packages/ragbits-document-search/examples/?
Additionally, adding some tests would be great.

packages/ragbits-document-search/src/ragbits/document_search/_main.py

packages/ragbits-document-search/tests/unit/test_document_search.py

packages/ragbits-document-search/examples/from_config_example.py

packages/ragbits-document-search/src/ragbits/document_search/_main.py

…entsearch-instances-from-config

mhordynski

lgtm

akotyla added 3 commits September 26, 2024 14:15

Add semantic release workflow

6055d6f

Allow to create DocumentSearch instances from config

164623f

Small fix

b658be6

akotyla added feature New feature or request document search Changes to the document search package labels Oct 3, 2024

akotyla added this to the Ragbits 0.3 milestone Oct 3, 2024

Restore example

2cc5b01

ludwiktrammer reviewed Oct 3, 2024

View reviewed changes

Fixes after review

8bf521c

akonarski-ds reviewed Oct 4, 2024

View reviewed changes

packages/ragbits-document-search/src/ragbits/document_search/_main.py Outdated Show resolved Hide resolved

micpst reviewed Oct 4, 2024

View reviewed changes

packages/ragbits-document-search/src/ragbits/document_search/_main.py Show resolved Hide resolved

akotyla added 4 commits October 7, 2024 08:51

Fixes after comments

e494ef9

Resolve conflicts

843311f

Small fixes

d306f84

Fix tests

227fddc

mhordynski reviewed Oct 7, 2024

View reviewed changes

akotyla added 4 commits October 7, 2024 20:12

Fixes after review

db87234

Integrate config with providers

60a37ec

Add docstring

7badfa0

Fix typing

651ec3e

mhordynski reviewed Oct 10, 2024

View reviewed changes

packages/ragbits-document-search/src/ragbits/document_search/_main.py Show resolved Hide resolved

akotyla added 3 commits October 14, 2024 14:06

Merge branch 'main' into 18-featdocument-search-allow-to-create-docum…

ef9d871

…entsearch-instances-from-config

Fix linter

bf46c47

Fix tests

b268df1

mhordynski approved these changes Oct 14, 2024

View reviewed changes

akotyla merged commit cc33d4c into main Oct 14, 2024
3 checks passed

mhordynski mentioned this pull request Oct 14, 2024

feat(document-search): allow to create DocumentSearch instances from config #18

Closed

akotyla deleted the 18-featdocument-search-allow-to-create-documentsearch-instances-from-config branch October 18, 2024 13:15

mhordynski mentioned this pull request Oct 23, 2024

chore: release ragbits v0.2.0 #142

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(document-search): Allow to create DocumentSearch instances from config #62

feat(document-search): Allow to create DocumentSearch instances from config #62

akotyla commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

akonarski-ds commented Oct 4, 2024

mhordynski left a comment

feat(document-search): Allow to create DocumentSearch instances from config #62

feat(document-search): Allow to create DocumentSearch instances from config #62

Conversation

akotyla commented Oct 3, 2024

github-actions bot commented Oct 3, 2024 • edited Loading

Code Coverage Summary

Diff against main

github-actions bot commented Oct 3, 2024 • edited Loading

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

.venv/lib/python3.10/site-packages/litellm/llms/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

akonarski-ds commented Oct 4, 2024

mhordynski left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading