[DERCBOT-1314] Add documentsRequired Setting to BotRAGConfiguration

[scezen]

Related Tickets

JIRA: [DERCBOT-1314]

Description

This PR introduces a new boolean attribute documentsRequired to the BotRAGConfiguration and integrates it across the system. The documentsRequired attribute determines whether documents are mandatory for processing the RAG chain.

Key Changes:

1. Model Updates:

   • Added documentsRequired to BotRAGConfiguration (default: true) and BotRAGConfigurationDTO
   • Ensured backward compatibility by handling cases where documentsRequired is absent in the database.

2. Orchestrator Enhancements:

   • Updated RAGQuery to include documentsRequired as an optional parameter.
   • Modified __rag_guard in rag_chain.py to respect the documentsRequired setting.

3. Bot API Modifications:

   • Updated RAGAnswerHandler to fetch the documentsRequired setting from the bot configuration and pass it to the RAG query.

Testing :

• We can block the retrieval of documents by adding a filter to pgvectors_params.py as so:

    def to_dict(self):
        return {
            'k': self.k,
            'filter': {'tag1': 'vector stores'},
        }

• Then we are free to test if an answer is still provided while no retrieval of document is made.

[assouktim]

LGTM, just update the unit tests (python) and everything should be ready.

[scezen]

Tests added @assouktim

[assouktim]

Thanks a lot.

[Benvii]

Thanks good PR, I just found an edge case maybe we can argue about it @assouktim @scezen but it's also easy to handle it if we want.

[scezen]

Fixed the edge case, now handles missing documents without 'no_answer' configuration @Benvii

[Benvii]

• Remove the last condition see my comment.
• Add a test without the 'no_answer' input to check that it works

[Benvii]

Thanks for the last updates ok for me.