Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MistralAI PromptDriver #1322

Open
1 task done
shhlife opened this issue Nov 7, 2024 · 1 comment
Open
1 task done

Add MistralAI PromptDriver #1322

shhlife opened this issue Nov 7, 2024 · 1 comment
Labels
type: feature request New feature or request

Comments

@shhlife
Copy link

shhlife commented Nov 7, 2024

Is your feature request related to a problem? Please describe.
A customer from ComfyUI requested access to MistralAI's api, as it offers a nice free tier. I tried using the OpenAiChatPromptDriver to use it by setting the base_url, but received a number of errors.

import os

from dotenv import load_dotenv
from griptape.drivers import OpenAiChatPromptDriver
from griptape.structures import Agent

load_dotenv()

API_KEY = os.getenv("MISTRAL_API_KEY")

# Create an agent

agent = Agent(
    stream=True,
    prompt_driver=OpenAiChatPromptDriver(
        base_url="https://api.mistral.ai/v1",
        api_key=API_KEY,
        model="mistral-small-latest",
    ),
)

agent.run("Hello")

The response was Error code: 422:

[11/08/24 05:19:08] INFO     PromptTask d2e9f21a4d6d459c8c318314ea068c0d
                             Input: Hello
[11/08/24 05:19:09] ERROR    PromptTask d2e9f21a4d6d459c8c318314ea068c0d
                             Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'seed'], 'msg': 'Extra inputs are not permitted', 'input':     
                             None}, {'type': 'extra_forbidden', 'loc': ['body', 'stream_options'], 'msg': 'Extra inputs are not permitted', 'input': {'include_usage': True}}, {'type':
                             'extra_forbidden', 'loc': ['body', 'user'], 'msg': 'Extra inputs are not permitted', 'input': ''}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}      
                             Traceback (most recent call last):
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\tasks\base_task.py", line 158, in run
                                 self.output = self.try_run()
                                               ^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\tasks\prompt_task.py", line 98, in try_run
                                 message = self.prompt_driver.run(self.prompt_stack)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\common\decorators.py", line 14, in decorator
                                 return Observability.observe(
                                        ^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\observability\observability.py", line 36, in observe
                                 return driver.observe(call)
                                        ^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\observability\no_op_observability_driver.py", line 16, in observe 
                                 return call()
                                        ^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\common\observable.py", line 19, in __call__
                                 return self.func(*self.args, **self.kwargs)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\base_prompt_driver.py", line 75, in run
                                 for attempt in self.retrying():
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\tenacity\__init__.py", line 443, in __iter__
                                 do = self.iter(retry_state=retry_state)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\tenacity\__init__.py", line 376, in iter
                                 result = action(retry_state)
                                          ^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\tenacity\__init__.py", line 398, in <lambda>
                                 self._add_action_func(lambda rs: rs.outcome.result())
                                                                  ^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 449, in result
                                 return self.__get_result()
                                        ^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result
                                 raise self._exception
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\base_prompt_driver.py", line 79, in run
                                 result = self.__process_stream(prompt_stack) if self.stream else self.__process_run(prompt_stack)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\base_prompt_driver.py", line 128, in __proc                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\base_prompt_driver.py", line 128, in __process_stream                       for message_delta in message_deltas:
                                 for message_delta in message_deltas:
ess_stream
                                 for message_delta in message_deltas:
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\openai_chat_prompt_driver.py", line 126, in try_stream
                                 result = self.client.chat.completions.create(**params, stream=True)
                                 for message_delta in message_deltas:
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\griptape\drivers\prompt\openai_chat_prompt_driver.py", line 126, in try_stream
                                 result = self.client.chat.completions.create(**params, stream=True)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_uti                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_ut                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_ut                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_utils\_utils.py", line 274, in wrapper
                                 return func(*args, **kwargs)
_utils\_utils.py", line 274, in wrapper
_utils\_utils.py", line 274, in wrapper
                                 return func(*args, **kwargs)
                                        ^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\resources\chat\completions.py", line 668, in create
                                 return self._post(
                                        ^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_base_client.py", line 1260, in post
                                 return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_base_client.py", line 937, in request
                                 return self._request(
                                        ^^^^^^^^^^^^^^
                               File "C:\Users\jason\Documents\GitHub\griptape-intro-demos\.venv\Lib\site-packages\openai\_base_client.py", line 1041, in _request
                                 raise self._make_status_error_from_response(err.response) from None
                             openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'seed'], 'msg': 'Extra inputs
                             are not permitted', 'input': None}, {'type': 'extra_forbidden', 'loc': ['body', 'stream_options'], 'msg': 'Extra inputs are not permitted', 'input': {'include_usage':
                             True}}, {'type': 'extra_forbidden', 'loc': ['body', 'user'], 'msg': 'Extra inputs are not permitted', 'input': ''}]}, 'type': 'invalid_request_error', 'param': None,
                             'code': None}
@shhlife shhlife added the type:enhancement Improvements to existing features label Nov 7, 2024
@collindutter collindutter changed the title Add MistralAI PromptDriver to the framework OpenAiChatPromptDriver Does Not Work With MistralAI Nov 11, 2024
@collindutter collindutter added type:bug Something isn't working and removed type:enhancement Improvements to existing features labels Nov 11, 2024
@collindutter collindutter self-assigned this Nov 11, 2024
@collindutter collindutter changed the title OpenAiChatPromptDriver Does Not Work With MistralAI Add MistralAI PromptDriver Nov 11, 2024
@collindutter collindutter added type:enhancement Improvements to existing features and removed type:bug Something isn't working labels Nov 11, 2024
@collindutter
Copy link
Member

Reverted back to enhancement since I had misinterpreted the issue. Mistral does not claim to be openai-compatible so it is expected that it does not work.

@collindutter collindutter removed their assignment Dec 17, 2024
@collindutter collindutter added type: feature request New feature or request and removed type:enhancement Improvements to existing features labels Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants