Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable the integration tests for Ollama #802

Merged
merged 3 commits into from
Jan 29, 2025
Merged

Conversation

rdimitrov
Copy link
Member

@rdimitrov rdimitrov commented Jan 29, 2025

The following PR enables the integration tests for Ollama.

Details:

  • Starts the CodeGate container in the host network (so everything can talk to each other without the need to specify explict ports), we don't have port conflicts anyway
  • Starts the Ollama container and tests if it runs
  • Downloads and runs the qwen2.5-coder:0.5b model
  • Updates the tests to use qwen2.5-coder:0.5b
  • Updates the chat test to expect a simpler welcome message
  • Adds a missing header in the tests

Fixes: #815

Signed-off-by: Radoslav Dimitrov <[email protected]>
@rdimitrov rdimitrov force-pushed the ollama-integration-tests branch from 1da49fc to e217405 Compare January 29, 2025 07:36
@jhrozek
Copy link
Contributor

jhrozek commented Jan 29, 2025

I think this is a bug in the text that uncovers a bug in codegate:

decoded_line = ' {"error": "[Errno -2] Name or service not known", "type": "error",      │ │
│ │                    "choices": ['+2                                                           │ │
│ │                e = IndexError('list index out of range')                                     │ │
│ │        json_line = {                                                                         │ │
│ │                    │   'error': '[Errno -2] Name or service not known',                      │ │
│ │                    │   'type': 'error',                                                      │ │
│ │                    │   'choices': []                                                         │ │
│ │                    }                                                                         │ │
│ │             line = b'data: {"error": "[Errno -2] Name or service not known", "type":         │ │
│ │                    "error", "choice'+7                                                       │ │

So what happens here is that we get a line and expect it to cointain choices but it only contains an error. We should handle the error, perhaps by raising an HTTP exception. But the root cause is that we cannot reach a service (or ollama can't?)

@rdimitrov rdimitrov force-pushed the ollama-integration-tests branch from 8a6881f to bc8bd40 Compare January 29, 2025 12:15
@rdimitrov
Copy link
Member Author

I think this is a bug in the text that uncovers a bug in codegate:

decoded_line = ' {"error": "[Errno -2] Name or service not known", "type": "error",      │ │
│ │                    "choices": ['+2                                                           │ │
│ │                e = IndexError('list index out of range')                                     │ │
│ │        json_line = {                                                                         │ │
│ │                    │   'error': '[Errno -2] Name or service not known',                      │ │
│ │                    │   'type': 'error',                                                      │ │
│ │                    │   'choices': []                                                         │ │
│ │                    }                                                                         │ │
│ │             line = b'data: {"error": "[Errno -2] Name or service not known", "type":         │ │
│ │                    "error", "choice'+7                                                       │ │

So what happens here is that we get a line and expect it to cointain choices but it only contains an error. We should handle the error, perhaps by raising an HTTP exception. But the root cause is that we cannot reach a service (or ollama can't?)

Thanks! I'll file a separate issue for this and fix it there.

@rdimitrov rdimitrov changed the title Add integration tests for Ollama Enable the integration tests for Ollama Jan 29, 2025
Copy link
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yay!

@rdimitrov rdimitrov merged commit d897f7e into main Jan 29, 2025
4 checks passed
@rdimitrov rdimitrov deleted the ollama-integration-tests branch January 29, 2025 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable the integration tests for Ollama
2 participants