Can't Use Ollama Models with LATS Example Notebook

[rakataprime]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.

Example Code

so taking the lats example notebook from the repo and trying to switch out the OpenAI models for ollama models
gives bugs related to the tools either bind_tools not found or doesn't return a tool correctly. The following 4 approaches didn't work including the OpenAI llama api.

import os 
from langchain_openai import ChatOpenAI
from langchain_experimental.llms.ollama_functions import OllamaFunctions
from langchain_community.chat_models import ChatOllama
from langchain_community.llms import Ollama

os.environ["OPENAI_API_BASE"] = "http://localhost:11434/v1"
os.environ["OPENAI_MODEL_NAME"] = "eramax/opencodeinterpreter:ds-33b-q4"
os.environ["OPENAI_API_KEY"] =  "NA"

llm = ChatOpenAI(model="eramax/opencodeinterpreter:ds-33b-q4")
# llm = ChatOllama(model="eramax/opencodeinterpreter:ds-33b-q4", temperature=0)
# llm = OllamaFunctions(model="eramax/opencodeinterpreter:ds-33b-q4")
# llm = Ollama(model="eramax/opencodeinterpreter:ds-33b-q4")```

### Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "/home/ubuntu/lats.py", line 547, in <module>
    for step in graph.stream({"input": prompt}):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 615, in transform
    for chunk in self._transform_stream_with_config(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1513, in _transform_stream_with_config
    chunk: Output = context.run(next, iterator)  # type: ignore
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 355, in _transform
    _interrupt_or_proceed(done, inflight, step)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langgraph/pregel/__init__.py", line 698, in _interrupt_or_proceed
    raise exc
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 4069, in invoke
    return self.bound.invoke(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2075, in invoke
    input = step.invoke(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3523, in invoke
    return self._call_with_config(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1262, in _call_with_config
    context.run(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3397, in _invoke
    output = call_func_with_variable_args(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/home/ubuntu/thumperai/crew/lats.py", line 295, in generate_initial_response
    reflection = reflection_chain.invoke(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3523, in invoke
    return self._call_with_config(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1262, in _call_with_config
    context.run(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3397, in _invoke
    output = call_func_with_variable_args(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/langchain_core/runnables/config.py", line 326, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
  File "/home/ubuntu/lats.py", line 242, in reflection_chain
    reflection = tool_choices[0]
IndexError: list index out of range

### Description

I'm trying to make the lats example work with an Ollama model using the langchain Ollama api's, but there is a bug or issue with the tool support.

### System Info

langchain==0.1.11
langchain-community==0.0.27
langchain-core==0.1.30
langchain-experimental==0.0.53
langchain-openai==0.0.5
langchain-text-splitters==0.0.1
ubuntu 22.04
python 3.10.12

[hinthornw]

Yeah ollama would need to support tool calling for the notebook to work without modification..

[hinthornw]

Would be happy to review a PR by anyone who makes a reliable example of LATS with ollama or transformers or another OSS model provider!

[rakataprime]

@hinthornw I would gladly take a shot at trying to complete this. Is there an recommended starting place for what OSS models / API are easiest to add support for or experiment with for trying to add tool calling support? I have struggled to find a documented list of recommended OSS models/ API for langchain tool compatibility / function calling.

[hinthornw]

What size model are you able to run locally? There still isn't excellent multi-turn tool calling out of the box for most OSS models, though mixtral seems to be the best (maybe gemma is good too - haven't checked)

[rakataprime]

I have a dev server with 2 4090s (24gb VRAM) but also can use a100 / a6000 cards if the larger models would improve the performance. I have been interested in some of the more recent coding models that beat gpt-3.5 turbo on some of the coding benchmarks eval plus like the opencoder-ds 30B, deepseek coder, and starcoder2 models. I am not sure what benchmark or dataset would be best for measuring multiturn performance, tool usage performance, or function calling performance. I had thought that some of these coding models like open code interpreter-ds had been tuned to try to be able to do multiturn coding approaches, but i don't know if they do multiturn AND function calling AND tool use all together. I may have some interest in building a dataset for finetuning some of the OSS models for function calling and tool support if there aren't any other alternatives.

[robertmujica]

i have same issue,

and problem is here,

tool_choices has no items, i don't know if it is a misconfiguration in the graph or actually a missing condition in this function

btw I'm using Llama-2 on-prem and it happens when i run following cell / line