Tool Calling - A newbies challenge unsolved

[kimamil]

My challenge is, that I have an idea of what tool-calling is, how it's supposed to work - but can't get it to work using LangGraph. I don't want a simple code-review, but I'd rather want to discuss where my thinking is wrong.

Here's my concept:

• I use LangGraph to design the processing workflow of prompts and inspect what's happening in realtime
• I use the gpt-4o-mini model from OpenAI for the ChatBot and to avoid any side effects of a self-designed ChatBot
• I create a simple custom tool to create passwords using the @tool decorator
• I will tell the chatbot to create a safe password of 8 characters length
• The ChatBot will interpret my prompt and understand, that I ask for a password of a certain length
• The ChatBot will recognizing access to a tool to create passwords, extract the information that is requested as arguments for the tool and suggest to use it (creates a "tool_call" in the AI message)
• The tool is executed and will return a safe password which is communicated back to me as a user

Here's my simple setup:

import random
import string
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode, tools_condition


memory = MemorySaver()


@tool
def password(length: int) -> str:
    """Create strong passwords."""
    return ''.join(
        random.choices(
            string.ascii_letters
            + string.digits
            + string.punctuation,
            k=length)
        )


tools = [password]

tool_node = ToolNode(tools=tools)

chatbot = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2
)

chatbot.bind_tools(tools=tools)


def chatbot_node(state: MessagesState):
    return {"messages": [chatbot.invoke(state.get("messages"))]}


graph_builder = StateGraph(MessagesState)

graph_builder.add_node("chatbot", chatbot_node)
graph_builder.add_node("tools", tool_node)

graph_builder.add_edge(START, "chatbot")

graph_builder.add_conditional_edges("chatbot", tools_condition)
graph_builder.add_edge("tools", "chatbot")

graph_builder.add_edge("chatbot", END)

graph = graph_builder.compile(checkpointer=memory)

Here's a LangSmith trace to an exemplary call: https://smith.langchain.com/public/9c7d0a6d-ab3d-47c1-8958-49bdfba2c32d/r

Whatever prompts I try, I don't get the tool to be called.

So what am I doing wrong?

[kimamil]

So, with a bit of experimenting and further reading, I found, that using the prebuilt create_react_agent (https://langchain-ai.github.io/langgraph/reference/prebuilt/#langgraph.prebuilt.chat_agent_executor.create_react_agent) works as expected ...

modifying the code like this creates tool_calls as expected:

import random
import string
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode, tools_condition

memory = MemorySaver()

@tool
def password(length: int) -> str:
"""Create strong passwords."""
return ''.join(
  random.choices(
    string.ascii_letters
    + string.digits
    + string.punctuation,
    k=length)
)

tools = [password]

chatbot = ChatOpenAI(
  model="gpt-4o-mini",
  temperature=0,
  max_tokens=None,
  timeout=None,
  max_retries=2
)

graph = create_react_agent(
  model=chatbot,
  tools=tools,
  checkpointer=memory)

So I assume there's a difference in how the model is used - maybe a different prompt template?

[kimamil]

The issue is with the way I used .bind_tools()

.bind_tools() returns a new Runnable, the result must be reassigned to the variable of the initial Runnable.

Slightly modified code that runs perfectly:

import random
import string
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.prebuilt import ToolNode, tools_condition


memory = MemorySaver()


@tool
def password(length: int) -> str:
    """Create strong passwords."""
    return ''.join(
        random.choices(
            string.ascii_letters
            + string.digits
            + string.punctuation,
            k=length)
        )


tools = [password]

tool_node = ToolNode(tools=tools)

chatbot = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2
)

chatbot = chatbot.bind_tools(tools=tools) # reassign the result of .bind_tools() to chatbot


def chatbot_node(state: MessagesState):
    return {"messages": [chatbot.invoke(state.get("messages"))]}


graph_builder = StateGraph(MessagesState)

graph_builder.add_node("chatbot", chatbot_node)
graph_builder.add_node("tools", tool_node)

graph_builder.add_edge(START, "chatbot")

graph_builder.add_conditional_edges("chatbot", tools_condition)
graph_builder.add_edge("tools", "chatbot")

graph_builder.add_edge("chatbot", END)

graph = graph_builder.compile(checkpointer=memory)