====== Tool Use for LLM Agents ======
===== Introduction =====
Tool use is a critical capability for Large Language Model (LLM) agents, enabling them to interact with external systems, access up-to-date information, and perform actions beyond their inherent knowledge(([[https://arxiv.org/abs/2302.04761|Schick, T. et al. "Toolformer: Language Models Can Teach Themselves to Use Tools." arXiv:2302.04761, 2023.]])). This functionality allows LLMs to handle complex tasks that require real-time data retrieval or specific operations. Tool use represents a fundamental paradigm shift from passive LLM processing to active decision-making, where agents determine actions rather than simply process inputs, with key concepts including agent autonomy, tool calling, function calling, memory management, and planning(([[https://cobusgreyling.substack.com/p/the-evolution-of-shared-language|Cobus Greyling - Agency & Tools (2026.]])).

===== Python Example =====
<code python>
import json
from [[openai|openai]] import [[openai|OpenAI]]

client = [[openai|OpenAI]]()

Define tools with JSON Schema for the model
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City name"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["location"],
        },
    },
}]

Tool implementation registry
def get_weather(location: str, unit: str = "celsius") -> str:
    # In production, call a real weather API here
    return json.dumps({"location": location, "temp": 22, "unit": unit, "condition": "sunny"})

TOOL_REGISTRY = {"get_weather": get_weather}

def run_with_tools(user_message: str) -> str:
    """Complete tool-use loop: send message, execute tools, return final response."""
    messages = [{"role": "user", "content": user_message}]
    response = client.chat.completions.create(
        model="gpt-4o", messages=messages, tools=tools
    )
    msg = response.choices[0].message

    # If the model wants to call tools, execute them
    while msg.tool_calls:
        messages.append(msg)
        for call in msg.tool_calls:
            func = TOOL_REGISTRY[call.function.name]
            args = json.loads(call.function.arguments)
            result = func(**args)
            messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
        response = client.chat.completions.create(
            model="gpt-4o", messages=messages, tools=tools
        )
        msg = response.choices[0].message
    return msg.content

print(run_with_tools("What is the weather in Paris?"))
</code>

===== Frameworks and Libraries =====
==== LangChain ====
  * **Website**: [[https://www.[[langchain|langchain]].com/]]
  * **[[github|GitHub]]**: [[https://github.com/[[langchain|langchain]]-ai/langchain]]
  * **Features**:
    * Extensive toolkit for agent-tool interactions
    * Pre-built tools for tasks such as web search, mathematical computations, and code execution
    * Capabilities for creating custom tools

==== AutoGen ====
  * **[[github|GitHub]]**: [[https://github.com/[[microsoft|microsoft]]/autogen]]
  * **Features**:
    * Flexible tool integration within [[multi_agent_systems|multi-agent systems]]
    * Support for code execution and external API calls

==== LlamaIndex ====
  * **Website**: [[https://www.[[llamaindex|llamaindex]].ai/]]
  * **[[github|GitHub]]**: [[https://github.com/run-llama/llama_index]]
  * **Features**:
    * Data connectors and structuring for tool creation
    * Advanced retrieval interfaces for data-centric tools

==== Haystack ====
  * **Website**: [[https://[[haystack|haystack]].deepset.ai/]]
  * **[[github|GitHub]]**: [[https://github.com/deepset-ai/haystack]]
  * **Features**:
    * SearchEngine tool for information retrieval
    * TopPSampler for text generation tasks

==== BMTools ====
  * **[[github|GitHub]]**: [[https://github.com/OpenBMB/BMTools]]
  * **Features**:
    * Platform for building and sharing tools
    * Extends language models with various capabilities

===== Types of Tools =====
  * **Web search tools**
  * **Mathematical computation tools** (e.g., WolframAlpha)
  * **Code execution tools**
  * **Database query tools** (e.g., SQL, CSV, JSON)
  * **API interaction tools**
  * **Vector store tools** for efficient data retrieval
  * **Image analysis tools**

===== Tool Integration Approaches =====
==== Function Calling ====
  * Allows agents to select and use appropriate tools via [[function_calling|function calling]] interfaces(([[https://arxiv.org/abs/2210.03629|Yao, S. et al. "ReAct: Synergizing Reasoning and Acting in Language Models." arXiv:2210.03629, 2022.]])) based on task requirements. Tool calling mechanisms enable LLMs to invoke external tools and functions to accomplish tasks beyond text generation, with OpenAI's function calling API and LangChain's agent abstractions converging on similar implementations from different directions to become a standard capability in agent development(([[https://cobusgreyling.substack.com/p/the-evolution-of-shared-language|Cobus Greyling (LLMs) (2026]])).
  * Supports dynamic tool selection and parameter passing

==== Retrieval-Augmented Generation (RAG) ====
  * Enhances tool use by providing relevant context from external data sources
  * Improves the accuracy and relevance of tool outputs

==== Tool-Augmented Language Models ====
  * Models like [[toolformer|Toolformer]](([[https://arxiv.org/abs/2302.04761|Schick et al., 2023]])) and TALM are fine-tuned for tool interactions
  * Exhibit enhanced ability to use external APIs and tools effectively

===== Models Optimized for Tool Use =====
The emergence of models specifically designed for agentic and tool-heavy tasks represents a significant advancement in agent capabilities. MiniMax M2.5 and M2.7 models have gained prominence for their effectiveness in local agent development, providing reliable tool integration and functional execution without reliance on external cloud services(([[https://www.latent.space/p/ainews-top-local-models-list-april|Latent Space - AI News: Top Local Models List (2024]])). These models are preferred by developers building self-contained agent systems that prioritize tool reliability and local inference.

===== Harness Engineering =====
Harness engineering represents a technical shift where performance gains are derived from the surrounding infrastructure and toolsets rather than solely from [[modelweights|model weights]](([[https://sub.thursdai.news/p/thursdai-ai-engineer-europe-mythos|Thursdai - "Harness Engineering: The AI Engineering Paradigm Shift" (2024]])). This approach emphasizes building complex agentic environments with extensive tool ecosystems. Organizations like Meta have demonstrated this principle by constructing systems with 16 hidden tools, while projects like [[openclaw|OpenClaw]] showcase the value of extensive codebases supporting LLM agents. This paradigm suggests that future performance improvements will increasingly come from intelligent tool orchestration and environmental design rather than model scaling alone.

===== Challenges in Tool Use =====
  * **Quality and Availability of Tool Documentation**: Diverse, redundant, or incomplete documentation can hinder effective [[tool_utilization|tool utilization]].
  * **Decision-Making in Tool Use**(([[https://arxiv.org/abs/2205.00445|Karpas, E. et al. "MRKL Systems." arXiv:2205.00445, 2022.]])): LLMs may struggle to determine when and which tools to use, affecting performance.

===== Recent Advancements =====
  * **ToolLLM**: An open platform that enables LLMs to master thousands of real-world APIs, improving their ability to execute complex instructions and generalize to unseen APIs(([[https://arxiv.org/abs/2307.16789|Qin, Y. et al. "ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs." arXiv:2307.16789, 2023.]])).
  * **GPT4Tools**(([[https://arxiv.org/abs/2305.18752|Yang, R. et al. "GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction." arXiv:2305.18752, 2023.]])): A framework that allows open-source LLMs to use multimodal tools through self-instruction, enhancing their problem-solving capabilities.

===== See Also =====

  * [[memory|Memory Management for LLM Agents]]
  * [[agent_task_management|Agent-Based Task Management with LLMs]]
  * [[agenttuning|AgentTuning: Enabling Generalized Agent Capabilities in LLMs]]
  * [[tool_using_agents|Tool-Using Agents]]
  * [[toolllm|ToolLLM: Facilitating Large Language Models to Master 16,000+ Real-World APIs]]

===== References =====