Tool Calling

Let the model propose tool calls, then execute them safely in your application and return the results.

Tool Calling lets a model select one or more tools based on the conversation and generate the structured arguments needed to call them. The model does not access databases, call internal services, or perform write operations for your application; your program always performs the actual tool execution.

It works well for looking up orders, reading weather data, searching a knowledge base, calculating values, and performing clearly defined business operations after authorization.

Never execute model-generated arguments directly

Tool names and arguments are model output. Before execution, perform JSON parsing, Schema validation, permission checks, and business allowlist checks. High-risk operations such as payments, deletions, or sending messages should also require user confirmation and idempotency controls.

Complete call loop

1. Define tools

Use a clear name, purpose description, and JSON Schema to tell the model when a tool can be called and what argument structure it expects.

2. Request the model

Send the user input together with the tool definitions. The model can answer directly or return zero, one, or multiple tool calls.

3. Validate and execute

Look up the local implementation by tool name, validate all arguments and caller permissions, then execute it with a timeout. Never allow the model to specify arbitrary functions, URLs, SQL, or shell commands.

4. Return the tool result

Use the call ID returned by the protocol to associate the execution result with the original tool call, then request the model again.

5. Read the final answer

The model can generate a user-facing answer from the tool result or propose additional tool calls. The application should set a maximum loop count and recognize termination conditions.

Define a read-only tool

The following tool only looks up weather data and does not perform external writes:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a specified city",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "City name, such as Shanghai"
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"]
        }
      },
      "required": ["city", "unit"],
      "additionalProperties": false
    }
  }
}

A tool description should explain what the tool does and when to use it. Keep argument names stable, narrow enums when possible, and use required and additionalProperties to restrict unnecessary input.

Complete one Chat tool loop

The following example shows the complete main path: declare a tool, read the call, execute a local function, return the result, and read the final answer.

import json

from openai import OpenAI

client = OpenAI(
    api_key="your-api-key",
    base_url="https://api.agtcloud.cn/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a specified city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string"},
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                    },
                },
                "required": ["city", "unit"],
                "additionalProperties": False,
            },
        },
    }
]


def get_weather(city: str, unit: str) -> dict[str, str | int]:
    if city != "Shanghai" or unit != "celsius":
        raise ValueError("This example only supports Celsius weather for Shanghai")
    return {"city": city, "unit": unit, "temperature": 25}


messages = [{"role": "user", "content": "What is the temperature in Shanghai?"}]
response = client.chat.completions.create(
    model="MODEL_ID",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

assistant_message = response.choices[0].message
messages.append(assistant_message)

for tool_call in assistant_message.tool_calls or []:
    if tool_call.function.name != "get_weather":
        raise ValueError("The model requested an unregistered tool")

    arguments = json.loads(tool_call.function.arguments)
    result = get_weather(**arguments)
    messages.append(
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result, ensure_ascii=False),
        }
    )

if assistant_message.tool_calls:
    final_response = client.chat.completions.create(
        model="MODEL_ID",
        messages=messages,
        tools=tools,
    )
    print(final_response.choices[0].message.content)
else:
    print(assistant_message.content)

Production applications should validate arguments with a JSON Schema validation library and give each tool implementation independent permissions, timeouts, error types, and audit records.

How protocols represent tool calls

Protocol	Tool definition	Model call	Returning a tool result
Chat Completions	`tools[].function`	`tool_calls` in an Assistant Message	`role: tool` with `tool_call_id`
Responses	Function Tool in `tools`	`function_call` Output Item	`function_call_output` with `call_id`
Anthropic Messages	`tools` with `input_schema`	`tool_use` Content Block	`tool_result` Content Block
Gemini	`functionDeclarations`	`functionCall` in a Part	`functionResponse` in a Part

Field names and nesting cannot be mixed across protocols. See the complete structures in Chat Completions, Responses, Anthropic Messages, and Gemini Generate Content.

Multiple calls and Streaming

Do not assume a request can call only one tool. Validate, execute, and return each call separately by call ID.
Only independent, business-approved read-only calls are suitable for parallel execution.
Streaming arguments can be split across multiple deltas; assemble them completely before parsing the JSON.
After a tool result is returned, the model can propose another round of calls; the application needs a maximum round count and total timeout.
Use a business idempotency key when retrying the same write operation to prevent duplicate charges, messages, or resources.

Handle failures

When tool execution fails, do not fabricate a successful result. You can return a redacted error category to the model, such as invalid arguments, insufficient permissions, or a temporarily unavailable upstream service, so the model can explain the issue or choose an allowed alternative.

Do not return internal stack traces, database schemas, access tokens, private network addresses, or unredacted third-party responses directly to the model or user.

Pre-production checklist

Tools can be selected only from a registry or allowlist.
Arguments pass Schema and business-rule validation before execution.
Every tool has permissions, a timeout, a maximum result size, and error handling.
High-risk write operations require user confirmation and support idempotency and auditing.
The call loop has a maximum round count and cannot continue indefinitely under model control.
Streaming tool arguments execute only after they are complete.

To assemble tool arguments while receiving them, continue with Streaming. To make the final answer follow a JSON Schema, see Structured Output.

On this page