tool_use/parallel_tools_claude_3_7_sonnet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parallel tool calls on Claude 3.7 Sonnet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Claude 3.7 Sonnet may be less likely to make make parallel tool calls in a response, even when you have not set `disable_parallel_tool_use`. To work around this, we recommend introducing a \"batch tool\" that can act as a meta-tool to wrap invocations to other tools simultaneously. We find that if this tool is present, the model will use it to simultaneously call multiple tools in parallel for you.\n",
    "\n",
    "Let's take a look at the problem, and examine this workaround in more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from anthropic import Anthropic\n",
    "\n",
    "client = Anthropic()\n",
    "MODEL_NAME = \"claude-3-7-sonnet-20250219\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performing a query with multiple tool calls\n",
    "\n",
    "Recall that the default behavior is for Claude to be allowed parallel tool calls. Combined with the default `tool_choice` of `auto`, this means that Claude can call any of the specified tools, or call more than one of them in a single assistant turn.\n",
    "\n",
    "Let's set Claude up with a `get_weather` and `get_time` tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_weather(location):\n",
    "    # Pretend to get the weather, and just return a fixed value.\n",
    "    return f\"The weather in {location} is 72 degrees and sunny.\"\n",
    "\n",
    "def get_time(location):\n",
    "    # Pretend to get the time, and just return a fixed value.\n",
    "    return f\"The time in {location} is 12:32 PM.\"\n",
    "\n",
    "weather_tool = {\n",
    "    \"name\": \"get_weather\",\n",
    "    \"description\": \"Gets the weather for in a given location\",\n",
    "    \"input_schema\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"location\": {\n",
    "                \"type\": \"string\",\n",
    "                \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"location\"]\n",
    "    }\n",
    "}\n",
    "\n",
    "time_tool = {\n",
    "    \"name\": \"get_time\",\n",
    "    \"description\": \"Gets the time in a given location\",\n",
    "    \"input_schema\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"location\": {\n",
    "                \"type\": \"string\",\n",
    "                \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"location\"]\n",
    "    }\n",
    "}\n",
    "\n",
    "def process_tool_call(tool_name, tool_input):\n",
    "    if tool_name == \"get_weather\":\n",
    "        return get_weather(tool_input[\"location\"])\n",
    "    elif tool_name == \"get_time\":\n",
    "        return get_time(tool_input[\"location\"])\n",
    "    else:\n",
    "        raise ValueError(f\"Unexpected tool name: {tool_name}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's provide Claude with these tools and perform a query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I'll check the current weather and time in San Francisco for you.\n",
      "Tool: get_weather({'location': 'San Francisco, CA'})\n"
     ]
    }
   ],
   "source": [
    "def make_query_and_print_result(messages, tools=None):\n",
    "    response = client.messages.create(\n",
    "        model=MODEL_NAME,\n",
    "        messages=messages,\n",
    "        max_tokens=1000,\n",
    "        tool_choice={\"type\": \"auto\"},\n",
    "        tools=tools or [weather_tool, time_tool],\n",
    "    )\n",
    "\n",
    "    for block in response.content:\n",
    "        match block.type:\n",
    "            case \"text\":\n",
    "                print(block.text)\n",
    "            case \"tool_use\":\n",
    "                print(f\"Tool: {block.name}({block.input})\")\n",
    "            case _:\n",
    "                raise ValueError(f\"Unexpected block type: {block.type}\")\n",
    "\n",
    "    return response\n",
    "\n",
    "\n",
    "MESSAGES = [\n",
    "    {\"role\": \"user\", \"content\": \"What's the weather and time in San Francisco?\"}\n",
    "]\n",
    "\n",
    "response = make_query_and_print_result(MESSAGES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how claude returned with a single tool call for the weather, even though we asked for both?\n",
    "\n",
    "Let's see what happens if we call the weather tool and proceed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tool: get_time({'location': 'San Francisco, CA'})\n"
     ]
    }
   ],
   "source": [
    "last_tool_call = response.content[1]\n",
    "\n",
    "MESSAGES.append({\"role\": \"assistant\", \"content\": response.content})\n",
    "MESSAGES.append(\n",
    "    {\n",
    "\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            {\n",
    "                \"type\": \"tool_result\",\n",
    "                \"tool_use_id\": last_tool_call.id,\n",
    "                \"content\": process_tool_call(response.content[1].name, response.content[1].input),\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "\n",
    "response = make_query_and_print_result(MESSAGES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice now that Claude made a second tool call to get the time. While this technically happened immediately, this is potentially wasteful because it required \"back and forth\" – first Claude asked for the weather, then we had to process it, and _then_ Claude asked for the time, and now we have to process _that_.\n",
    "\n",
    "Claude will still do the right thing with the results, but it may be beneficial to encourage Claude to use both in one call, so we can process it simultaneously."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introducing a batch tool\n",
    "\n",
    "Let's introduce a `batch_tool`, so that Claude can have an opportunity to use it to combine multiple tool calls into one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "batch_tool = {\n",
    "    \"name\": \"batch_tool\",\n",
    "    \"description\": \"Invoke multiple other tool calls simultaneously\",\n",
    "    \"input_schema\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"invocations\": {\n",
    "                \"type\": \"array\",\n",
    "                \"description\": \"The tool calls to invoke\",\n",
    "                \"items\": {\n",
    "                    \"types\": \"object\",\n",
    "                    \"properties\": {\n",
    "                        \"name\": {\n",
    "                            \"types\": \"string\",\n",
    "                            \"description\": \"The name of the tool to invoke\"\n",
    "                        },\n",
    "                        \"arguments\": {\n",
    "                            \"types\": \"string\",\n",
    "                            \"description\": \"The arguments to the tool\"\n",
    "                        }\n",
    "                    },\n",
    "                    \"required\": [\"name\", \"arguments\"]\n",
    "                }\n",
    "            }\n",
    "        },\n",
    "        \"required\": [\"invocations\"]\n",
    "    }\n",
    "}\n",
    "\n",
    "def process_tool_with_maybe_batch(tool_name, tool_input):\n",
    "    if tool_name == \"batch_tool\":\n",
    "        results = []\n",
    "        for invocation in tool_input[\"invocations\"]:\n",
    "            results.append(process_tool_call(invocation[\"name\"], json.loads(invocation[\"arguments\"])))\n",
    "        return '\\n'.join(results)\n",
    "    else:\n",
    "        return process_tool_call(tool_name, tool_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try to provide Claude with the existing weather and time tool, along with this new batch tool, and see what happens when we make a query requiring the weather and time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I can help you check both the weather and the time in San Francisco. Let me get that information for you right away.\n",
      "Tool: batch_tool({'invocations': [{'name': 'get_weather', 'arguments': '{\"location\": \"San Francisco, CA\"}'}, {'name': 'get_time', 'arguments': '{\"location\": \"San Francisco, CA\"}'}]})\n"
     ]
    }
   ],
   "source": [
    "MESSAGES = [\n",
    "    {\"role\": \"user\", \"content\": \"What's the weather and time in San Francisco?\"}\n",
    "]\n",
    "\n",
    "response = make_query_and_print_result(MESSAGES, tools=[weather_tool, time_tool, batch_tool])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how this time, Claude used the batch tool to query both the time and weather in one go. This allows us to process them simultaneously, potentially improving overall latency to the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's the information you requested:\n",
      "\n",
      "Weather in San Francisco, CA: 72 degrees and sunny\n",
      "Time in San Francisco, CA: 12:32 PM\n",
      "\n",
      "Is there anything else you'd like to know about San Francisco?\n"
     ]
    }
   ],
   "source": [
    "last_tool_call = response.content[1]\n",
    "\n",
    "MESSAGES.append({\"role\": \"assistant\", \"content\": response.content})\n",
    "MESSAGES.append(\n",
    "    {\n",
    "\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            {\n",
    "                \"type\": \"tool_result\",\n",
    "                \"tool_use_id\": last_tool_call.id,\n",
    "                \"content\": process_tool_with_maybe_batch(response.content[1].name, response.content[1].input),\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "\n",
    "response = make_query_and_print_result(MESSAGES)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py311",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}