smolagents_doc/en/tools.ipynb (582 lines of code) (raw):
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Installation\n",
"! pip install smolagents\n",
"# To install from source instead of the last release, comment the command above and uncomment the following one.\n",
"# ! pip install git+https://github.com/huggingface/smolagents.git"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, we're going to see advanced tool usage.\n",
"\n",
"> [!TIP]\n",
"> If you're new to building agents, make sure to first read the [intro to agents](https://huggingface.co/docs/smolagents/main/en/tutorials/../conceptual_guides/intro_agents) and the [guided tour of smolagents](https://huggingface.co/docs/smolagents/main/en/tutorials/../guided_tour)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What is a tool, and how to build one?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A tool is mostly a function that an LLM can use in an agentic system.\n",
"\n",
"But to use it, the LLM will need to be given an API: name, tool description, input types and descriptions, output type.\n",
"\n",
"So it cannot be only a function. It should be a class.\n",
"\n",
"So at core, the tool is a class that wraps a function with metadata that helps the LLM understand how to use it.\n",
"\n",
"Here's how it looks:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import Tool\n",
"\n",
"class HFModelDownloadsTool(Tool):\n",
" name = \"model_download_counter\"\n",
" description = \"\"\"\n",
" This is a tool that returns the most downloaded model of a given task on the Hugging Face Hub.\n",
" It returns the name of the checkpoint.\"\"\"\n",
" inputs = {\n",
" \"task\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"the task category (such as text-classification, depth-estimation, etc)\",\n",
" }\n",
" }\n",
" output_type = \"string\"\n",
"\n",
" def forward(self, task: str):\n",
" from huggingface_hub import list_models\n",
"\n",
" model = next(iter(list_models(filter=task, sort=\"downloads\", direction=-1)))\n",
" return model.id\n",
"\n",
"model_downloads_tool = HFModelDownloadsTool()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The custom tool subclasses [Tool](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool) to inherit useful methods. The child class also defines:\n",
"- An attribute `name`, which corresponds to the name of the tool itself. The name usually describes what the tool does. Since the code returns the model with the most downloads for a task, let's name it `model_download_counter`.\n",
"- An attribute `description` is used to populate the agent's system prompt.\n",
"- An `inputs` attribute, which is a dictionary with keys `\"type\"` and `\"description\"`. It contains information that helps the Python interpreter make educated choices about the input.\n",
"- An `output_type` attribute, which specifies the output type. The types for both `inputs` and `output_type` should be [Pydantic formats](https://docs.pydantic.dev/latest/concepts/json_schema/#generating-json-schema), they can be either of these: `~AUTHORIZED_TYPES()`.\n",
"- A `forward` method which contains the inference code to be executed.\n",
"\n",
"And that's all it needs to be used in an agent!\n",
"\n",
"There's another way to build a tool. In the [guided_tour](https://huggingface.co/docs/smolagents/main/en/tutorials/../guided_tour), we implemented a tool using the `@tool` decorator. The [tool()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.tool) decorator is the recommended way to define simple tools, but sometimes you need more than this: using several methods in a class for more clarity, or using additional class attributes.\n",
"\n",
"In this case, you can build your tool by subclassing [Tool](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool) as described above."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Share your tool to the Hub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can share your custom tool to the Hub as a Space repository by calling [push_to_hub()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool.push_to_hub) on the tool. Make sure you've created a repository for it on the Hub and are using a token with read access."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model_downloads_tool.push_to_hub(\"{your_username}/hf-model-downloads\", token=\"<YOUR_HUGGINGFACEHUB_API_TOKEN>\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For the push to Hub to work, your tool will need to respect some rules:\n",
"- All methods are self-contained, e.g. use variables that come either from their args.\n",
"- As per the above point, **all imports should be defined directly within the tool's functions**, else you will get an error when trying to call [save()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool.save) or [push_to_hub()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool.push_to_hub) with your custom tool.\n",
"- If you subclass the `__init__` method, you can give it no other argument than `self`. This is because arguments set during a specific tool instance's initialization are hard to track, which prevents from sharing them properly to the hub. And anyway, the idea of making a specific class is that you can already set class attributes for anything you need to hard-code (just set `your_variable=(...)` directly under the `class YourTool(Tool):` line). And of course you can still create a class attribute anywhere in your code by assigning stuff to `self.your_variable`.\n",
"\n",
"\n",
"Once your tool is pushed to Hub, you can visualize it. [Here](https://huggingface.co/spaces/m-ric/hf-model-downloads) is the `model_downloads_tool` that I've pushed. It has a nice gradio interface.\n",
"\n",
"When diving into the tool files, you can find that all the tool's logic is under [tool.py](https://huggingface.co/spaces/m-ric/hf-model-downloads/blob/main/tool.py). That is where you can inspect a tool shared by someone else.\n",
"\n",
"Then you can load the tool with [load_tool()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.load_tool) or create it with [from_hub()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool.from_hub) and pass it to the `tools` parameter in your agent.\n",
"Since running tools means running custom code, you need to make sure you trust the repository, thus we require to pass `trust_remote_code=True` to load a tool from the Hub."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import load_tool, CodeAgent\n",
"\n",
"model_download_tool = load_tool(\n",
" \"{your_username}/hf-model-downloads\",\n",
" trust_remote_code=True\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Use tools from an MCP server"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our `MCPClient` allows you to load tools from an MCP server, and gives you full control over the connection and tool management:\n",
"\n",
"For stdio-based MCP servers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import MCPClient, CodeAgent\n",
"from mcp import StdioServerParameters\n",
"import os\n",
"\n",
"server_parameters = StdioServerParameters(\n",
" command=\"uvx\", # Using uvx ensures dependencies are available\n",
" args=[\"--quiet\", \"pubmedmcp@0.1.3\"],\n",
" env={\"UV_PYTHON\": \"3.12\", **os.environ},\n",
")\n",
"\n",
"with MCPClient(server_parameters) as tools:\n",
" agent = CodeAgent(tools=tools, model=model, add_base_tools=True)\n",
" agent.run(\"Please find the latest research on COVID-19 treatment.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For Streamable HTTP-based MCP servers:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import MCPClient, CodeAgent\n",
"\n",
"with MCPClient({\"url\": \"http://127.0.0.1:8000/mcp\", \"transport\": \"streamable-http\"}) as tools:\n",
" agent = CodeAgent(tools=tools, model=model, add_base_tools=True)\n",
" agent.run(\"Please find a remedy for hangover.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also manually manage the connection lifecycle with the try...finally pattern:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import MCPClient, CodeAgent\n",
"from mcp import StdioServerParameters\n",
"import os\n",
"\n",
"# Initialize server parameters\n",
"server_parameters = StdioServerParameters(\n",
" command=\"uvx\",\n",
" args=[\"--quiet\", \"pubmedmcp@0.1.3\"],\n",
" env={\"UV_PYTHON\": \"3.12\", **os.environ},\n",
")\n",
"\n",
"# Manually manage the connection\n",
"try:\n",
" mcp_client = MCPClient(server_parameters)\n",
" tools = mcp_client.get_tools()\n",
"\n",
" # Use the tools with your agent\n",
" agent = CodeAgent(tools=tools, model=model, add_base_tools=True)\n",
" result = agent.run(\"What are the recent therapeutic approaches for Alzheimer's disease?\")\n",
"\n",
" # Process the result as needed\n",
" print(f\"Agent response: {result}\")\n",
"finally:\n",
" # Always ensure the connection is properly closed\n",
" mcp_client.disconnect()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also connect to multiple MCP servers at once by passing a list of server parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import MCPClient, CodeAgent\n",
"from mcp import StdioServerParameters\n",
"import os\n",
"\n",
"server_params1 = StdioServerParameters(\n",
" command=\"uvx\",\n",
" args=[\"--quiet\", \"pubmedmcp@0.1.3\"],\n",
" env={\"UV_PYTHON\": \"3.12\", **os.environ},\n",
")\n",
"\n",
"server_params2 = {\"url\": \"http://127.0.0.1:8000/sse\"}\n",
"\n",
"with MCPClient([server_params1, server_params2]) as tools:\n",
" agent = CodeAgent(tools=tools, model=model, add_base_tools=True)\n",
" agent.run(\"Please analyze the latest research and suggest remedies for headaches.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> [!WARNING]\n",
"> **Security Warning:** Using MCP servers comes with security risks:\n",
"> - **Trust is essential:** Always verify the source and integrity of any MCP server before connecting to it, especially for production environments. Malicious servers can execute harmful code on your machine.\n",
"> - **Stdio-based MCP servers** will always execute code on your machine (that's their intended functionality).\n",
"> - **Streamable HTTP-based MCP servers** while the remote MCP servers will not be able to execute code on your machine, still proceed with caution."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import a Space as a tool"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can directly import a Gradio Space from the Hub as a tool using the [Tool.from_space()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.Tool.from_space) method!\n",
"\n",
"You only need to provide the id of the Space on the Hub, its name, and a description that will help your agent understand what the tool does. Under the hood, this will use [`gradio-client`](https://pypi.org/project/gradio-client/) library to call the Space.\n",
"\n",
"For instance, let's import the [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) Space from the Hub and use it to generate an image."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image_generation_tool = Tool.from_space(\n",
" \"black-forest-labs/FLUX.1-schnell\",\n",
" name=\"image_generator\",\n",
" description=\"Generate an image from a prompt\"\n",
")\n",
"\n",
"image_generation_tool(\"A sunny beach\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And voilà, here's your image! 🏖️\n",
"\n",
"<img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/sunny_beach.webp\">\n",
"\n",
"Then you can use this tool just like any other tool. For example, let's improve the prompt `a rabbit wearing a space suit` and generate an image of it. This example also shows how you can pass additional arguments to the agent."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import CodeAgent, InferenceClientModel\n",
"\n",
"model = InferenceClientModel(model_id=\"Qwen/Qwen2.5-Coder-32B-Instruct\")\n",
"agent = CodeAgent(tools=[image_generation_tool], model=model)\n",
"\n",
"agent.run(\n",
" \"Improve this prompt, then generate an image of it.\", additional_args={'user_prompt': 'A rabbit wearing a space suit'}\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```text\n",
"=== Agent thoughts:\n",
"improved_prompt could be \"A bright blue space suit wearing rabbit, on the surface of the moon, under a bright orange sunset, with the Earth visible in the background\"\n",
"\n",
"Now that I have improved the prompt, I can use the image generator tool to generate an image based on this prompt.\n",
">>> Agent is executing the code below:\n",
"image = image_generator(prompt=\"A bright blue space suit wearing rabbit, on the surface of the moon, under a bright orange sunset, with the Earth visible in the background\")\n",
"final_answer(image)\n",
"```\n",
"\n",
"<img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit_spacesuit_flux.webp\">\n",
"\n",
"How cool is this? 🤩"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Use LangChain tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We love Langchain and think it has a very compelling suite of tools.\n",
"To import a tool from LangChain, use the `from_langchain()` method.\n",
"\n",
"Here is how you can use it to recreate the intro's search result using a LangChain web search tool.\n",
"This tool will need `pip install langchain google-search-results -q` to work properly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import load_tools\n",
"\n",
"search_tool = Tool.from_langchain(load_tools([\"serpapi\"])[0])\n",
"\n",
"agent = CodeAgent(tools=[search_tool], model=model)\n",
"\n",
"agent.run(\"How many more blocks (also denoted as layers) are in BERT base encoder compared to the encoder from the architecture proposed in Attention is All You Need?\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Manage your agent's toolbox"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can manage an agent's toolbox by adding or replacing a tool in attribute `agent.tools`, since it is a standard dictionary.\n",
"\n",
"Let's add the `model_download_tool` to an existing agent initialized with only the default toolbox."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import InferenceClientModel\n",
"\n",
"model = InferenceClientModel(model_id=\"Qwen/Qwen2.5-Coder-32B-Instruct\")\n",
"\n",
"agent = CodeAgent(tools=[], model=model, add_base_tools=True)\n",
"agent.tools[model_download_tool.name] = model_download_tool"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can leverage the new tool:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"agent.run(\n",
" \"Can you give me the name of the model that has the most downloads in the 'text-to-video' task on the Hugging Face Hub but reverse the letters?\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> [!TIP]\n",
"> Beware of not adding too many tools to an agent: this can overwhelm weaker LLM engines."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Use a collection of tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can leverage tool collections by using [ToolCollection](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.ToolCollection). It supports loading either a collection from the Hub or an MCP server tools."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tool Collection from any MCP server"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Leverage tools from the hundreds of MCP servers available on [glama.ai](https://glama.ai/mcp/servers) or [smithery.ai](https://smithery.ai/).\n",
"\n",
"The MCP servers tools can be loaded with [ToolCollection.from_mcp()](https://huggingface.co/docs/smolagents/main/en/reference/tools#smolagents.ToolCollection.from_mcp).\n",
"\n",
"> [!WARNING]\n",
"> **Security Warning:** The same security warnings mentioned for `MCPClient` apply when using `MCPClient` directly.\n",
"\n",
"\n",
"For stdio-based MCP servers, pass the server parameters as an instance of `mcp.StdioServerParameters`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import ToolCollection, CodeAgent\n",
"from mcp import StdioServerParameters\n",
"\n",
"server_parameters = StdioServerParameters(\n",
" command=\"uvx\",\n",
" args=[\"--quiet\", \"pubmedmcp@0.1.3\"],\n",
" env={\"UV_PYTHON\": \"3.12\", **os.environ},\n",
")\n",
"\n",
"with ToolCollection.from_mcp(server_parameters, trust_remote_code=True) as tool_collection:\n",
" agent = CodeAgent(tools=[*tool_collection.tools], model=model, add_base_tools=True)\n",
" agent.run(\"Please find a remedy for hangover.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For Streamable HTTP-based MCP servers, simply pass a dict with parameters to `mcp.client.streamable_http.streamablehttp_client` and add the key `transport` with the value `\"streamable-http\"`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import ToolCollection, CodeAgent\n",
"\n",
"with ToolCollection.from_mcp({\"url\": \"http://127.0.0.1:8000/mcp\", \"transport\": \"streamable-http\"}, trust_remote_code=True) as tool_collection:\n",
" agent = CodeAgent(tools=[*tool_collection.tools], add_base_tools=True)\n",
" agent.run(\"Please find a remedy for hangover.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tool Collection from a collection in the Hub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can leverage it with the slug of the collection you want to use.\n",
"Then pass them as a list to initialize your agent, and start using them!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from smolagents import ToolCollection, CodeAgent\n",
"\n",
"image_tool_collection = ToolCollection.from_hub(\n",
" collection_slug=\"huggingface-tools/diffusion-tools-6630bb19a942c2306a2cdb6f\",\n",
" token=\"<YOUR_HUGGINGFACEHUB_API_TOKEN>\"\n",
")\n",
"agent = CodeAgent(tools=[*image_tool_collection.tools], model=model, add_base_tools=True)\n",
"\n",
"agent.run(\"Please draw me a picture of rivers and lakes.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To speed up the start, tools are loaded only if called by the agent."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 4
}