gemini/function-calling/intro_function_calling.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ijGzTHJJUCPY"
      },
      "outputs": [],
      "source": [
        "# Copyright 2024 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VEqbX8OhE8y9"
      },
      "source": [
        "# Intro to Function Calling with the Gemini API & Python SDK\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Ffunction-calling%2Fintro_function_calling.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>      \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/function-calling/intro_function_calling.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://goo.gle/4jeQxBO\">\n",
        "      <img width=\"32px\" src=\"https://cdn.qwiklabs.com/assets/gcp_cloud-e3a77215f0b8bfa9b3f611c0d2208c7e8708ed31.svg\" alt=\"Google Cloud logo\"><br> Open in  Cloud Skills Boost\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/function-calling/intro_function_calling.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>            \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "84e7e432e6ff"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Kristopher Overholt](https://github.com/koverholt) [Holt Skinner](https://github.com/holtskinner) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CkHPv2myT2cx"
      },
      "source": [
        "## Overview\n",
        "\n",
        "**YouTube Video: AI + your code: Function Calling**\n",
        "\n",
        "<a href=\"https://www.youtube.com/watch?v=NbAGbXr4DME&list=PLIivdWyY5sqLvGdVLJZh2EMax97_T-OIB\" target=\"_blank\">\n",
        "  <img src=\"https://img.youtube.com/vi/NbAGbXr4DME/maxresdefault.jpg\" alt=\"AI + your code: Function Calling\" width=\"500\">\n",
        "</a>\n",
        "\n",
        "### Gemini\n",
        "\n",
        "Gemini is a family of generative AI models developed by Google DeepMind that is designed for multimodal use cases.\n",
        "\n",
        "### Calling functions from Gemini\n",
        "\n",
        "[Function Calling](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling) in Gemini lets developers create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.\n",
        "\n",
        "### Why function calling?\n",
        "\n",
        "Imagine asking someone to write down important information without giving them a form or any guidelines on the structure. You might get a beautifully crafted paragraph, but extracting specific details like names, dates, or numbers would be tedious! Similarly, trying to get consistent structured data from a generative text model without function calling can be frustrating. You're stuck explicitly prompting for things like JSON output, often with inconsistent and frustrating results.\n",
        "\n",
        "This is where Gemini Function Calling comes in. Instead of hoping for the best in a freeform text response from a generative model, you can define clear functions with specific parameters and data types. These function declarations act as structured guidelines, guiding the Gemini model to structure its output in a predictable and usable way. No more parsing text responses for important information!\n",
        "\n",
        "Think of it like teaching Gemini to speak the language of your applications. Need to retrieve information from a database? Define a `search_db` function with parameters for search terms. Want to integrate with a weather API? Create a `get_weather` function that takes a location as input. Function calling bridges the gap between human language and the structured data needed to interact with external systems."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DrkcqHrrwMAo"
      },
      "source": [
        "### Objectives\n",
        "\n",
        "In this tutorial, you will learn how to use the Gemini API in Vertex AI with the Vertex AI SDK for Python to make function calls via the Gemini 2.0 Flash (`gemini-2.0-flash`) model.\n",
        "\n",
        "You will complete the following tasks:\n",
        "\n",
        "- Install the Google Gen AI SDK for Python\n",
        "- Use the Gemini API in Vertex AI to interact with the Gemini model:\n",
        "- Use Function Calling in a chat session to answer user's questions about products in the Google Store\n",
        "- Use Function Calling to geocode addresses with a maps API\n",
        "- Use Function Calling for entity extraction on raw logging data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C9nEPojogw-g"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "- Vertex AI\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r11Gu7qNgx1p"
      },
      "source": [
        "## Getting Started\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "No17Cw5hgx12"
      },
      "source": [
        "### Install Google Gen AI SDK\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 83,
      "metadata": {
        "id": "tFy3H3aPgx12"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --quiet google-genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment. This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DF4l8DTdWgPY"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "Nqwi-5ufWp_B"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
        "\n",
        "from google import genai\n",
        "\n",
        "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92e02c3e0375"
      },
      "source": [
        "## Code Examples"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5671450907ec"
      },
      "source": [
        "### Choose a model\n",
        "\n",
        "For more information about all AI models and APIs on Vertex AI, see [Google Models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models) and [Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "41e499d90618"
      },
      "outputs": [],
      "source": [
        "MODEL_ID = \"gemini-2.0-flash-001\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jXHfaVS66_01"
      },
      "source": [
        "### Import libraries\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "lslYAvw37JGQ"
      },
      "outputs": [],
      "source": [
        "from IPython.display import Markdown, display\n",
        "from google.genai.types import FunctionDeclaration, GenerateContentConfig, Part, Tool\n",
        "import requests"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "28f36bd968b4"
      },
      "source": [
        "### Chat example: Using Function Calling in a chat session to answer user's questions about the Google Store"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2d28287bde87"
      },
      "source": [
        "In this example, you'll use Function Calling along with the chat modality in the Gemini model to help customers get information about products in the Google Store.\n",
        "\n",
        "You'll start by defining three functions: one to get product information, another to get the location of the closest stores, and one more to place an order:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "3d4ed7ccc094"
      },
      "outputs": [],
      "source": [
        "get_product_info = FunctionDeclaration(\n",
        "    name=\"get_product_info\",\n",
        "    description=\"Get the stock amount and identifier for a given product\",\n",
        "    parameters={\n",
        "        \"type\": \"OBJECT\",\n",
        "        \"properties\": {\n",
        "            \"product_name\": {\"type\": \"STRING\", \"description\": \"Product name\"}\n",
        "        },\n",
        "    },\n",
        ")\n",
        "\n",
        "get_store_location = FunctionDeclaration(\n",
        "    name=\"get_store_location\",\n",
        "    description=\"Get the location of the closest store\",\n",
        "    parameters={\n",
        "        \"type\": \"OBJECT\",\n",
        "        \"properties\": {\"location\": {\"type\": \"STRING\", \"description\": \"Location\"}},\n",
        "    },\n",
        ")\n",
        "\n",
        "place_order = FunctionDeclaration(\n",
        "    name=\"place_order\",\n",
        "    description=\"Place an order\",\n",
        "    parameters={\n",
        "        \"type\": \"OBJECT\",\n",
        "        \"properties\": {\n",
        "            \"product\": {\"type\": \"STRING\", \"description\": \"Product name\"},\n",
        "            \"address\": {\"type\": \"STRING\", \"description\": \"Shipping address\"},\n",
        "        },\n",
        "    },\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e7d7319febd8"
      },
      "source": [
        "Note that function parameters are specified as a Python dictionary in accordance with the [OpenAPI JSON schema format](https://spec.openapis.org/oas/v3.0.3#schemawr).\n",
        "\n",
        "Define a tool that allows the Gemini model to select from the set of 3 functions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "4b2d1900730d"
      },
      "outputs": [],
      "source": [
        "retail_tool = Tool(\n",
        "    function_declarations=[\n",
        "        get_product_info,\n",
        "        get_store_location,\n",
        "        place_order,\n",
        "    ],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2b3781f6fd83"
      },
      "source": [
        "Now you can initialize the Gemini model with Function Calling in a multi-turn chat session.\n",
        "\n",
        "You can specify the `tools` kwarg when initializing the chat session to avoid having to send it with every subsequent request:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "ef8c2d811321"
      },
      "outputs": [],
      "source": [
        "chat = client.chats.create(\n",
        "    model=MODEL_ID,\n",
        "    config=GenerateContentConfig(\n",
        "        temperature=0,\n",
        "        tools=[retail_tool],\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adc8022b2461"
      },
      "source": [
        "**Note:** The `temperature` parameter controls the degree of randomness in this generation. Lower temperatures are good for functions that require deterministic parameter values, while higher temperatures are good for functions with parameters that accept more diverse or creative parameter values. A temperature of `0` is deterministic. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible.\n",
        "\n",
        "We're ready to chat! Let's start the conversation by asking if a certain product is in stock:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "9556d1ebcc1f"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Do you have the Pixel 9 in stock?\n",
        "\"\"\"\n",
        "\n",
        "response = chat.send_message(prompt)\n",
        "response.function_calls[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3111780745fc"
      },
      "source": [
        "The response from the Gemini API consists of a structured data object that contains the name and parameters of the function that Gemini selected out of the available functions.\n",
        "\n",
        "Since this notebook focuses on the ability to extract function parameters and generate function calls, you'll use mock data to feed synthetic responses back to the Gemini model rather than sending a request to an API server (not to worry, we'll make an actual API call in a later example!):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "0c3f7b5474da"
      },
      "outputs": [],
      "source": [
        "# Here you can use your preferred method to make an API request and get a response.\n",
        "# In this example, we'll use synthetic data to simulate a payload from an external API response.\n",
        "\n",
        "api_response = {\"sku\": \"GA04834-US\", \"in_stock\": \"yes\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d86f58489be"
      },
      "source": [
        "In reality, you would execute function calls against an external system or database using your desired client library or REST API.\n",
        "\n",
        "Now, you can pass the response from the (mock) API request and generate a response for the end user:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "5bbc8135093d"
      },
      "outputs": [],
      "source": [
        "response = chat.send_message(\n",
        "    Part.from_function_response(\n",
        "        name=\"get_product_info\",\n",
        "        response={\n",
        "            \"content\": api_response,\n",
        "        },\n",
        "    ),\n",
        ")\n",
        "display(Markdown(response.text))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "186d7afafee9"
      },
      "source": [
        "Next, the user might ask where they can buy a different phone from a nearby store:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "0258f7777226"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "What about the Pixel 9 Pro XL? Is there a store in\n",
        "Mountain View, CA that I can visit to try one out?\n",
        "\"\"\"\n",
        "\n",
        "response = chat.send_message(prompt)\n",
        "response.function_calls"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "da19e8e5292c"
      },
      "source": [
        "Again, you get a response with structured data, but notice that there are two function calls instead of one!\n",
        "\n",
        "The Gemini model identified that it needs both the `get_product_info` and `get_store_location` functions.\n",
        "Look closely at the prompt that you used in this conversation turn a few cells up, and you'll notice that the user asked about a product -and- the location of a store.\n",
        "\n",
        "In cases like this when two or more functions are defined (or when the model predicts multiple function calls to the same function), the Gemini model might sometimes return back-to-back or parallel function call responses within a single conversation turn.\n",
        "\n",
        "This is expected behavior since the Gemini model predicts which functions it should call at runtime, what order it should call dependent functions in, and which function calls can be parallelized, so that the model can gather enough information to generate a natural language response.\n",
        "\n",
        "Not to worry! You can repeat the same steps as before and build synthetic payloads that would come from an external APIs:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "fba8fb03a8f9"
      },
      "outputs": [],
      "source": [
        "# Here you can use your preferred method to make an API request and get a response.\n",
        "# In this example, we'll use synthetic data to simulate a payload from an external API response.\n",
        "\n",
        "product_info_api_response = {\"sku\": \"GA08475-US\", \"in_stock\": \"yes\"}\n",
        "store_location_api_response = {\n",
        "    \"store\": \"2000 N Shoreline Blvd, Mountain View, CA 94043, US\"\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adc1530ec2b1"
      },
      "source": [
        "Again, you can pass the responses from the (mock) API requests back to the Gemini model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "3d8728b830d0"
      },
      "outputs": [],
      "source": [
        "response = chat.send_message(\n",
        "    [\n",
        "        Part.from_function_response(\n",
        "            name=\"get_product_info\",\n",
        "            response={\n",
        "                \"content\": product_info_api_response,\n",
        "            },\n",
        "        ),\n",
        "        Part.from_function_response(\n",
        "            name=\"get_store_location\",\n",
        "            response={\n",
        "                \"content\": store_location_api_response,\n",
        "            },\n",
        "        ),\n",
        "    ]\n",
        ")\n",
        "display(Markdown(response.text))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "02f7d52fbe71"
      },
      "source": [
        "Nice work!\n",
        "\n",
        "Within a single conversation turn, the Gemini model requested 2 function calls in a row before returning a natural language summary. In reality, you might follow this pattern if you need to make an API call to an inventory management system, and another call to a store location database, customer management system, or document repository.\n",
        "\n",
        "Finally, the user might ask to order a phone and have it shipped to their address:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "b430f3ea4f9a"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "I'd like to order a Pixel 9 Pro XL and have it shipped to 1155 Borregas Ave, Sunnyvale, CA 94089.\n",
        "\"\"\"\n",
        "\n",
        "response = chat.send_message(prompt)\n",
        "response.function_calls"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6b0c9fc9d581"
      },
      "source": [
        "Perfect! The Gemini model extracted the user's selected product and their address. Now you can call an API to place the order:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "55883a7238cf"
      },
      "outputs": [],
      "source": [
        "# This is where you would make an API request to return the status of their order.\n",
        "# Use synthetic data to simulate a response payload from an external API.\n",
        "\n",
        "order_api_response = {\n",
        "    \"payment_status\": \"paid\",\n",
        "    \"order_number\": 12345,\n",
        "    \"est_arrival\": \"2 days\",\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "51376798e2d6"
      },
      "source": [
        "And send the payload from the external API call so that the Gemini API returns a natural language summary to the end user."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "74f6d8722928"
      },
      "outputs": [],
      "source": [
        "response = chat.send_message(\n",
        "    Part.from_function_response(\n",
        "        name=\"place_order\",\n",
        "        response={\n",
        "            \"content\": order_api_response,\n",
        "        },\n",
        "    ),\n",
        ")\n",
        "display(Markdown(response.text))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9df66c601c36"
      },
      "source": [
        "And you're done!\n",
        "\n",
        "You were able to have a multi-turn conversation with the Gemini model using function calls, handling payloads, and generating natural language summaries that incorporated the information from the external systems."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46b6be36bf79"
      },
      "source": [
        "### Address example: Using Automatic Function Calling to geocode addresses with a maps API"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7845554ca0a5"
      },
      "source": [
        "In this example, you'll define a function that takes multiple parameters as inputs. Then you'll use automatic function calling in the Gemini API to  make a live API call to convert an address to latitude and longitude coordinates.\n",
        "\n",
        "Start by writing a Python function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "cMLEbK60ZuZ6"
      },
      "outputs": [],
      "source": [
        "def get_location(\n",
        "    amenity: str | None = None,\n",
        "    street: str | None = None,\n",
        "    city: str | None = None,\n",
        "    county: str | None = None,\n",
        "    state: str | None = None,\n",
        "    country: str | None = None,\n",
        "    postalcode: str | None = None,\n",
        ") -> list[dict]:\n",
        "    \"\"\"\n",
        "    Get latitude and longitude for a given location.\n",
        "\n",
        "    Args:\n",
        "        amenity (str | None): Amenity or Point of interest.\n",
        "        street (str | None): Street name.\n",
        "        city (str | None): City name.\n",
        "        county (str | None): County name.\n",
        "        state (str | None): State name.\n",
        "        country (str | None): Country name.\n",
        "        postalcode (str | None): Postal code.\n",
        "\n",
        "    Returns:\n",
        "        list[dict]: A list of dictionaries with the latitude and longitude of the given location.\n",
        "                    Returns an empty list if the location cannot be determined.\n",
        "    \"\"\"\n",
        "    base_url = \"https://nominatim.openstreetmap.org/search\"\n",
        "    params = {\n",
        "        \"amenity\": amenity,\n",
        "        \"street\": street,\n",
        "        \"city\": city,\n",
        "        \"county\": county,\n",
        "        \"state\": state,\n",
        "        \"country\": country,\n",
        "        \"postalcode\": postalcode,\n",
        "        \"format\": \"json\",\n",
        "    }\n",
        "    # Filter out None values from parameters\n",
        "    params = {k: v for k, v in params.items() if v is not None}\n",
        "\n",
        "    try:\n",
        "        response = requests.get(base_url, params=params, headers={\"User-Agent\": \"none\"})\n",
        "        response.raise_for_status()\n",
        "        return response.json()\n",
        "    except requests.RequestException as e:\n",
        "        print(f\"Error fetching location data: {e}\")\n",
        "        return []"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2dd17419f473"
      },
      "source": [
        "In this example, you're asking the Gemini model to extract components of the address into specific fields within a structured data object. These fields are then passed to the function you defined and the result is returned to Gemini to make a natural language response.\n",
        "\n",
        "Send a prompt that includes an address, such as:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "id": "715c7a7437e9"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "I want to get the coordinates for the following address:\n",
        "1600 Amphitheatre Pkwy, Mountain View, CA 94043\n",
        "\"\"\"\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=prompt,\n",
        "    config=GenerateContentConfig(tools=[get_location], temperature=0),\n",
        ")\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ab8b57e204a6"
      },
      "source": [
        "Great work! You were able to define a function that the Gemini model used to extract the relevant parameters from the prompt. Then you made a live API call to obtain the coordinates of the specified location.\n",
        "\n",
        "Here we used the [OpenStreetMap Nominatim API](https://nominatim.openstreetmap.org/ui/search.html) to geocode an address to keep the number of steps in this tutorial to a reasonable number. If you're working with large amounts of address or geolocation data, you can also use the [Google Maps Geocoding API](https://developers.google.com/maps/documentation/geocoding), or any mapping service with an API!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47d9ae0b4b79"
      },
      "source": [
        "## Conclusions\n",
        "\n",
        "You have explored the function calling feature through the Google Gen AI Python SDK.\n",
        "\n",
        "The next step is to enhance your skills by exploring this [documentation page](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "intro_function_calling.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}