gemini/getting-started/intro_gemini_chat.ipynb (648 lines of code) (raw):
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ijGzTHJJUCPY"
},
"outputs": [],
"source": [
"# Copyright 2024 Google LLC\n",
"#\n",
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VEqbX8OhE8y9"
},
"source": [
"# Getting Started with Chat with Gemini\n",
"\n",
"<table align=\"left\">\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_chat.ipynb\">\n",
" <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
" </a>\n",
" </td> \n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_chat.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
"</table>\n",
"\n",
"<div style=\"clear: both;\"></div>\n",
"\n",
"<b>Share to:</b>\n",
"\n",
"<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
"</a>\n",
"\n",
"<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_chat.ipynb\" target=\"_blank\">\n",
" <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
"</a> \n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "f95c904716cd"
},
"source": [
"| Authors |\n",
"| --- |\n",
"| [Eric Dong](https://github.com/gericdong), [Holt Skinner](https://github.com/holtskinner) |"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CkHPv2myT2cx"
},
"source": [
"## Overview\n",
"\n",
"**YouTube Video: Introduction to Gemini on Vertex AI**\n",
"\n",
"<a href=\"https://www.youtube.com/watch?v=YfiLUpNejpE&list=PLIivdWyY5sqJio2yeg1dlfILOUO2FoFRx\" target=\"_blank\">\n",
" <img src=\"https://img.youtube.com/vi/YfiLUpNejpE/maxresdefault.jpg\" alt=\"Introduction to Gemini on Vertex AI\" width=\"500\">\n",
"</a>\n",
"\n",
"This notebook demonstrates how to send chat prompts to the Gemini model. Gemini supports prompts with multimodal input, including natural language tasks, multi-turn text, images, video, audio, and code generation. It can output text and code.\n",
"\n",
"Learn more about [Sending chat prompt requests (Gemini)](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/send-chat-prompts-gemini)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DrkcqHrrwMAo"
},
"source": [
"### Objectives\n",
"\n",
"In this tutorial, you learn how to send chat prompts to the Gemini model.\n",
"\n",
"You will complete the following tasks:\n",
"\n",
"- Sending chat prompts using Google Gen AI SDK for Python\n",
"- Sending chat prompts using LangChain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "C9nEPojogw-g"
},
"source": [
"### Costs\n",
"This tutorial uses billable components of Google Cloud:\n",
"\n",
"- Vertex AI\n",
"\n",
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "r11Gu7qNgx1p"
},
"source": [
"## Getting Started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "No17Cw5hgx12"
},
"source": [
"### Install libraries\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tFy3H3aPgx12"
},
"outputs": [],
"source": [
"%pip install --upgrade --quiet google-genai langchain-google-vertexai langchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dmWOrTJ3gx13"
},
"source": [
"### Authenticate your notebook environment (Colab only)\n",
"\n",
"If you are running this notebook on Google Colab, run the cell below to authenticate your environment.\n",
"\n",
"This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NyKGtVQjgx13"
},
"outputs": [],
"source": [
"import sys\n",
"\n",
"# Additional authentication is required for Google Colab\n",
"if \"google.colab\" in sys.modules:\n",
" # Authenticate user to Google Cloud\n",
" from google.colab import auth\n",
"\n",
" auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DF4l8DTdWgPY"
},
"source": [
"### Define Google Cloud project information and initialize Vertex AI\n",
"\n",
"To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
"\n",
"Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Nqwi-5ufWp_B"
},
"outputs": [],
"source": [
"# Use the environment variable if the user doesn't provide Project ID.\n",
"import os\n",
"\n",
"from google import genai\n",
"\n",
"PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
"\n",
"LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
"\n",
"client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jXHfaVS66_01"
},
"source": [
"### Import libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lslYAvw37JGQ"
},
"outputs": [],
"source": [
"from IPython.display import Markdown, display\n",
"from google.genai.types import GenerateContentConfig, ModelContent, UserContent\n",
"from langchain.chains import ConversationChain\n",
"from langchain.memory import ConversationBufferMemory\n",
"from langchain.prompts import (\n",
" ChatPromptTemplate,\n",
" HumanMessagePromptTemplate,\n",
" MessagesPlaceholder,\n",
" SystemMessagePromptTemplate,\n",
")\n",
"from langchain_core.messages import HumanMessage, SystemMessage\n",
"from langchain_google_vertexai import ChatVertexAI, HarmBlockThreshold, HarmCategory"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4437b7608c8e"
},
"source": [
"## Sending chat prompts using Gen AI SDK for Python\n",
"\n",
"### Load the Gemini model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2998506fe6d1"
},
"outputs": [],
"source": [
"MODEL_ID = \"gemini-2.0-flash\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wl2AZceWjXoy"
},
"source": [
"### Start a chat session\n",
"\n",
"You start a stateful chat session and then send chat prompts with configuration parameters including generation configurations and safety settings."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_vLprtHAjNOO"
},
"outputs": [],
"source": [
"chat = client.chats.create(\n",
" model=MODEL_ID,\n",
" config=GenerateContentConfig(\n",
" system_instruction=\"You are an astronomer, knowledgeable about the solar system..\"\n",
" ),\n",
")\n",
"\n",
"response = chat.send_message(\n",
" \"\"\"How many moons does Mars have? Tell me some fun facts about them.\"\"\"\n",
")\n",
"\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "V2axT2nKuVzP"
},
"source": [
"You can check out the metadata of the response including the `safety_ratings` and `usage_metadata`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ih0v9B1vspiF"
},
"outputs": [],
"source": [
"print(response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZuHvZevgwdj5"
},
"source": [
"You can retrieve the history of the chat session."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FYuqKyyktFq7"
},
"outputs": [],
"source": [
"print(chat.get_history())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kiS2CywXxU2y"
},
"source": [
"### Code chat\n",
"\n",
"Gemini also supports code generation from a text prompt."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1lWiPGQ-cDqC"
},
"outputs": [],
"source": [
"code_chat = client.chats.create(\n",
" model=MODEL_ID,\n",
" config=GenerateContentConfig(\n",
" system_instruction=\"You are an expert software engineer, proficient in Python.\"\n",
" ),\n",
")\n",
"\n",
"\n",
"response = code_chat.send_message(\n",
" \"Write a function that checks if a year is a leap year\"\n",
")\n",
"\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kI781aqpy-lH"
},
"source": [
"You can generate unit tests to test the function in this multi-turn chat."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TGHToON4xyOV"
},
"outputs": [],
"source": [
"response = code_chat.send_message(\"Write a unit test of the generated function\")\n",
"\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jeZNScL7Ci2A"
},
"source": [
"### Add chat history\n",
"\n",
"You can add chat history to a chat by adding messages from role `user` and `model` alternately. System messages can be set in the first part for the first message."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JzqUThJv77G9"
},
"outputs": [],
"source": [
"chat2 = client.chats.create(\n",
" model=MODEL_ID,\n",
" history=[\n",
" UserContent(\n",
" \"\"\"My name is Ned. You are my personal assistant. My favorite movies are Lord of the Rings and Hobbit.\n",
" Who do you work for?\n",
" \"\"\"\n",
" ),\n",
" ModelContent(\"I work for Ned.\"),\n",
" UserContent(\"What do I like?\"),\n",
" ModelContent(\"Ned likes watching movies.\"),\n",
" ],\n",
")\n",
"\n",
"response = chat2.send_message(\"Are my favorite movies based on a book series?\")\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "G64BrDoxC-K3"
},
"outputs": [],
"source": [
"response = chat2.send_message(\"When were these books published?\")\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c4e8deb52116"
},
"source": [
"### Multimodal "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WFlnwb0u54iN"
},
"source": [
"## Sending chat prompts using LangChain\n",
"\n",
"The Gemini API in Vertex AI is integrated with the LangChain Python SDK, making it convenient to build applications on top of Gemini models."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ga2YnFYPXZgh"
},
"source": [
"### Start a chat session\n",
"\n",
"You can start a chat by sending chat prompts to the Gemini 2.0 model directly. Gemini 2.0 doesn't support `SystemMessage` at the moment, but `SystemMessage` can be added to the first human message by setting the `convert_system_message_to_human` to `True`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "s41n0UEwGVZV"
},
"outputs": [],
"source": [
"system_message = \"You are a helpful assistant who translates English to French.\"\n",
"human_message = \"Translate this sentence from English to French. I love programming.\"\n",
"\n",
"messages = [SystemMessage(content=system_message), HumanMessage(content=human_message)]\n",
"\n",
"chat = ChatVertexAI(\n",
" model_name=MODEL_ID,\n",
" convert_system_message_to_human=True,\n",
" safety_settings={\n",
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
" },\n",
")\n",
"\n",
"result = chat.generate([messages])\n",
"print(result.generations[0][0].text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4VprmGzu4FPO"
},
"source": [
"You can check out the metadata of the generated content."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hBLFXgzsIJhw"
},
"outputs": [],
"source": [
"print(result.generations[0][0].generation_info)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RgYHzIUP4PMo"
},
"source": [
"### Use a chat chain with chat prompt template"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9vZr5vyrV-eI"
},
"outputs": [],
"source": [
"system_message = \"You are a helpful assistant who translates English to French.\"\n",
"human_message = \"Translate this sentence from English to French. I love programming.\"\n",
"\n",
"messages = [SystemMessage(content=system_message), HumanMessage(content=human_message)]\n",
"prompt = ChatPromptTemplate.from_messages(messages)\n",
"\n",
"chat = ChatVertexAI(\n",
" model_name=MODEL_ID,\n",
" convert_system_message_to_human=True,\n",
" safety_settings={\n",
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
" },\n",
")\n",
"\n",
"chain = prompt | chat\n",
"chain.invoke({})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lB50i-JG9RQp"
},
"source": [
"### Use a conversation chain\n",
"\n",
"You also can wrap up a chat in `ConversationChain`, which has built-in memory for remembering past user inputs and model outputs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UFxQ6AN_5vDQ"
},
"outputs": [],
"source": [
"model = ChatVertexAI(\n",
" model_name=MODEL_ID,\n",
" convert_system_message_to_human=True,\n",
" safety_settings={\n",
" HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE\n",
" },\n",
")\n",
"\n",
"prompt = ChatPromptTemplate(\n",
" messages=[\n",
" SystemMessagePromptTemplate.from_template(\n",
" \"You are a helpful assistant who is good at language translation.\"\n",
" ),\n",
" MessagesPlaceholder(variable_name=\"history\"),\n",
" HumanMessagePromptTemplate.from_template(\"{input}\"),\n",
" ]\n",
")\n",
"\n",
"memory = ConversationBufferMemory(memory_key=\"history\", return_messages=True)\n",
"conversation = ConversationChain(llm=model, prompt=prompt, verbose=True, memory=memory)\n",
"\n",
"conversation.invoke(\n",
" input=\"Translate this sentence from English to French. I love programming.\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AJMP2JrMDmS-"
},
"outputs": [],
"source": [
"conversation.invoke(\"Translate it to Spanish\")"
]
}
],
"metadata": {
"colab": {
"name": "intro_gemini_chat.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}