quickstarts/Safety.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2025 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# Gemini API: Safety Quickstart"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Safety.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" height=30/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uOxMUKTxR-_j"
      },
      "source": [
        "The Gemini API has adjustable safety settings. This notebook walks you through how to use them. You'll write a prompt that's blocked, see the reason why, and then adjust the filters to unblock it.\n",
        "\n",
        "Safety is an important topic, and you can learn more with the links at the end of this notebook. Here, you will focus on the code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "9OEoeosRTv-5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/200.0 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.2/200.0 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━\u001b[0m \u001b[32m194.6/200.0 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m200.0/200.0 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "%pip install -q -U \"google-genai>=1.0.0\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3VAUtJubX7MG"
      },
      "source": [
        "## Import the Gemini python SDK\n",
        "\n",
        "Once the kernel is restarted, you can import the Gemini SDK:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gHYFrFPjSGNq"
      },
      "source": [
        "## Set up your API key\n",
        "\n",
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) quickstart for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "ab9ASynfcIZn"
      },
      "outputs": [],
      "source": [
        "from google import genai\n",
        "from google.colab import userdata\n",
        "\n",
        "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
        "client = genai.Client(api_key=GOOGLE_API_KEY)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LZfoK3I3hu6V"
      },
      "source": [
        "## Send your prompt request to Gemini\n",
        "\n",
        "Pick the prompt you want to use to test the safety filters settings. An examples could be `Write a list of 5 very rude things that I might say to the universe after stubbing my toe in the dark` which was previously tested and trigger the `HARM_CATEGORY_HARASSMENT` and `HARM_CATEGORY_DANGEROUS_CONTENT` categories.\n",
        "\n",
        "The result returned by the [Model.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) method is a [genai.protos.GenerateContentResponse](https://ai.google.dev/api/python/google/generativeai/types/GenerateContentResponse)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "2bcfnGEviwTI"
      },
      "outputs": [],
      "source": [
        "MODEL_ID=\"gemini-2.5-flash-lite-preview-06-17\" # @param [\"gemini-2.5-flash-lite-preview-06-17\",\"gemini-2.0-flash\",\"gemini-2.5-flash\",\"gemini-2.5-pro\"] {\"allow-input\":true, isTemplate: true}\n",
        "\n",
        "unsafe_prompt = \"\"\"\n",
        "  I support Martians Soccer Club and I think Jupiterians Football Club\n",
        "  sucks! Write a ironic phrase about them.\n",
        "\"\"\"\n",
        "response = client.models.generate_content(\n",
        "    model = MODEL_ID,\n",
        "    contents = unsafe_prompt\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WR_2A_sxk8sK"
      },
      "source": [
        "This response object gives you safety feedback about the candidate answers Gemini generates to you.\n",
        "\n",
        "For each candidate answer you need to check `response.candidates.finish_reason`.\n",
        "\n",
        "As you can find on the [Gemini API safety filters documentation](https://ai.google.dev/gemini-api/docs/safety-settings#safety-feedback):\n",
        "- if the `candidate.finish_reason` is `FinishReason.STOP` means that your generation request ran successfully\n",
        "- if the `candidate.finish_reason` is `FinishReason.SAFETY` means that your generation request was blocked by safety reasons. It also means that the `response.text` structure will be empty."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "8887de812dc0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "FinishReason.STOP\n"
          ]
        }
      ],
      "source": [
        "print(response.candidates[0].finish_reason)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XBdqPso3kamW"
      },
      "source": [
        "If the `finish_reason` is `FinishReason.SAFETY` you can check which filter caused the block checking the `safety_ratings` list for the candidate answer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "he-OfzBbhACQ"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "None\n"
          ]
        }
      ],
      "source": [
        "from pprint import pprint\n",
        "\n",
        "pprint(response.candidates[0].safety_ratings, indent=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z9-SdzjbxWXT"
      },
      "source": [
        "As the request was blocked by the safety filters, the `response.text` field will be empty (as nothing as generated by the model):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "L1Da4cJ3xej3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Oh, Jupiterians? They're so good at losing, it's almost an art form. You have to admire their dedication to consistently disappointing their fans!\n",
            "\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    print(response.text)\n",
        "except:\n",
        "    print(\"No information generated by the model.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4672af98ac57"
      },
      "source": [
        "## Customizing safety settings\n",
        "\n",
        "Depending on the scenario you are working with, it may be necessary to customize the safety filters behaviors to allow a certain degree of unsafety results.\n",
        "\n",
        "To make this customization you must define a `safety_settings` dictionary as part of your `model.generate_content()` request. In the example below, all the filters are being set to do not block contents.\n",
        "\n",
        "**Important:** To guarantee the Google commitment with the Responsible AI development and its [AI Principles](https://ai.google/responsibility/principles/), for some prompts Gemini will avoid generating the results even if you set all the filters to none."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "338fb9a6af78"
      },
      "outputs": [],
      "source": [
        "from google.genai import types\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model = MODEL_ID,\n",
        "    contents = unsafe_prompt,\n",
        "    config = types.GenerateContentConfig(\n",
        "        safety_settings=[\n",
        "            types.SafetySetting(\n",
        "              category=types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,\n",
        "              threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
        "            ),\n",
        "            types.SafetySetting(\n",
        "              category=types.HarmCategory.HARM_CATEGORY_HARASSMENT,\n",
        "              threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
        "            ),\n",
        "            types.SafetySetting(\n",
        "              category=types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,\n",
        "              threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
        "            ),\n",
        "            types.SafetySetting(\n",
        "              category=types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,\n",
        "              threshold=types.HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,\n",
        "            )\n",
        "        ]\n",
        "    )\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "564K7R8rwWhs"
      },
      "source": [
        "Checking again the `candidate.finish_reason` information, if the request was not too unsafe, it must show now the value as `FinishReason.STOP` which means that the request was successfully processed by Gemini."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "LazB08GBpc1w"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "FinishReason.SAFETY\n"
          ]
        }
      ],
      "source": [
        "print(response.candidates[0].finish_reason)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86c560e0a641"
      },
      "source": [
        "Since the request was successfully generated, you can check the result on the `response.text`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "0c2847c49262"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "None\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    print(response.text)\n",
        "except:\n",
        "    print(\"No information generated by the model.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47298a4eef40"
      },
      "source": [
        "And if you check the safety filters ratings, as you set all filters to be ignored, no filtering category was trigerred:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "028febe8df68"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[ SafetyRating(blocked=None, category=<HarmCategory.HARM_CATEGORY_HATE_SPEECH: 'HARM_CATEGORY_HATE_SPEECH'>, probability=<HarmProbability.NEGLIGIBLE: 'NEGLIGIBLE'>, probability_score=None, severity=None, severity_score=None),\n",
            "  SafetyRating(blocked=None, category=<HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: 'HARM_CATEGORY_DANGEROUS_CONTENT'>, probability=<HarmProbability.NEGLIGIBLE: 'NEGLIGIBLE'>, probability_score=None, severity=None, severity_score=None),\n",
            "  SafetyRating(blocked=True, category=<HarmCategory.HARM_CATEGORY_HARASSMENT: 'HARM_CATEGORY_HARASSMENT'>, probability=<HarmProbability.LOW: 'LOW'>, probability_score=None, severity=None, severity_score=None),\n",
            "  SafetyRating(blocked=None, category=<HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: 'HARM_CATEGORY_SEXUALLY_EXPLICIT'>, probability=<HarmProbability.NEGLIGIBLE: 'NEGLIGIBLE'>, probability_score=None, severity=None, severity_score=None)]\n"
          ]
        }
      ],
      "source": [
        "from pprint import pprint\n",
        "\n",
        "pprint(response.candidates[0].safety_ratings, indent=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n1UdbxVt3ysY"
      },
      "source": [
        "## Learning more\n",
        "\n",
        "Learn more with these articles on [safety guidance](https://ai.google.dev/docs/safety_guidance) and [safety settings](https://ai.google.dev/docs/safety_setting_gemini).\n",
        "\n",
        "## Useful API references\n",
        "\n",
        "There are 4 configurable safety settings for the Gemini API:\n",
        "* `HARM_CATEGORY_DANGEROUS`\n",
        "* `HARM_CATEGORY_HARASSMENT`\n",
        "* `HARM_CATEGORY_SEXUALLY_EXPLICIT`\n",
        "* `HARM_CATEGORY_DANGEROUS`\n",
        "\n",
        "You can refer to the safety settings using either their full name, or the aliases like `DANGEROUS` used in the Python code above.\n",
        "\n",
        "Safety settings can be set in the [genai.GenerativeModel](https://ai.google.dev/api/python/google/generativeai/GenerativeModel) constructor.\n",
        "\n",
        "* They can also be passed on each request to [GenerativeModel.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) or [ChatSession.send_message](https://ai.google.dev/api/python/google/generativeai/ChatSession?hl=en#send_message).\n",
        "\n",
        "- The [genai.protos.GenerateContentResponse](https://ai.google.dev/api/python/google/generativeai/protos/GenerateContentResponse) returns [SafetyRatings](https://ai.google.dev/api/python/google/generativeai/protos/SafetyRating) for the prompt in the [GenerateContentResponse.prompt_feedback](https://ai.google.dev/api/python/google/generativeai/protos/GenerateContentResponse/PromptFeedback), and for each [Candidate](https://ai.google.dev/api/python/google/generativeai/protos/Candidate) in the `safety_ratings` attribute.\n",
        "\n",
        "- A [genai.protos.SafetySetting](https://ai.google.dev/api/python/google/generativeai/protos/SafetySetting)  contains: [genai.protos.HarmCategory](https://ai.google.dev/api/python/google/generativeai/protos/HarmCategory) and a [genai.protos.HarmBlockThreshold](https://ai.google.dev/api/python/google/generativeai/types/HarmBlockThreshold)\n",
        "\n",
        "- A [genai.protos.SafetyRating](https://ai.google.dev/api/python/google/generativeai/protos/SafetyRating) contains a [HarmCategory](https://ai.google.dev/api/python/google/generativeai/protos/HarmCategory) and a [HarmProbability](https://ai.google.dev/api/python/google/generativeai/types/HarmProbability)\n",
        "\n",
        "The [genai.protos.HarmCategory](https://ai.google.dev/api/python/google/generativeai/protos/HarmCategory) enum includes both the categories for PaLM and Gemini models.\n",
        "\n",
        "- When specifying enum values the SDK will accept the enum values themselves, or their integer or string representations.\n",
        "\n",
        "- The SDK will also accept abbreviated string representations: `[\"HARM_CATEGORY_DANGEROUS_CONTENT\", \"DANGEROUS_CONTENT\", \"DANGEROUS\"]` are all valid. Strings are case insensitive."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Safety.ipynb",
      "toc_visible": true
    },
    "google": {
      "image_path": "/static/site-assets/images/docs/logo-python.svg",
      "keywords": [
        "examples",
        "gemini",
        "beginner",
        "googleai",
        "quickstart",
        "python",
        "text",
        "chat",
        "vision",
        "embed"
      ]
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}