notebooks/community/vision/video_warehouse_curl.ipynb (2,046 lines of code) (raw):
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FX2wUzd3gjTc"
},
"outputs": [],
"source": [
"# Copyright 2023 Google LLC\n",
"#\n",
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JAPoU8Sm5E6e"
},
"source": [
"# Video Warehouse Curl Demo\n",
"\n",
"<table align=\"left\">\n",
"\n",
" <td>\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vision/video_warehouse_curl.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vision/video_warehouse_curl.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" </a>\n",
" </td> \n",
" <td>\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/vision/video_warehouse_curl.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" </a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ENEut1m0h3Uo"
},
"source": [
"## Overview\n",
"\n",
"Learn how to issue curl requests to [Video Warehouse](https://cloud.google.com/vision-ai/docs)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gMF4pbEIuZS2"
},
"source": [
"### Objective\n",
"The objective is to demostrate how to issue curl requests to Video Warehouse.\n",
"\n",
"The curl examples are organized by resource types: Corpus, Asset, Data Schema, Annotation, Collection.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yQV4SMLUqchE"
},
"source": [
"### Costs\n",
"\n",
"This tutorial uses billable components of Google Cloud:\n",
"\n",
"Vertex AI Vision ([Pricing](https://cloud.google.com/vision-ai/pricing))\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qeRSrNTGZVBJ"
},
"source": [
"## Before you begin\n",
"\n",
"### Set up your Google Cloud project\n",
"\n",
"**The following steps are required, regardless of your notebook environment.**\n",
"\n",
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
"\n",
"2. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z3i7BDALZqeQ"
},
"source": [
"#### Set your project ID\n",
"\n",
"**If you don't know your project ID**, try the following:\n",
"* Run `gcloud config list`.\n",
"* Run `gcloud projects list`.\n",
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mOkFz5v6Z5mL"
},
"outputs": [],
"source": [
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"\n",
"# Set the project id\n",
"! gcloud config set project {PROJECT_ID}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sBCra4QMA2wR"
},
"source": [
"### Authenticate your Google Cloud account\n",
"\n",
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "74ccc9e52986"
},
"source": [
"**1. Vertex AI Workbench**\n",
"* Do nothing as you are already authenticated."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "de775a3773ba"
},
"source": [
"**2. Local JupyterLab instance, uncomment and run:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "254614fa0c46"
},
"outputs": [],
"source": [
"# ! gcloud auth login"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "iXNewDH2wkV-"
},
"outputs": [],
"source": [
"# ! gcloud auth application-default login"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ef21552ccea8"
},
"source": [
"**3. Colab, uncomment and run:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "603adbbf0532"
},
"outputs": [],
"source": [
"# from google.colab import auth\n",
"# auth.authenticate_user(project_id=PROJECT_ID)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ixfp-_Ne09EK"
},
"source": [
"### Set Up Other Constants"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "M00JvF4Qs-kZ"
},
"outputs": [],
"source": [
"ENDPOINT_WITH_VERSION = (\n",
" \"https://warehouse-visionai.googleapis.com/v1\" # @param {type: \"string\"}\n",
")\n",
"PROJECT_NUMBER_STR = !gcloud projects describe $PROJECT_ID --format=\"value(projectNumber)\"\n",
"PROJECT_NUMBER = int(PROJECT_NUMBER_STR[0])\n",
"LOCATION_ID = \"us-central1\" # @param [\"us-central1\"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4XItfSfLsuwc"
},
"source": [
"# Corpus Management"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8-mJjcyoqPtM"
},
"source": [
"## CreateCorpus"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qYlSW2H7t3Y0"
},
"outputs": [],
"source": [
"# @title Create VOD corpus\n",
"CORPUS_DISPLAY_NAME=\"Test corpus\" # @param {type: \"string\"}\n",
"CORPUS_DESCRIPTION=\"Test corpus description\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" 'display_name': '%s',\n",
" 'description': '%s',\n",
" 'type': 'VIDEO_ON_DEMAND',\n",
" 'search_capability_setting' : {\n",
" 'search_capabilities': {\n",
" 'type':'EMBEDDING_SEARCH'\n",
" }\n",
" }\n",
"}\"\"\"%(CORPUS_DISPLAY_NAME, CORPUS_DESCRIPTION)\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mMlmL0Way4qB"
},
"outputs": [],
"source": [
"# @title Poll create corpus operation\n",
"CREATE_CORPUS_OPERATION_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/warehouseOperations/$CREATE_CORPUS_OPERATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T-RuQskau5Qm"
},
"source": [
"## GetCorpus\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Y6kFFI8ju4ds"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cNTVp34vqVRN"
},
"source": [
"## ListCorpora"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AMmMqK3hpG7-"
},
"outputs": [],
"source": [
"# @title List all corpora under one project.\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0mE1Bb7F1Pe0"
},
"outputs": [],
"source": [
"# @title List all corpora with page size specified.\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "t0DCAEcKmHPd"
},
"outputs": [],
"source": [
"# @title List copora with page token specified.\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6yOES49jnFsY"
},
"outputs": [],
"source": [
"# @title List corpus filter by certain type (type!=, type=).\n",
"TYPE='VIDEO_ON_DEMAND' # @param {type: \"string\"}\n",
"PAGE_SIZE=2\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora?'page_size='{PAGE_SIZE}'&filter=type!='{TYPE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\"\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora?'page_size='{PAGE_SIZE}'&filter=type='{TYPE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cbcb_F4XwQhQ"
},
"source": [
"## UpdateCorpus"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9tiFzcEXexQ3"
},
"outputs": [],
"source": [
"# @title Use update mask * to do a full replacement\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"*\"\n",
"REQUEST=\"\"\"{\n",
" 'display_name': 'test-update',\n",
" 'description': 'update description',\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JbJkA423AtWp"
},
"outputs": [],
"source": [
"# @title Error: TTL is not allowed to be updated for VOD\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"display_name,default_ttl\"\n",
"REQUEST=\"\"\"{\n",
" 'display_name':'updated',\n",
" 'description': 'sample description_ignore',\n",
" 'default_ttl': {'seconds':'1000000'}}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UKzlmC3DyQEJ"
},
"outputs": [],
"source": [
"# @title Error: search_capability_setting is not allowed to be updated for VOD.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"search_capability_setting\"\n",
"REQUEST=\"\"\"{\n",
" 'display_name': 'test-update',\n",
" 'description': 'update description',\n",
" 'search_capability_setting': { 'search_capabilities': { 'type': 'EMBEDDING_SEARCH'} }\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID?'update_mask='{UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nqwA2p7lLU11"
},
"source": [
"## DeleteCorpus\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SaLYXfb_CIvg"
},
"outputs": [],
"source": [
"# @title Please run other sections first which requires a valid Corpus before delete the corpus.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X DELETE"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Hat_wS7v37wi"
},
"source": [
"# Asset Management"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HG0519pd5zz5"
},
"source": [
"## CreateAsset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "code",
"id": "35ejTcHl3_Fy"
},
"outputs": [],
"source": [
"# @title CreateAsset (System generates asset Id)\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1ZNGFTVd5N14"
},
"outputs": [],
"source": [
"# @title CreateAsset (User specified asset Id)\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets?asset_id={ASSET_ID} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5-xaVGpnC77h"
},
"source": [
"## UploadAsset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tU8xaSt6C7qz"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"GCS_URI=\"gs://cloud-samples-data/video/animals.mp4\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" 'asset_source': {\n",
" 'asset_gcs_source': {\n",
" 'gcs_uri': '%s'}}\n",
"}\"\"\"%(GCS_URI)\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID:upload \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "myHppVS4ENnN"
},
"outputs": [],
"source": [
"# @title Poll upload asset operation\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"UPLOAD_ASSET_OPERATION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/operations/$UPLOAD_ASSET_OPERATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8rYNeOpHXJeh"
},
"source": [
"## GenerateRetrievalUrl"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jfWvmsu3XMJP"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID:generateRetrievalUrl \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zr3-3QrSHCSu"
},
"source": [
"## AnalyzeAsset\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H5etICYTHFjf"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID:analyze \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "v3lIFrgjHWsz"
},
"outputs": [],
"source": [
"# @title Poll analyze asset operation\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ANALYZE_ASSET_OPERATION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/operations/$ANALYZE_ASSET_OPERATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eEW8mWyvCog_"
},
"source": [
"## GetAsset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Uru7cisJ5tR9"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QlHRY-f8Hpkd"
},
"source": [
"## ListAsset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ox5LL1DoJTKx"
},
"outputs": [],
"source": [
"# @title List all assets under one corpus.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "L2CLfCYVJTKy"
},
"outputs": [],
"source": [
"# @title List all assets with page size specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "CJi4OekSJTKy"
},
"outputs": [],
"source": [
"# @title List assets with page token specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AO8emOORJTKy"
},
"outputs": [],
"source": [
"# @title List assets filter by has content or not (assets_with_contents = true, assets_with_contents = false).\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets?'page_size='{PAGE_SIZE}'&filter=assets_with_contents=true' \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\"\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets?'page_size='{PAGE_SIZE}'&filter=assets_with_contents=false' \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pYC_vpu1Huks"
},
"source": [
"## UpdateAsset"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0gQ_k2yKLDri"
},
"source": [
"Nothing in VOD asset can be updated. Expected all updates below to fail."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0M74XlX3K815"
},
"outputs": [],
"source": [
"# @title TTL is not allowed to be updated.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"*\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" 'ttl': {'seconds':'1000000'},\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "l7PP1CX3Lt8o"
},
"outputs": [],
"source": [
"# @title asset_gcs_source is output only, not allowed to be updated.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"*\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" 'asset_gcs_source': {\n",
" 'gcs_uri': 'gs://cloud-samples-data/video/cat2.mp4'\n",
" }\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Igyss_kBITkp"
},
"source": [
"## DeleteAsset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qT6xDT3MC4AM"
},
"outputs": [],
"source": [
"# @title Please run other sections first which requires a valid Asset before delete the asset.\n",
"\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X DELETE"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PyX-iSvuM931"
},
"outputs": [],
"source": [
"# @title Poll delete asset operation\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DELETE_ASSET_OPERATION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/operations/$DELETE_ASSET_OPERATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JTvSJjGeDKl0"
},
"source": [
"# Data Schema Management"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DkE_NpDsDsO4"
},
"source": [
"## CreateDataSchema"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ALrAmpnQKCn8"
},
"outputs": [],
"source": [
"# @title CreateDataSchema (primitive data type)\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"test\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_TYPE=\"STRING\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_GRANULARITY=\"GRANULARITY_PARTITION_LEVEL\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_SEARCH_STRATEGY_TYPE=\"SMART_SEARCH\" # @param {type: \"string\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" }\n",
"}\"\"\"%(DATA_SCHEMA_KEY, DATA_SCHEMA_TYPE, DATA_SCHEMA_GRANULARITY,DATA_SCHEMA_SEARCH_STRATEGY_TYPE)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QSw18vwjQJMF"
},
"outputs": [],
"source": [
"# @title CreateDataSchema (customized struct data type)\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"tutorial\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_GRANULARITY=\"GRANULARITY_PARTITION_LEVEL\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_KEY_1=\"text\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_TYPE_1=\"STRING\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_SEARCH_STRATEGY_TYPE_1=\"SMART_SEARCH\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_KEY_2=\"score\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_TYPE_2=\"FLOAT\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_FIELD_SEARCH_STRATEGY_TYPE_2=\"EXACT_SEARCH\" # @param {type: \"string\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'CUSTOMIZED_STRUCT',\n",
" 'granularity':'%s',\n",
" 'customized_struct_config': {\n",
" 'field_schemas': {\n",
" '%s': {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" },\n",
" '%s': {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"%(DATA_SCHEMA_KEY, DATA_SCHEMA_GRANULARITY, DATA_SCHEMA_FIELD_KEY_1, DATA_SCHEMA_FIELD_TYPE_1,DATA_SCHEMA_GRANULARITY,DATA_SCHEMA_FIELD_SEARCH_STRATEGY_TYPE_1, DATA_SCHEMA_FIELD_KEY_2, DATA_SCHEMA_FIELD_TYPE_2,DATA_SCHEMA_GRANULARITY, DATA_SCHEMA_FIELD_SEARCH_STRATEGY_TYPE_2)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WJosAlFibJJC"
},
"source": [
"## GetDataSchema\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_5AQkmZ1bOj9"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"test\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas/$DATA_SCHEMA_KEY \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bTCM1IJqbdmt"
},
"source": [
"## ListDataSchemas"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "sR0FyUPQbi5e"
},
"outputs": [],
"source": [
"# @title List all data schemas under one corpus.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "rwpW_gcybwtN"
},
"outputs": [],
"source": [
"# @title List all data schemas with page size specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mQL_zLECb9pd"
},
"outputs": [],
"source": [
"# @title List data schemas with page token specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WnkRM6Q9VCyx"
},
"source": [
"## UpdateDataSchema"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "K9eQjfeCVKz4"
},
"outputs": [],
"source": [
"# @title Use update mask * to do a full replacement\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"test\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_TYPE=\"STRING\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_GRANULARITY=\"GRANULARITY_PARTITION_LEVEL\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_SEARCH_STRATEGY_TYPE=\"SMART_SEARCH\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"*\"\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" }\n",
"}\"\"\"%(DATA_SCHEMA_KEY, DATA_SCHEMA_TYPE, DATA_SCHEMA_GRANULARITY,DATA_SCHEMA_SEARCH_STRATEGY_TYPE)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas/$DATA_SCHEMA_KEY?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DhnbMgAcZEqs"
},
"outputs": [],
"source": [
"# @title Use update mask to do partial replacement\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"test\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_TYPE=\"FLOAT\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_GRANULARITY=\"GRANULARITY_ASSET_LEVEL\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_SEARCH_STRATEGY_TYPE=\"SMART_SEARCH\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"schema_details.granularity\" # @param {type: \"string\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" }\n",
"}\"\"\"%(DATA_SCHEMA_KEY, DATA_SCHEMA_TYPE, DATA_SCHEMA_GRANULARITY,DATA_SCHEMA_SEARCH_STRATEGY_TYPE)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas/$DATA_SCHEMA_KEY?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "t_Ir_OKWZbNf"
},
"outputs": [],
"source": [
"# @title Error: key is not allowed to be updated\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_ID=\"test\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_KEY=\"verbal\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_TYPE=\"FLOAT\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_GRANULARITY=\"GRANULARITY_PARTITION_LEVEL\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_SEARCH_STRATEGY_TYPE=\"SMART_SEARCH\" # @param {type: \"string\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'%s',\n",
" 'granularity':'%s',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'%s'\n",
" }\n",
" }\n",
"}\"\"\"%(DATA_SCHEMA_KEY, DATA_SCHEMA_TYPE, DATA_SCHEMA_GRANULARITY,DATA_SCHEMA_SEARCH_STRATEGY_TYPE)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas/$DATA_SCHEMA_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N6a9j0NudJwY"
},
"source": [
"## DeleteDataSchema\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Lvy5dCWCdNSh"
},
"outputs": [],
"source": [
"# @title Please run other sections first which requires a valid Data Schema before deleting the data schema.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"DATA_SCHEMA_ID=\"test\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas/$DATA_SCHEMA_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X DELETE"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c_7BVWYbBpcP"
},
"source": [
"# Annotation Management"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-hqqOCA8BwYP"
},
"outputs": [],
"source": [
"# @title CreateAnnotation (System generates asset id with primitive data type on partition level), the corresponding data schema needs to be created in advance with annotation key.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_KEY=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_STRING_VALUE=\"STRING\" # @param {type: \"string\"}\n",
"ANNOTATION_START_OFFSET_SECONDS=1 # @param {type: \"integer\"}\n",
"ANNOTATION_START_OFFSET_NANO_SECONDS=0 # @param {type: \"integer\"}\n",
"ANNOTATION_END_OFFSET_SECONDS=5 # @param {type: \"integer\"}\n",
"ANNOTATION_END_OFFSET_NANO_SSECONDS=0 # @param {type: \"integer\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'user_specified_annotation' : {\n",
" 'key':'%s',\n",
" 'value': {\n",
" 'str_value':'%s'\n",
" },\n",
" 'partition': {\n",
" 'relative_temporal_partition': {\n",
" 'start_offset': {\n",
" 'seconds': %d,\n",
" 'nanos': %d\n",
" },\n",
" 'end_offset': {\n",
" 'seconds': %d,\n",
" 'nanos': %d\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"%(ANNOTATION_KEY, ANNOTATION_STRING_VALUE, ANNOTATION_START_OFFSET_SECONDS,ANNOTATION_START_OFFSET_NANO_SECONDS,ANNOTATION_END_OFFSET_SECONDS,ANNOTATION_END_OFFSET_NANO_SSECONDS)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "bX1wfQ3OZPVa"
},
"outputs": [],
"source": [
"# @title CreateAnnotation (User specified asset id with customized struct on asset level), the corresponding data schema needs to be created in advance with annotation key.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_KEY=\"\" # @param {type: \"string\"}\n",
"\n",
"# Corresponding create data schema request for the annotation, could be skipped if it already exist.\n",
"CREATE_DATA_SCHEMA_REQUEST=\"\"\"{\n",
" 'key': '%s',\n",
" 'schema_details' : {\n",
" 'type':'CUSTOMIZED_STRUCT',\n",
" 'granularity':'GRANULARITY_ASSET_LEVEL',\n",
" 'customized_struct_config': {\n",
" 'field_schemas': {\n",
" 'name': {\n",
" 'type':'STRING',\n",
" 'granularity':'GRANULARITY_ASSET_LEVEL',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'SMART_SEARCH'\n",
" }\n",
" },\n",
" 'score': {\n",
" 'type':'FLOAT',\n",
" 'granularity':'GRANULARITY_ASSET_LEVEL',\n",
" 'search_strategy': {\n",
" 'search_strategy_type':'EXACT_SEARCH'\n",
" }\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"%(ANNOTATION_KEY)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/dataSchemas \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$CREATE_DATA_SCHEMA_REQUEST\"\n",
"\n",
"CREATE_ANNOTATION_REQUEST=\"\"\"{\n",
" 'user_specified_annotation' : {\n",
" 'key':'%s',\n",
" 'value': {\n",
" 'customized_struct_value':{\n",
" 'elements' : {\n",
" 'name': {\n",
" 'str_value':'intro 101'\n",
" },\n",
" 'score': {\n",
" 'float_value':0.98\n",
" }\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"%(ANNOTATION_KEY)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations?annotation_id={ANNOTATION_ID} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST -d \"$CREATE_ANNOTATION_REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eEls217rdkr7"
},
"outputs": [],
"source": [
"# @title GetAnnotation\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations/$ANNOTATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8rETVcKhe4A3"
},
"source": [
"## ListAnnotations"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1s2LRpq_eycx"
},
"outputs": [],
"source": [
"# @title List all annotations under one asset.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VVjdl-PqfN68"
},
"outputs": [],
"source": [
"# @title List all annotations under one asset with page size specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DffNRHQ2fgfL"
},
"outputs": [],
"source": [
"# @title List all annotations under one asset with page size and page token specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "18_EFuUNgwGy"
},
"outputs": [],
"source": [
"# @title List all annotations under one asset filter by key.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_KEY=\"player\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations?'filter=key='$ANNOTATION_KEY \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "V2ZGGVzDhaF1"
},
"outputs": [],
"source": [
"# @title List all annotations under one asset filter by relative temporal partition.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_START_TIME=\"3s\" # @param {type: \"string\"}\n",
"ANNOTATION_END_TIME=\"5s\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations?'filter=partition.relative_temporal_partition.start_offset>'$ANNOTATION_START_TIME'%20AND%20partition.relative_temporal_partition.end_offset<'$ANNOTATION_END_TIME \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JDejvffEmHOY"
},
"source": [
"## UpdateAnnotation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-ePi9Ps_mNvk"
},
"outputs": [],
"source": [
"# @title Use update mask * to do a full replacement\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_KEY=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_STRING_VALUE=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_START_OFFSET_SECONDS=3 # @param {type: \"integer\"}\n",
"ANNOTATION_START_OFFSET_NANO_SECONDS=0 # @param {type: \"integer\"}\n",
"ANNOTATION_END_OFFSET_SECONDS=7 # @param {type: \"integer\"}\n",
"ANNOTATION_END_OFFSET_NANO_SSECONDS=0 # @param {type: \"integer\"}\n",
"UPDATE_MASK=\"*\"\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'user_specified_annotation' : {\n",
" 'key':'%s',\n",
" 'value': {\n",
" 'str_value':'%s'\n",
" },\n",
" 'partition': {\n",
" 'relative_temporal_partition': {\n",
" 'start_offset': {\n",
" 'seconds': %d,\n",
" 'nanos': %d\n",
" },\n",
" 'end_offset': {\n",
" 'seconds': %d,\n",
" 'nanos': %d\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"%(ANNOTATION_KEY, ANNOTATION_STRING_VALUE, ANNOTATION_START_OFFSET_SECONDS,ANNOTATION_START_OFFSET_NANO_SECONDS,ANNOTATION_END_OFFSET_SECONDS,ANNOTATION_END_OFFSET_NANO_SSECONDS)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations/$ANNOTATION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "njhiN2uAq4Pe"
},
"outputs": [],
"source": [
"# @title Use update mask to do partial replacement\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"user_specified_annotation.value\" # @param {type: \"string\"}\n",
"\n",
"REQUEST=\"\"\"{\n",
" 'user_specified_annotation' : {\n",
" 'key':'random',\n",
" 'value': {\n",
" 'customized_struct_value':{\n",
" 'elements' : {\n",
" 'name': {\n",
" 'str_value':'math'\n",
" },\n",
" 'score': {\n",
" 'float_value':0.67\n",
" }\n",
" }\n",
" }\n",
" }\n",
" }\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations/$ANNOTATION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ClHt2X6kuRap"
},
"source": [
"## DeleteAnnotation\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "exi1EB8JuVi1"
},
"outputs": [],
"source": [
"# @title Please run other sections first which requires a valid Annotation before deleting the annotation.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"ANNOTATION_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/assets/$ASSET_ID/annotations/$ANNOTATION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X DELETE"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XKhi-2xdX8BT"
},
"source": [
"# Collection Management"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "n4CaeeLWX8Bc"
},
"source": [
"## CreateCollection"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "s3SBxH2vX8Bc"
},
"outputs": [],
"source": [
"# @title System generates collection Id.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zCP47_xGX8Bc"
},
"outputs": [],
"source": [
"# @title User specified collection Id.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections?collection_id={COLLECTION_ID} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" -X POST"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FlZ0MuxOX8Bd"
},
"source": [
"## UpdateCollection"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tgsEwdoyX8Bd"
},
"outputs": [],
"source": [
"# @title Use update mask * to do a full replacement.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"*\"\n",
"REQUEST=\"\"\"{\n",
" 'display_name': 'test-update',\n",
" 'description': 'update description'\n",
"}\"\"\"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ukXzVP-fX8Bd"
},
"outputs": [],
"source": [
"# @title Update diaplay_name only.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"display_name\"\n",
"REQUEST=\"\"\"{\n",
" 'display_name': 'test-update-2'\n",
"}\"\"\"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SCRMfSOwr8B0"
},
"outputs": [],
"source": [
"# @title Update description only.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"description\"\n",
"REQUEST=\"\"\"{\n",
" 'description': 'update description 2'\n",
"}\"\"\"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ahjnmPd7eNgj"
},
"outputs": [],
"source": [
"# @title Expire_time is output only, not allowed to be updated.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"UPDATE_MASK=\"expire_time\"\n",
"REQUEST=\"\"\"{\n",
" 'expire_time': {'seconds':'1688163855'}\n",
"}\"\"\"\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID?update_mask={UPDATE_MASK} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X PATCH \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zbwhU4GTX8Be"
},
"source": [
"## GetCollection"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "vx2Ykl1FX8Bf"
},
"outputs": [],
"source": [
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5cUMoWxwX8Bf"
},
"source": [
"## ListCollections"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "YcoJHcBFX8Bf"
},
"outputs": [],
"source": [
"# @title List all collections under one corpus.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5pSGmvPXX8Bf"
},
"outputs": [],
"source": [
"# @title List all collections with page size specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RBLlM8yaX8Bf"
},
"outputs": [],
"source": [
"# @title List collections with page token specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"!curl $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XKhGcR1fX8Bg"
},
"source": [
"## AddCollectionItem"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hIukj3aXX8Bg"
},
"outputs": [],
"source": [
"# @title Please run other sections first which requires a valid Asset and Collection before adding a collection item.\n",
"\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" item: {\n",
" 'type': 'ASSET',\n",
" 'item_resource': 'projects/%d/locations/%s/corpora/%s/assets/%s'\n",
" }\n",
"}\"\"\"%(PROJECT_NUMBER, LOCATION_ID, CORPUS_ID, ASSET_ID)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:addCollectionItem \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "X1es8bDRX8Bg"
},
"outputs": [],
"source": [
"# @title Item already added to collection, will succeed.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" item: {\n",
" 'type': 'ASSET',\n",
" 'item_resource': 'projects/%d/locations/%s/corpora/%s/assets/%s'\n",
" }\n",
"}\"\"\"%(PROJECT_NUMBER, LOCATION_ID, CORPUS_ID, ASSET_ID)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:addCollectionItem \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Nvbg9qnrepVp"
},
"source": [
"## ViewCollectionItems"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jvCbUx7LepVy"
},
"outputs": [],
"source": [
"# @title List all items in a collection.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:viewCollectionItems \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mASxIH-uepVz"
},
"outputs": [],
"source": [
"# @title List all items with page size specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:viewCollectionItems?'page_size='{PAGE_SIZE} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kARphKR4jEmm"
},
"outputs": [],
"source": [
"# @title List all items with page token specified.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"PAGE_SIZE=2 # @param {type: \"integer\"}\n",
"PAGE_TOKEN=\"\" # @param {type: \"string\"}\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:viewCollectionItems?'page_size='{PAGE_SIZE}'&page_token='{PAGE_TOKEN} \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GCZW4WxCeq5w"
},
"source": [
"## RemoveCollectionItem"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1p59TBd-eq55"
},
"outputs": [],
"source": [
"# @title Remove an item from a collection\n",
"\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" item: {\n",
" 'type': 'ASSET',\n",
" 'item_resource': 'projects/%d/locations/%s/corpora/%s/assets/%s'\n",
" }\n",
"}\"\"\"%(PROJECT_NUMBER, LOCATION_ID, CORPUS_ID, ASSET_ID)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:removeCollectionItem \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST \\\n",
" -d \"$REQUEST\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9LYEIugYeq56"
},
"outputs": [],
"source": [
"# @title Item already removed from a collection, will fail with a FAILED_PRECONDITION error.\n",
"CORPUS_ID=\"\" # @param {type: \"string\"}\n",
"COLLECTION_ID=\"\" # @param {type: \"string\"}\n",
"ASSET_ID=\"\" # @param {type: \"string\"}\n",
"REQUEST=\"\"\"{\n",
" item: {\n",
" 'type': 'ASSET',\n",
" 'item_resource': 'projects/%d/locations/%s/corpora/%s/assets/%s'\n",
" }\n",
"}\"\"\"%(PROJECT_NUMBER, LOCATION_ID, CORPUS_ID, ASSET_ID)\n",
"\n",
"!curl \\\n",
" $ENDPOINT_WITH_VERSION/projects/$PROJECT_NUMBER/locations/$LOCATION_ID/corpora/$CORPUS_ID/collections/$COLLECTION_ID:removeCollectionItem \\\n",
" -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
" -H \"Content-Type: application/json; charset=utf-8\" \\\n",
" -X POST \\\n",
" -d \"$REQUEST\""
]
}
],
"metadata": {
"colab": {
"name": "video_warehouse_curl.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}