pai-python-sdk/modelscope/modelscope_model_deploy/modelscope_model_deploy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "keep-output"
    ]
   },
   "source": [
    "# 部署ModelScope模型\n",
    "\n",
    "[ModelScope](https://www.modelscope.cn/)是一个开源的模型社区，提供了丰富的自然语言处理、计算机视觉、多模态等领域开源模型，并提供了[ModelScope library](https://github.com/modelscope/modelscope)，支持开发者可以方便得获取模型，使用模型进行推理。\n",
    "\n",
    "PAI支持开发者将ModelScope上的模型，简单快捷得部署为在线推理服务，本文将介绍使用PAI Python SDK完成ModelScope模型的部署。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 费用说明\n",
    "\n",
    "本示例将会使用以下云产品，并产生相应的费用账单：\n",
    "\n",
    "- PAI-EAS：部署推理服务，详细计费说明请参考[PAI-EAS计费说明](https://help.aliyun.com/zh/pai/product-overview/billing-of-eas)\n",
    "\n",
    "\n",
    "> 通过参与云产品免费试用，使用**指定资源机型**提交训练作业或是部署推理服务，可以免费试用PAI产品，具体请参考[PAI免费试用](https://help.aliyun.com/zh/pai/product-overview/free-quota-for-new-users)。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 安装和配置SDK\n",
    "\n",
    "我们需要首先安装PAI Python SDK以运行本示例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!python -m pip install --upgrade pai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "SDK需要配置访问阿里云服务需要的AccessKey，以及当前使用的工作空间和OSS Bucket。在PAI SDK安装之后，通过在 **命令行终端** 中执行以下命令，按照引导配置密钥、工作空间等信息。\n",
    "\n",
    "\n",
    "```shell\n",
    "\n",
    "# 以下命令，请在 命令行终端 中执行.\n",
    "\n",
    "python -m pai.toolkit.config\n",
    "\n",
    "```\n",
    "\n",
    "我们可以通过以下代码验证配置是否已生效。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pai\n",
    "from pai.session import get_default_session\n",
    "\n",
    "print(pai.__version__)\n",
    "\n",
    "sess = get_default_session()\n",
    "\n",
    "# 获取配置的工作空间信息\n",
    "assert sess.workspace_name is not None\n",
    "print(sess.workspace_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 部署ModelScope模型\n",
    "\n",
    "当前示例，我们将使用ModelScope上的[\"CSANMT连续语义增强机器翻译-英中-通用领域-large\"](https://modelscope.cn/models/damo/nlp_csanmt_translation_en2zh/summary)模型，他支持英文到中文的翻译任务。\n",
    "\n",
    "通过ModelScope的模型详情页，我们可以获取部署模型所需要的信息，包括**模型ID**，**模型版本**，以及**任务类型**，然后通过 `pai.modelscope.ModelScopeModel` 类，创建一个ModelScope模型对象，完成模型部署。\n",
    "\n",
    "![](../../images/modelscope-model.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.modelscope.model import ModelScopeModel\n",
    "\n",
    "# 配置待部署的模型信息\n",
    "m = ModelScopeModel(\n",
    "    command=\"python app.py\",  # 默认的ModelScope模型推理服务启动命令\n",
    "    modelscope_version=\"latest\",  # ModelScope library的版本号，latest表示最新版本\n",
    "    environment_variables={\n",
    "        \"MODEL_ID\": \"damo/nlp_csanmt_translation_en2zh\",  # ModelScope的模型ID\n",
    "        \"TASK\": \"translation\",  # 模型的任务类型\n",
    "        \"REVISION\": \"v1.0.1\",  # 模型的版本号\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.common.utils import random_str\n",
    "from pai.predictor import Predictor\n",
    "\n",
    "# 部署模型，在PAI-EAS创建一个推理服务\n",
    "p: Predictor = m.deploy(\n",
    "    service_name=\"ms_model_{0}\".format(random_str(8)),  # 配置推理服务名称\n",
    "    instance_type=\"ecs.gn6i-c4g1.xlarge\",  # 配置推理服务实例规格\n",
    "    options={\n",
    "        \"metadata.rpc.keepalive\": 20000,  # 配置推理服务RPC超时时间: 20s\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "通过以上方式部署的模型推理服务，支持通过空字符串的预测请求，获取模型的输入输出信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pprint import pprint\n",
    "from pai.predictor import RawResponse\n",
    "\n",
    "# 通过一个空的预测请求，获取模型的推理输入输出的数据格式\n",
    "res: RawResponse = p.raw_predict(data=\"\")\n",
    "\n",
    "pprint(res.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "基于以上获得的输入数据格式信息，我们可以构建相应的预测请求，发送给到推理，获取翻译结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = p.predict(\n",
    "    # 参考以上的获得的输入输出数据格式，配置推理请求的数据\n",
    "    data={\n",
    "        \"input\": {\n",
    "            \"text\": \"Alibaba Group's mission is to let the world have no difficult business\"\n",
    "        }\n",
    "    }\n",
    ")\n",
    "pprint(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在测试完成之后，删除推理服务，释放机器资源。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 删除推理服务\n",
    "p.delete_service()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}