notebooks/name-entity-extraction.ipynb (515 lines of code) (raw):
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Named Entity Recognition on IPU using BERT - Inference\n",
"\n",
"Integration of the Graphcore Intelligence Processing Unit (IPU) and the [š¤ Transformers library](https://huggingface.co/docs/transformers/index) means that it only takes a few lines of code to perform complex tasks which require deep learning.\n",
"\n",
"In this notebook we perform **name entity recognition (NER)** also known as token classification. Name entity recognition uses natural language processing models to classify the words inside a prompt. \n",
"\n",
"The ease-of-use of the `pipeline` interface lets us quickly experiment with the pre-trained models and identify which one will work best.\n",
"This simple interface means that it is straightforward to access the fast inference performance of the IPU on your application.\n",
"\n",
"<img src=\"images/name_entity_extraction.png\" alt=\"Widget inference on a token classification task\" style=\"width:800px;\">\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"| Domain | Tasks | Model | Datasets | Workflow | Number of IPUs | Execution time |\n",
"|---------|-------|-------|----------|----------|--------------|--------------|\n",
"| Natural language processing | Token classification | Multiple | - | Inference | 4 | ~4min |\n",
"\n",
"[](https://www.graphcore.ai/join-community)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Environment setup\n",
"\n",
"The best way to run this demo is on Paperspace Gradient's cloud IPUs because everything is already set up for you.\n",
"\n",
"[](https://ipu.dev/3XgZ7V2)\n",
"\n",
"To run the demo using other IPU hardware, you need to have the Poplar SDK enabled. Refer to the [Getting Started guide](https://docs.graphcore.ai/en/latest/getting-started.html#getting-started) for your system for details on how to enable the Poplar SDK. Also refer to the [Jupyter Quick Start guide](https://docs.graphcore.ai/projects/jupyter-notebook-quick-start/en/latest/index.html) for how to set up Jupyter to be able to run this notebook on a remote IPU machine."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Dependencies and configuration\n",
"\n",
"In order to improve usability and support for future users, Graphcore would like to collect information about the\n",
"applications and code being run in this notebook. The following information will be anonymised before being sent to Graphcore:\n",
"\n",
"- User progression through the notebook\n",
"- Notebook details: number of cells, code being run and the output of the cells\n",
"- Environment details\n",
"\n",
"You can disable logging at any time by running `%unload_ext graphcore_cloud_tools.notebook_logging.gc_logger` from any cell."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Install the dependencies for this notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install \"optimum-graphcore==0.7\"\n",
"%pip install emoji==0.6.0 gradio\n",
"%pip install graphcore-cloud-tools[logger]@git+https://github.com/graphcore/graphcore-cloud-tools\n",
"%load_ext graphcore_cloud_tools.notebook_logging.gc_logger"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The location of the cache directories can be configured through environment variables or directly in the notebook:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"executable_cache_dir = os.getenv(\"POPLAR_EXECUTABLE_CACHE_DIR\", \"./exe_cache/\")\n",
"share_gradio = bool(os.getenv(\"GRADIO_SHARE_APP\", False))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## NER with the `transformers` pipelines on the IPU\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The simplest way to get a model running on the IPU is through the `transformers` library, which provides the `pipeline` function that bundles together a set of models which have been validated to work on a range of different tasks. \n",
"\n",
"Let's load our model config to start using pipelines on the IPU:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from optimum.graphcore import pipelines\n",
"inference_config = dict(layers_per_ipu=[40], ipus_per_replica=1, enable_half_partials=True,\n",
" executable_cache_dir=executable_cache_dir)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"For our named entity extraction (NER) task, we can use the `pipeline` function and set our task to `ner` which loads the [TokenClassificationPipeline](https://huggingface.co/docs/transformers/v4.25.1/en/main_classes/pipelines#transformers.TokenClassificationPipeline).\n",
"\n",
"The `inference_config` can now be used to initialise the pipeline on the IPU:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ner_pipeline = pipelines.pipeline(\"ner\", \n",
" ipu_config=inference_config, \n",
" padding='max_length', \n",
" max_length=256)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can create a prompt which we can use to test our pipeline.\n",
"The general `ner_pipeline` should identify locations, names, organisations and miscellaneous items."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ner_examples = [\n",
" \"My name is Janet and I live in Berlin, I work at the hospital as a Doctor.\",\n",
" \"Anita was an incredible software developer working for Google, she lived in Spain but commuted to London regularly\",\n",
" \"The best thing about shopping at Walmart is the options! I never got this many options when I lived in Croatia.\"\n",
"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can use our model pipeline to do NER on our examples. For instance, let's look at our model outputs for our first prompt.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"output_ner = ner_pipeline(ner_examples[0])\n",
"output_ner"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The above output displays the results of our model for the first prompt in our examples list. This output is not very intuitive or immediately useful.\n",
"Instead, let's see what our model's outputs are if we build a fast and simple `gradio` app which uses our pipeline to process our outputs on the IPU.\n",
"\n",
"Using `gradio`, the `app_for_pipeline` function will build a small app which includes a text prompt and will render the entities which were identified:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import gradio as gr\n",
"prompt = \"Let's use an app to do some text summarization!\"\n",
"out = ner_pipeline(prompt)\n",
"\n",
"def app_for_pipeline(pipeline, examples=[], description=\"\", label_description=\"\"):\n",
" demo = gr.Blocks( \n",
" title=description,\n",
" )\n",
" with demo:\n",
" inputs = gr.Textbox(\n",
" lines=3,\n",
" )\n",
" outputs=gr.HighlightedText(\n",
" label=label_description,\n",
" combine_adjacent=True,\n",
" value=dict(text=prompt, entities=out)\n",
" )\n",
" examples_block = gr.Examples(examples=examples, inputs=inputs, outputs=outputs)\n",
" inputs.change(\n",
" fn=lambda x: dict(text=x, entities=pipeline(x)),\n",
" inputs=inputs, outputs=outputs, postprocess=True\n",
" )\n",
" return demo"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's see what our examples look like within the app."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"out = ner_pipeline(prompt)\n",
"demo = app_for_pipeline(ner_pipeline, ner_examples).launch(share=share_gradio)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"That looks great!\n",
"\n",
"Using `gradio` we are able to clearly tell which words are being correctly categorised by our model.\n",
"\n",
"This is all aided by the IPU which quickly processes our inputs and returns the model outputs to create a very responsive interface.\n",
"\n",
"Next we must detach our model from the IPU to release resources. You can learn more about this in our resource management notebook `useful-tips/managing_ipu_resources.ipynb`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ner_pipeline.model.detachFromDevice()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Pipelines on the IPU also provides us with the flexibility and simplicity to quickly change the task to suit our needs. \n",
"\n",
"In the next sections, we will see how easy it is to swap out the default `ner` model with a multilingual model and a biomedical model, enabling us to effectively run experiments on prompts specific to these applications. This will be achieved by creating pipelines which are just as responsive and interactive as our first experiment as we will be utilising the processing power of the IPU."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Multilingual model\n",
"\n",
"The advantage of using pipelines on the IPU is that we can quickly load different models for different tasks.\n",
"\n",
"The first pipeline was specifically trained for English, but we can look at other model checkpoints which are able to classify inputs from multiple languages.\n",
"\n",
"The [`Davlan/bert-base-multilingual-cased-ner-hrl`](https://huggingface.co/Davlan/bert-base-multilingual-cased-ner-hrl) has been fine-tuned for 10 languages: Arabic, German, English, Spanish, French, Italian, Latvian, Dutch, Portuguese and Chinese. \n",
"\n",
"This checkpoint is able to identify similar classes to our first pipeline. It can identify location (LOC), organizations (ORG), and person (PER). \n",
"\n",
"Let's load this checkpoint using the `pipeline` function:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"multilingual_model = \"Davlan/bert-base-multilingual-cased-ner-hrl\"\n",
"ner_pipeline_multilingual = pipelines.pipeline(\n",
" \"ner\", model=multilingual_model, ipu_config=inference_config,\n",
" padding='max_length', max_length=256\n",
")\n",
"multilingual_output = ner_pipeline_multilingual(prompt)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can create some prompts that should work within this new model. The following examples are in French, Latvian and Spanish."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"multilingual_examples = [\"A Budapest, il y a une grande piscine que les touristes visitent.\",\n",
" \"Vai Marriot viesnÄ«cÄ BarselonÄ ir palikusi brÄ«va vieta?\",\n",
" \"Usamos la aerolínea Easy Jet para llegar allí.\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Again, we can port our model pipeline to the `gradio` app which we created earlier.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"multilingual_demo = app_for_pipeline(ner_pipeline_multilingual, examples=multilingual_examples)\n",
"multilingual_demo.launch(share=share_gradio)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"From doing that we have seen how easy it is to swap to the multilingual model, which works really well at identifying and extracting information from a variety of different languages.\n",
"\n",
"We can now free up resources by detaching the model from the IPU."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ner_pipeline_multilingual.model.detachFromDevice()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### BioMedical BERT\n",
"\n",
"In this section, we will see how to use pipelines to execute name entity extraction within the biomedical field.\n",
"\n",
"Within the biomedical industry, hospital staff often have to read and analyse a large amount of text from patient records such as medical histories. \n",
"Their ability to reliably retrieve specific information about patients is extremely vital to their job, which could be challenging, particularly for patients with large medical histories. \n",
"\n",
"NER would be a powerful tool to utilise for assisted tagging. Highlighting the critical information within these records could enable hospital workers to analyse information with more ease and efficiency.\n",
"\n",
"Thankfully, we already have a model which is trained to do exactly that. The [`d4data/biomedical-ner-all`](https://huggingface.co/d4data/biomedical-ner-all?n.) has been fine tuned using biomedical case studies, and is able to extract 84 different entity types related to age, sex, medical history, symptoms, events, and many other classes.\n",
"\n",
"Let's load up this checkpoint and see how it performs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"medical_model = \"d4data/biomedical-ner-all\"\n",
"ner_pipeline_medical = pipelines.pipeline(\n",
" \"ner\", model= medical_model, \n",
" ipu_config=inference_config,\n",
" padding='max_length', \n",
" max_length=256\n",
")\n",
"medical_output = ner_pipeline_medical(prompt)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can create some examples which are more focused on medical cases to see how useful the model could be."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"medical_examples = [\n",
"\"The 56 year old patient had a really bad sprain in their knee. We might have to do surgery as they have a previous history of a damaged ACL.\",\n",
"\"This winter there were outbreaks of Covid-19 , flu and colds. The worst cases were in those over the age of 70 with pre-existing health conditions such as heart disease.\",\n",
"\"The 98 year old woman was extremely healthy with very few medical conditions, just arthritis and high cholesterol as expected for her age.\"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see another use case for NER with a model tuned for biomedical data!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"app_for_pipeline(\n",
" ner_pipeline_medical, \n",
" examples=medical_examples,\n",
" description=\"Try prompting me with some medical anecdotes!\"\n",
").launch(share=share_gradio)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"After testing out our app and model we must now free up resources by detaching the model from the IPU."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ner_pipeline_medical.model.detachFromDevice()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see, the results of the model are very descriptive and we are able to test out different inputs in our pipeline to enable us to identify important patient information using the biomedical model.\n",
"\n",
"## Conclusion\n",
"\n",
"This notebook showed us how easy it is to use the IPU interactively through a `gradio` app. The IPU was utilised as a powerful backend for inference, giving us an incredibly fast and responsive interface for real-time results on user inputs for the NER task.\n",
"\n",
"This was done using only 2 lines of code! All we had to do was define the IPU config and pass that to the pipeline. \n",
"\n",
"This ease-of-use allowed for flexibility when changing tasks to solve problems in the biomedical field and for multilingual inputs. Using this notebook you can go a step further by experimenting with many other NER models which are available on the [š¤ Models Hub](https://huggingface.co/models?pipeline_tag=token-classification&sort=downloads).\n",
"\n",
"While this notebook is focused on using the model for inference, our token classification `other-use-cases/token_classification.ipynb` notebook will show you how to use your own dataset to fine-tune a model using the [`datasets`](https://huggingface.co/docs/datasets/index) package.\n",
"\n",
"The method used to enable the IPU to use pipelines can even be replicated for other tasks such as sentiment analysis, translation and summarization, meaning that you can get started on any task with hundreds of models available on the [š¤ Models Hub.](https://huggingface.co/models?pipeline_tag=token-classification&sort=downloads). Look at our sentiment analysis notebook `sentiment_analysis.ipynb` to try out another example.\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
},
"vscode": {
"interpreter": {
"hash": "d1bd6c553dc7dbe296c3f04b9ff50a6085e8a81154db1d78d45e9882f1633497"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}