(title whooshes)
(logo pops)
(title whooshes)
- Let's have a look
inside the token classification pipeline.
In the pipeline video,
we looked at the different applications
the Transformers library supports out of the box,
one of them being token classification,
for instance predicting for each word in a sentence
whether they correspond to a person, an organization
or a location.
We can even group together the tokens corresponding
to the same entity, for instance all the tokens
that formed the word Sylvain here, or Hugging and Face.
The token classification pipeline works the same way
as the text classification pipeline we studied
in the previous video.
There are three steps.
The tokenization, the model, and the postprocessing.
The first two steps are identical
to the text classification pipeline,
except we use an auto token classification model
instead of a sequence classification one.
We tokenize our text then feed it to the model.
Instead of getting one number for each possible label
for the whole sentence, we get one number
for each of the possible nine labels
for every token in the sentence, here 19.
Like all the other models of the Transformers library,
our model outputs logits,
which we turn into predictions by using a SoftMax.
We also get the predicted label for each token
by taking the maximum prediction,
since the SoftMax function preserves the orders,
we could have done it on the logits
if we had no need of the predictions.
The model config contains the label mapping
in its id2label field.
Using it, we can map every token to its corresponding label.
The label, O, correspond to no entity,
which is why we didn't see it in our results
in the first slide.
On top of the label and the probability,
those results included the start
and end character in the sentence.
We'll need to use the offset mapping
of the tokenizer to get those.
Look at the video linked below
if you don't know about them already.
Then, looping through each token
that has a label distinct from O,
we can build the list of results we got
with our first pipeline.
The last step is to group together tokens
that correspond to the same entity.
This is why we had two labels for each type of entity,
I-PER and B-PER, for instance.
It allows us to know if a token is
in the same entity as the previous one.
Note, that there are two ways of labeling used
for token classification.
One, in pink here, uses the B-PER label
at the beginning of each new entity,
but the other, in blue,
only uses it to separate two adjacent entities
of the same type.
In both cases, we can flag a new entity
each time we see a new label appearing,
either with the I or B prefix,
then take all the following tokens labeled the same,
with an I-flag.
This, coupled with the offset mapping to get the start
and end characters allows us to get the span of texts
for each entity.
(title whooshes)
(title fizzles)