(light transition music)
- Let's have a look inside the question answering pipeline.
The question answering pipeline
can extract answers to questions
from a given context or passage of text
like this part of the Transformers repo README.
It also works for very long context,
even if the answer is at the very end, like in this example.
In this video, we'll see why.
The question answering pipeline
follows the same steps as the other pipelines.
The question and context are tokenized as a sentence pair,
fed to the model then some post-processing is applied.
So tokenization and model steps should be familiar.
We use the auto class suitable for question answering
instead of sequence classification,
but one key difference with text classification
is that our model outputs two tensors
named start logits and end logits.
Why is that?
Well, this is the way the model finds the answer
to the question.
First, let's have a look at the model inputs.
It's numbers associated with the tokenization
of the question,
followed by the context
with the usual CLS and SEP special tokens.
The answer is a part of those tokens.
So we ask the model to predict
which token starts the answer
and which ends the answer.
For our two logit outputs,
the theoretical labels are the pink and purple vectors.
To convert those logits into probabilities,
we will need to apply a SoftMax,
like in the text classification pipeline.
We just mask the tokens
that are not part of the context before doing that,
leaving the initial CLS token unmasked
as we use it to predict an impossible answer.
This is what it looks like in terms of code.
We use a large negative number for the masking
since its exponential will then be zero.
Now the probability for each start and end position
corresponding to a possible answer
will give a score that is a product
of the start probabilities
and end probabilities at those position.
Of course, a start index greater than an end index
corresponds to an impossible answer.
Here is the code to find the best score
for a possible answer.
Once we have the start and end position for the tokens,
we use the offset mappings provided by our tokenizer
to find the span of characters in the initial context,
and we get our answer.
Now, when the context is long,
it might get truncated by the tokenizer.
This might result in part of the answer, or worse,
the whole answer, being truncated.
So we don't discard the truncated tokens
but build new features with them.
Each of those features contains the question,
then a chunk of text in the context.
If we take disjoint chunks of texts,
we might end up with the answer
being split between two features.
So instead, we take overlapping chunks of text
to make sure at least one of the chunks
will fully contain the answer to the question.
So, tokenizers does all of this for us automatically
with the return overflowing tokens option.
The stride argument
controls the number of overlapping tokens.
Here is how our very long context
gets truncated in two features with some overlap.
By applying the same post-processing
we saw before for each feature,
we get the answer with a score for each of them,
and we take the answer with the best score
as a final solution.
(light transition music)