- [Instructor] Using the Python debugger in a notebook.
In this video, we'll learn how to use the Python debugger
in a Jupyter Notebook or a Colab.
For this example, we are running code
from the token classification section,
downloading the Conll dataset,
looking a little bit at data,
before loading a tokenizer
to preprocess the whole dataset.
Check out the section of the course linked below
for more information.
Once this is done,
we try to load eight features of the training dataset,
and then batch them together,
using tokenizer.pad,
and we get the following error.
We use PyTorch here,
with return_tensors="pt"
but you will get the same error with TensorFlow.
As we have seen in the "How to debug an error?" video,
the error message is at the end of the traceback.
Here, it indicates us we should use padding,
which we are actually trying to do.
So this is not useful at all,
and we will need to go a little deeper
to debug the problem.
Fortunately, you can use the Python debugger
at any time you get an error in a Jupyter Notebook
by typing the magic command, debug, in a cell.
Don't forget the percent at the beginning.
When executing that cell,
you go to the very bottom of the traceback
where you can type commands
that will help you debug your script.
The first two commands you should learn,
are u and d, for up and down.
Typing u and enter will take you up one step
in the traceback to the previous instruction.
Typing d and then enter will take you
one step down in the traceback.
Going up twice, we get to the point the error was reached.
The third command to learn for the debugger is p, for print.
It allows you to print any value you want.
For instance, typing p return_tensors and enter,
we see the value pt that we pass to the bad function.
We can also have a look at the batch outputs
this batch line coding object gets.
The batch outputs dictionary is a bit hard to dig in to,
so let's dive into smaller pieces of it.
Inside the debugger you can not only print any variable
but also evaluate any expression,
for instance, we can have a look at the input_ids keys
this batch_outputs object.
Or at the labels keys of this batch_outputs object.
Those labels are definitely weird:
they are of various sizes,
which we can actually confirm, if we want,
by printing the size with the least compression.
This is because the pad method of the tokenizer
only takes care of the tokenizer outputs:
input IDs, attention mask, and token type IDs,
so we have to pad the labels ourselves
before trying to create a tensor with them.
Once you are ready to exit the Python debugger,
you can press q and enter for quit.
One way to fix the error
is to manually pad the labels to the longest.
Another way is to use a data collator
specifically designed for token classification.
You can also use a Python debugger directly
in the terminal.
Check out the video link below to learn how.