(air whooshing)
(air popping)
(air whooshing)
- So Trainer API.
So Transformers Library provides a Trainer API
that allows you to easily find tune transformers models
on your dataset.
So Trainer class takes your datasets,
your model as well as the training hyperparameters
and can perform the training on any kind of setup,
CPU, GPU, multiple GPUs, TPUs
can also compute the predictions
on any dataset and if you provided metrics
evaluate your model on any dataset.
You can also involve final data processing
such as dynamic padding,
as long as you provide the tokenizer
or given data collator.
We will try this API on the MRPC dataset,
since it's relatively small and easy to preprocess.
As we saw in the Datasets overview video,
however we can preprocess it.
We do not apply padding during the preprocessing,
as we will use dynamic padding
before DataCollatorWithPadding.
Note that we don't do the final steps
of renaming removing columns
or set the format to torch tensors.
So Trainer will do all of this automatically for us
by analyzing the model signature.
The last step before creating the Trainer are
to define a model
and some training hyperparameters.
We saw to do the first in the model API video.
For the second we use the TrainingArguments class.
It only takes a path to a folder
where results and checkpoint will be saved,
but you can also customize all the hyperparameters
your Trainer will use,
learning weight, number of training impacts, et. cetera.
It's been very easy to create a Trainer
and launch a training.
You should display a progress bar
and after a few minutes if you're running on a GPU
you should have the training finished.
The result will be rather anticlimactic however,
as you will only get a training loss
which doesn't really tell you anything
about how well your model is performing.
This is because we didn't specify any metric
for the evaluation.
To get those metrics,
we will first gather the predictions
on the whole evaluation set using the predict method.
It returns a namedtuple with three fields,
Prediction, which contains the model of predictions.
Label_IDs, which contains the labels
if your dataset had them
and metrics which is empty here.
We're trying to do that.
The predictions are the logits of the models
for all the sentences in the dataset.
So a NumPy array of shape 408 by 2.
To match them with our labels,
we need to take the maximum logit
for each prediction
to know which of the two classes was predicted.
We do this with the argmax function.
Then we can use a metric from the Datasets library.
It can be loaded as easily as a dataset
with the load metric function
and each returns the evaluation metric used for the dataset.
We can see our model did learn something
as it is 85.7% accurate.
To monitor the evaluation matrix during training,
we need to define a compute_metrics function
that does the same step as before.
It takes a namedtuple with predictions and labels
and must return a dictionary
with the metrics we want to keep track of.
By passing the epoch evaluation strategy
to our training arguments,
we tell the Trainer to evaluate
at the end of every epoch.
Launching a training inside a notebook
will then display a progress bar
and complete the table you see here as you pass every epoch.
(air whooshing)
(air decrescendos)