(air whooshing)
(smiley face popping)
(air whooshing)
- [Instructor] So push to hub API.
Let's have a look at the push to hub API.
You will need to be logged in with your Hugging Face account
which you can do by executing this first cell,
or by typing huggingface-cli login in a terminal.
Just enter you username and password, then click login,
this will store a authentication token
in the cache of the machine you're using.
Now, let's launch a fine tuning of a BERT model
on the GLUE COLA dataset.
We won't go over the fine tuning code
because you can find it in any transformer tutorial,
or by looking at the videos link below.
What interests us here is
how we can leverage the model hub during training.
This is done with the "push_to_hub=true" argument
passed in your TrainingArguments.
This will automatically upload your model
to the Hub each time it is saved,
so every epoch in our case.
This allows you to resume training from a different machine
if the current one gets interrupted.
The model will be updated in your name space
with the name of the output directory you picked by default.
You can choose another name
by passing it to the hub_model_id argument.
You can also push inside an organization you are a member of
by passing a full repository name,
with the name of the organization/,
the model ID you want to pick.
With that done, we can just launch training,
and wait a little bit.
I'll cut the waiting time from the video.
Note that the model is pushed asynchronously,
meaning that the training continues
while your model is uploaded to the hub.
When your first commit is finished,
you can go inspect your model on the Hub
by looking inside your name space,
and you'll find it at the very top.
You can even start playing with its inference widget
while it's continuing the training.
The Cola data set tasks the model with determining
if the sentence is grammatically correct on that.
So we pick an example of incorrect sentence to test it.
Note that it'll take a bit of time
to load your model inside the inference APIs,
so first time you try to use it.
We'll cut by time from the video.
There is something wrong with the labels,
but we'll fix it later in this video.
Once your training is finished,
you should do one last push with the trainer
that pushed to a method.
This is for two reason.
First, this will make sure
you are predicting the final version of your model
if you didn't already.
For instance, if you used to save
every in step strategy instead of every second,
this will draft a model card
that will be the landing page of your model repo.
Once the commit is done,
let's go back on our model page and refresh.
We can see the drafters model card
which includes information,
and which one model we find tuned.
So final evaluation loss and metric,
the training hyperparameter used,
the intermediate training results,
and the framework versions we used
so that other people can easily reproduce our results.
On top of all that information,
the trainer also included some metadata that is interpreted
by the Hugging Face website in the model cloud.
You get the value of the metrics reported in a nice widget
as well as a link to a leaderboard with paper with code.
So the Tensorboard runs have also been pushed
to this report, and we can look at them
directly from the model hub
by clicking on the training metrics sub menu.
If you are not using the Trainer API
to fine-tune your model,
you can use a push_to_hub method
on the model, and tokenizer directly.
Let's test this to fix all labels in the inference widget.
The inference widget was using different names for labels
because we did not indicate the correspondence
between integer and label names.
We can fix this in the configuration
by sitting the label2id,
and id2label fields through the proper values
when pushing the model config to the hub.
Once this is done, we can check on the website,
and the model is now showing the proper label.
Now that the model is on the hub,
we can use it from anywhere
as we would any other Transformer model
with the from_pretrained method
or with the pipeline function.
We just have to use the identifier from the hub,
and we can see that the model configuration and weights
as well as the tokenized files are automatically downloaded.
Try the push_to_hub API in the next training
to easily share your model with the rest of the world.
(air whooshing)