- [Instructor] In this video, we're going to understand how
to manage a model repository
on the Hugging Face Hub Model Hub.
In order to handle a repository
you should first have a Hugging Face account.
A link to create a new account is available
in the description.
Once you are logged in, you can create a new repository
by clicking on the new model option.
You should be facing a similar modal to the following.
In the owner input, you can put either your own namespace
or any of your organization's namespaces.
The model name is the model identifier
that will then be used
to identify your model on the chosen namespace.
The final choice is between public and private.
Public models are accessible by anyone.
This is the recommended free option,
as this makes your model easily accessible and shareable.
The owners of your namespace are the only ones
who can update and change your model.
A more advanced option is the private option.
In this case,
only the owners of your namespace
will have visibility over your model.
Other users won't know it exists
and will not be able to use it.
Let's create a dummy model to play with.
Once your model is created,
comes the management of that model.
Three tabs are available to you.
You're facing the first one, which is the model card page.
This is the page you use to showcase your model
to the world.
We'll see how it can be completed in a bit.
The second one is the files and versions tab.
Your model itself is a Git repository.
If you're unaware of what is a Git repository,
you can think of it as a folder containing files.
If you have never used Git before,
we recommend looking at an introduction
like the one provided in this video's description.
The Git repository allows you to see the changes happening
over time in this folder, hence the term versions.
We'll see how to add files and versions in a bit.
The final tab is the settings tab,
which allows you to manage your model's visibility
and availability.
Let's first start by adding files to the repository.
Files can be added
through the web interface thanks to the add file button.
The added files can be of any type, python, JSON, text,
you name it.
Alongside your added file and its content,
you should name your change or commit.
Generally, adding files is simpler
by using the Hugging Face Hub Python library
or by using the command-line.
We'll showcase how to do this
using the Hugging Face Hub Python library,
and there is a link in the description
to the previous version of this video,
showcasing how to do this using Git and the command-line.
First, make sure you're logged
into your Hugging Face account,
either through the command-line or in a Python runtime.
The first approach we'll take a look at
is using the upload file method.
This offers an extremely simple API
to upload files through the hub.
The three required parameters
are the current location of the file,
the path of that file in the repository,
and the idea of the repository to which you're pushing.
There are a few additional parameters.
The token parameter,
if you would like to specify a different token
than the one saved in your cache with your login,
the repo type parameter,
if you would like to push to a data set or a space.
We'll upload a file called readme.md to the repository
using this method.
We first start by saving a file with that name,
which contains some information
about the repository itself.
Here, a title.
Now that the file is saved,
let's use the upload file method to upload it to the hub.
If we switch to the web interface for a second
and refresh the page, we'll see that the README is shown.
The file upload was a success.
Alongside this method exists a delete file method
so that you may manage your repository fully.
We'll use it to delete the file we have just created.
If we refresh the page once again, good,
the file was indeed deleted.
This approach using only these two methods is super simple.
It doesn't need Git or Git LFS installed,
but it does come with a limitation.
The maximum file size one can upload
is limited to five gigabytes.
To overcome this limit,
let's take a look at the second method
which is the repository utility.
This class is a wrapper over Git and Git LFS methods,
which abstracts most of the complexity
and offers a flexible API
to manage your online repositories.
Let's take a look at how it works.
We first start by instantiating the repository utility.
We provide the clone from parameter,
in order to clone the repository we just created.
The repository is now cloned in the local folder.
The repo object that we have just initialized
offers quite a few methods which are useful for us.
We're interested in pushing a model to the hub.
I'll start by loading a model and tokenizer
I trained a few hours ago.
We'll now follow the traditional Git approach
by first pulling latest changes
using the Git pull method.
We just cloned the repository,
so unless this is a super active repository,
it's unlikely that new changes are available.
But it's always a good idea to pull the latest changes
before doing anything new.
Now that we have pulled the repository,
I'll save the model and tokenizer inside that folder.
This includes the model weights, configuration file,
and tokenizer files.
Now that the model is saved,
we'll continue with the traditional Git approach
and push it to the remote repository.
If we were using the command-line,
there are a few Git LFS specific commands
we would have to invoke.
But here, the Hugging Face hub package
takes care of all of that.
We'll start by staging the files using the Git add method.
We'll then commit these changes using Git commit method,
and providing a helpful commit message.
Finally, we'll push the changes to the remote,
using the Git push method.
If we go back to the files and versions tab,
we can now see the newly committed files.
We can even play with the model in the inference API.
Unfortunately, the front page of our model
is still very empty.
Let's add a README markdown file
to complete it a little bit.
This README is known as the model card
and it's arguably as important
as the model and tokenizer files in the model repository.
It is the central definition
and documentation of your model,
ensuring reusability by fellow community members
and reproducibility of results.
Providing a platform on which other members
may build their artifacts.
We'll only add a title and a small description here
for simplicity's sake,
but we encourage you to add information relevant
to how was the model trained,
it's intended use and limitations,
as well as it's identified potential biases,
evaluation results,
and code samples on how to use your model.
Great work contributing a model to the Model Hub.
This model can now be used in downstream libraries
simply by specifying your model identifier.