(air whooshing)
- What is transfer learning?
The idea of transfer learning
is to leverage the knowledge acquired
by a model trained with lots of data on another task.
The model A will be trained specifically for task A.
Now let's say you want to train a model B
for a different task.
One option would be to train the model from scratch.
This could take lots of computation, time and data.
Instead, we could initialize model B
with the same weights as model A,
transferring the knowledge of model A on task B.
When training from scratch,
all the model's weight are initialized randomly.
In this example, we are training a BERT model
on the task of recognizing
if two sentences are similar or not.
On the left, it's trained from scratch,
and on the right it's fine-tuning a pretrained model.
As we can see, using transfer learning
and the pretrained model yields better results.
And it doesn't matter if we train longer.
The training from scratch is capped around 70% accuracy
while the pretrained model beats the 86% easily.
This is because pretrained models
are usually trained on large amounts of data
that provide the model with a statistical understanding
of the language used during pretraining.
In computer vision,
transfer learning has been applied successfully
for almost ten years.
Models are frequently pretrained on ImageNet,
a dataset containing 1.2 millions of photo images.
Each image is classified by one of 1000 labels.
Training like this, on labeled data
is called supervised learning.
In Natural Language Processing,
transfer learning is a bit more recent.
A key difference with ImageNet is that the pretraining
is usually self-supervised,
which means it doesn't require humans annotations
for the labels.
A very common pretraining objective
is to guess the next word in a sentence.
Which only requires lots and lots of text.
GPT-2 for instance, was pretrained this way
using the content of 45 millions links
posted by users on Reddit.
Another example of self-supervised pretraining objective
is to predict the value of randomly masked words.
Which is similar to fill-in-the-blank tests
you may have done in school.
BERT was pretrained this way using the English Wikipedia
and 11,000 unpublished books.
In practice, transfer learning is applied on a given model
by throwing away its head, that is,
its last layers focused on the pretraining objective,
and replacing it with a new, randomly initialized head
suitable for the task at hand.
For instance, when we fine-tuned a BERT model earlier,
we removed the head that classified mask words
and replaced it with a classifier with 2 outputs.
Since our task had two labels.
To be as efficient as possible, the pretrained model used
should be as similar as possible
to the task it's fine-tuned on.
For instance, if the problem
is to classify German sentences,
it's best to use a German pretrained model.
But with the good comes the bad.
The pretrained model does not only transfer its knowledge,
but also any bias it may contain.
ImageNet mostly contains images
coming from the United States and Western Europe.
So models fine-tuned with it
usually will perform better on images from these countries.
OpenAI also studied the bias
in the predictions of its GPT-3 model
which was pretrained
using the guess the next word objective.
Changing the gender of the prompt
from he was very to she was very
changed the predictions from mostly neutral adjectives
to almost only physical ones.
In their model card of the GPT-2 model,
OpenAI also acknowledges its bias
and discourages its use
in systems that interact with humans.
(air whooshing)