- In this video, we'll study the decoder architecture.
An example
of a popular decoder only architecture is GPT two.
In order to understand how decoders work
we recommend taking a look at the video regarding encoders.
They're extremely similar to decoders.
One can use a decoder
for most of the same tasks as an encoder
albeit with generally a little loss of performance.
Let's take the same approach we have taken
with the encoder to try
and understand the architectural differences
between an encoder and decoder.
We'll use a small example using three words.
We pass them through their decoder.
We retrieve a numerical representation for each word.
Here for example, the decoder converts the three words.
Welcome to NYC, and these three sequences of numbers.
The decoder outputs exactly one sequence
of numbers per input word.
This numerical representation can also
be called a feature vector or a feature sensor.
Let's dive in this representation.
It contains one vector
per word that was passed through the decoder.
Each of these vectors is a numerical representation
of the word in question.
The dimension of that vector is defined
by the architecture of the model.
Where the decoder differs from the encoder is principally
with its self attention mechanism.
It's using what is called masked self attention.
Here, for example, if we focus on the word "to"
we'll see that is vector is absolutely unmodified
by the NYC word.
That's because all the words on the right, also known
as the right context of the word is masked rather
than benefiting from all the words on the left and right.
So the bidirectional context.
Decoders only have access to a single context
which can be the left context or the right context.
The masked self attention mechanism differs
from the self attention mechanism
by using an additional mask to hide the context
on either side of the word
the words numerical representation will not be affected
by the words in the hidden context.
So when should one use a decoder?
Decoders like encoders can be used as standalone models
as they generate a numerical representation.
They can also be used in a wide variety of tasks.
However, the strength of a decoder lies in the way.
A word can only have access to its left context
having only access to their left context.
They're inherently good at text generation
the ability to generate a word
or a sequence of words given a known sequence of words.
This is known
as causal language modeling or natural language generation.
Here's an example of how causal language modeling works.
We start with an initial word, which is my
we use this as input for the decoder.
The model outputs a vector of numbers
and this vector contains information about the sequence
which is here a single word.
We apply a small transformation
to that vector so that it maps
to all the words known by the model, which is a mapping
that we'll see later called a language modeling head.
We identify that the model believes
that the most probable following word is name.
We then take that new word and add it
to the initial sequence from my, we are now at my name.
This is where the auto regressive aspect comes in.
Auto regressive models.
We use their past outputs as inputs and the following steps.
Once again, we do the exact same operation.
We cast that sequence through the decoder
and retrieve the most probable following word.
In this case, it is the word "is", we repeat the operation
until we're satisfied, starting from a single word.
We've now generated a full sentence.
We decide to stop there, but we could continue for a while.
GPT two, for example, has a maximum context size of 1,024.
We could eventually generate up to a 1,024 words
and the decoder would still have some memory
of the first words in this sequence.