（嗖嗖） (swoosh)
- 在这个视频中， - In this video,
我们将研究编码-解码器架构。 we'll study the encoder-decoder architecture.
流行的编码-解码器模型的一个示例是 T5。 An example of a popular encoder-decoder model is T5.
为了理解编码-解码器是如何工作的， In order to understand how the encoder-decoder works,
我们建议您查看 we recommend you check out the videos
将编码-解码器作为独立的模型（Encoders and Decoders as standalone models）这一视频。 on encoders and decoders as standalone models.
了解它们如何单独工作 Understanding how they work individually
将有助于理解编码-解码器的工作原理。 will help understanding how an encoder-decoder works.
让我们从我们已了解的编码器开始。 Let's start from what we've seen about the encoder.
编码器将单词作为输入， The encoder takes words as inputs,
通过编码器进行转换， casts them through the encoder,
并检索每个单词的 and retrieves a numerical representation
数值表示。 for each word cast through it.
我们现在知道这个数值表示 We now know that this numerical representation
包含关于序列意义的信息。 holds information about the meaning of the sequence.
让我们把这个放在一边，将解码器添加到图中。 Let's put this aside and add the decoder to the diagram.
在这种情况下， In this scenario,
我们以某种我们以前没见过的方式 we're using the decoder in a manner
使用解码器。 that we haven't seen before.
我们将编码器的输出直接传递给它。 We're passing the outputs of the encoder directly to it.
另外，在给解码器输入序列的同时 Additionally to the encoder outputs,
我们还需要编码器的输出。 we also give the decoder a sequence.
在不给定初始序列的情况下 When prompting the decoder for an output
向解码器提示输出时， with no initial sequence,
我们可以给它一个 we can give it the value that indicates
表示序列开头的值。 the start of a sequence.
这就是编码-解码器魔术发生的地方。 And that's where the encoder-decoder magic happens.
编码器接受一个序列作为输入。 The encoder accepts a sequence as input.
它计算一个预测， It computes a prediction,
并输出一个数值表示。 and outputs a numerical representation.
然后，它将其发送到解码器。 Then, it sends that over to the decoder.
从某种意义上说，它编码了那个序列。 It has, in a sense, encoded that sequence.
反过来，解码器 And the decoder, in turn,
将此输入与其通常的序列输入一起使用， using this input alongside its usual sequence input,
将尝试解码序列。 will take a stab at decoding the sequence.
解码器解码序列， The decoder decodes the sequence,
并输出一个词。 and outputs a word.
到目前为止，我们不需要理解这个词， As of now, we don't need to make sense of that word,
但我们可以理解解码器 but we can understand that the decoder
本质上是解码编码器输出的内容。 is essentially decoding what the encoder has output.
序列词的开头在这里 The start of sequence word here
表示它应该开始解码序列。 indicates that it should start decoding the sequence.
现在我们有了编码器的数字表示 Now that we have both the encoder numerical representation
和一个初始生成的词， and an initial generated word,
我们不再需要编码器了。 we don't need the encoder anymore.
正如我们之前在解码器中看到的那样， As we have seen before with the decoder,
它可以以自回归的方式起作用。 it can act in an auto-regressive manner.
它刚刚输出的单词现在可以用作输入。 The word it has just output can now be used as an input.
这个编码器输出的数值表示 This, in combination with the numerical representation
和初始化的值结合， output by the encoder,
可以被用于生成第二个单词。 can now be used to generate a second word.
请注意，第一个词仍然在这里， Please note that the first word is still here,
因为模型仍然输出它。 as the model still outputs it.
但是，我们已将其变灰 However, we have grayed it out
因为我们不再需要它了。 as we have no need for it anymore.
我们可以继续下去，例如， We can continue on and on, for example,
直到解码器输出一个 until the decoder outputs a value
我们认为是停止值的数值， that we consider a stopping value,
比如句号表示序列的结束。 like a dot meaning the end of a sequence.
在这里，我们已经看到了编码-解码 transformer Here, we've seen the full mechanism
完整的机制。 of the encoder-decoder transformer.
让我们再看一遍。 Let's go over it one more time.
我们有一个初始序列被送到编码器中。 We have an initial sequence that is sent to the encoder.
编码器的输出发送到解码器 That encoder output is then sent to the decoder
以便对其进行解码。 for it to be decoded.
虽然在一次使用后可以丢弃编码器， While it can now discard the encoder after a single use,
但解码器将被多次使用 the decoder will be used several times
直到我们生成了所需要的每一个词。 until we have generated every word that we need.
那么让我们结合翻译语言建模 So let's see a concrete example
看一个具体的例子。 with Translation Language Modeling.
也称为传导， Also called transduction,
这是翻译序列的行为。 which is the act of translating a sequence.
在这里，我们想翻译这个英文序列 Here, we would like to translate this English sequence
“Welcome to NYC”到法语。 "Welcome to NYC" in French.
我们正在使用 transformer 模型 We're using a transformer model
明确针对该任务进行了训练。 that is trained for that task explicitly.
我们使用编码器来创建英语句子 We use the encoder to create a representation
的表示。 of the English sentence.
我们使用编码器来创建英语句子的表示形式， We cast this to the decoder,
然后将其传递给解码器， with the use of the start of sequence word,
在使用开始序列单词的情况下，请求它输出第一个单词。 we ask it to output the first word.
输出 bienvenue，表示欢迎。 It outputs bienvenue, which means welcome.
然后我们使用 bienvenue And we then use bienvenue
作为解码器的输入序列。 as the input sequence for the decoder.
这与编码器数字表示一起， This, alongside the encoder numerical representation,
允许解码器预测第二个词， allows the decoder to predict the second word, Ã,
这是英文的。 which is to in English.
最后，我们要求解码器预测第三个词 Finally, we ask the decoder to predict a third word
它预测 NYC，这是正确的。 It predicts NYC, which is correct.
我们已经翻译了这句话。 We've translated the sentence.
编码-解码器真正发挥作用的地方， Where the encoder-decoder really shines,
是我们有一个编码器和一个解码器， is that we have an encoder and a decoder,
通常不共享权重。 which often do not share weights.
因此，我们有一个完整的块，编码器， Therefore, we have an entire block, the encoder,
可以被训练，从而理解序列 that can be trained to understand the sequence
并提取相关信息。 and extract the relevant information.
对于我们之前看到的翻译场景， For the translation scenario we've seen earlier,
例如，这意味着解析 for example, this would mean parsing
并理解用英语说的内容。 and understanding what was said in the English language.
这意味着从该语言中提取信息， It would mean extracting information from that language,
并将所有这些放在一个信息密集的向量中。 and putting all of that in a vector dense in information.
另一方面，我们有解码器， On the other hand, we have the decoder,
它的唯一目的是解码 whose sole purpose is to decode the numerical representation
编码器输出的数值表示。 output by the encoder.
这个解码器可以专门 This decoder can be specialized
用完全不同的语言， in a completely different language,
甚至像图像或语音这样的模态。 or even modality like images or speech.
编码-解码器之所以特殊，有几个原因。 Encoders-decoders are special for several reasons.
首先，它们能够管理任务的顺序， Firstly, they're able to manage sequence to sequence tasks,
就像我们刚刚看到的翻译一样。 like translation that we have just seen.
其次，编码器和解码器之间的权重 Secondly, the weights between the encoder
并不一定共享。 and the decoder parts are not necessarily shared.
再举一个翻译的例子。 Let's take another example of translation.
这里我们用法语 Here we're translating "Transformers are powerful"
翻译 Transformers are powerful in French.
首先，这意味着从三个单词的序列中， Firstly, this means that from a sequence of three words,
我们能够生成一个包含四个单词的序列。 we're able to generate a sequence of four words.
有人可能会争辩说这可以用解码器来处理 One could argue that this could be handled with a decoder
那会以自回归的方式 that would generate the translation
生成翻译结果， in an auto-regressive manner,
他们是对的。 and they would be right.
基于 Transformers Seq2Seq 模型的 Another example of where sequence to sequence
另一个亮点是进行总结的能力 transformers shine is in summarization.
这里我们有一个很长的序列，通常是全文， Here we have a very long sequence, generally a full text,
我们想总结一下。 and we want to summarize it.
由于编码器和解码器是分开的， Since the encoder and decoders are separated,
我们可以有不同的上下文长度。 we can have different context lengths.
例如，编码器的一个非常长的上下文， For example, a very long context for the encoder,
处理文本， which handles the text,
尔解码器上下文则较小 and a smaller context for the decoder
它处理汇总序列。 which handles the summarized sequence.
有很多序列到序列的模型。 There are a lot of sequence to sequence models.
这包含了 Transformers 库中 This contains a few examples
几个受欢迎的 of popular encoder-decoder models
编码-解码器模型的示例。 available in the transformers library.
此外，您可以在编码-解码器模型中 Additionally, you can load an encoder and a decoder
加载编码器和解码器。 inside an encoder-decoder model.
因此，根据您要解决的具体任务， Therefore, according to the specific task you are targeting,
您可能会选择使用在这些具体任务上证明 you may choose to use specific encoders and decoders,
其价值的特定编码器和解码器。 which have proven their worth on these specific tasks.
（嗖嗖） (swoosh)