（引人注目的介绍） (intro striking)
- 在本视频中，我们将研究编码器架构。 - In this video, we'll study the encoder architecture.
一个流行的仅使用编码器架构的例子是 BURT An example of a popular encoder only architecture is BURT
这是同类产品中最受欢迎的模型。 which is the most popular model of its kind.
让我们首先了解它是如何工作的。 Let's first start by understanding how it works.
我们将使用一个三个单词的小例子。 We'll use a small example using three words.
我们使用这些作为输入传递给编码器。 We use these as inputs and pass them through the encoder.
得到了每个单词的数值表示。 We retrieve a numerical representation of each word.
例如，在这里，编码器将这三个词， Here, for example, the encoder converts those three words,
welcome to NYC，转换为这三个数字序列。 welcome to NYC, in these three sequences of numbers.
编码器对于每个输入单词 The encoder outputs exactly one sequence of numbers
精确输出一个数字序列。 per input word.
这种数值表示也可以称为 This numerical representation can also be called
特征向量（feature vector）或特征张量（feature tensor）。 a feature vector, or a feature tensor.
让我们深入研究这种表示。 Let's dive into this representation.
每个词包含一个向量 It contains one vector per word
这是通过编码器传递的。 that was passed through the encoder.
每个向量都是 Each of these vector is a numerical representation
该词的数字表示。 of the word in question.
该向量的维度由 The dimension of that vector is defined
模型的架构所决定。 by the architecture of the model.
对于基本 BERT 模型，它是 768。 For the base BERT model, it is 768.
这些表示包含一个词的值， These representations contain the value of a word,
但包含上下文化的处理。 but contextualized.
例如，与单词 "to" 相关联的向量 For example, the vector attributed to the word "to"
不只是 “to” 这个词的表示。 isn't the representation of only the "to" word.
它还考虑了它周围的词 It also takes into account the words around it
我们称之为上下文。 which we call the context.
正如它在左侧上下文中所看到的那样， As in it looks to the left context,
我们正在学习的左边的单词， the words on the left of the one we're studying,
这里是 “Welcome” 这个词， here the word "Welcome",
和右边的上下文，这里是 “NYC” 这个词， and the context on the right, here the word "NYC",
并在给定上下文的情况下输出单词的值。 and it outputs a value for the word given its context.
因此，它是一个上下文化的值。 It is therefore a contextualized value.
可以说 768 个值的向量 One could say that the vector of 768 values
保留文本中单词的含义。 holds the meaning of the word within the text.
由于自注意力机制，它做到了这一点。 It does this thanks to the self-attention mechanism.
自注意力机制指的是与单个序列中的不同位置 The self-attention mechanism relates to different positions,
或不同单词相关联 or different words in a single sequence
以计算该序列的表示形式。 in order to compute a representation of that sequence.
正如我们之前所见，这意味着 As we've seen before, this means that
一个词的结果表示 the resulting representation of a word
已被序列中的其他词影响。 has been affected by other words in the sequence.
我们不会在这里深入细节 We won't dive into the specifics here
我们会提供一些进一步的阅读资料 which will offer some further readings
如果您想对底层发生了什么 if you want to get a better understanding
有更好的理解。 at what happens under the hood.
那么为什么要使用编码器呢？ So why should one use and encoder?
编码器可用作独立模型 Encoders can be used as stand-alone models
在各种各样的任务中。 in a wide variety of tasks.
例如，BERT， For example, BERT,
可以说是最著名的 transformer 模型， arguably the most famous transformer model,
它是一个独立的编码器模型， is a standalone encoder model,
并且在发布时， and at the time of release,
它是许多 it'd be the state of the art
序列分类任务 in many sequence classification tasks,
问答任务， question answering tasks,
和掩码语言建模等任务中的最先进技术。 and mask language modeling to only cite of few.
编码器非常擅长 The idea is that encoders are very powerful
提取包含有意义信息的 at extracting vectors that carry
关于序列的向量。 meaningful information about a sequence.
这个向量可以被传递给后续的神经元来进一步处理 This vector can then be handled down the road
以便理解其中包含的信息。 by additional neurons to make sense of them.
让我们看一些例子 Let's take a look at some examples
编码器真正闪耀的地方。 where encoder really shine.
首先，掩码语言建模或 MLM。 First of all, Masked Language Modeling, or MLM.
这是在一个单词序列中 It's the task of predicting a hidden word
预测隐藏词的任务。 in a sequence of word.
在这里，例如，我们在 “My” 和 “is” 之间 Here, for example, we have hidden the word
隐藏了这个词。 between "My" and "is".
这是训练 BERT 的目标之一。 This is one of the objectives with which BERT was trained.
它被训练来预测序列中的隐藏单词。 It was trained to predict hidden words in a sequence.
编码器在这种情况下尤其大放异彩 Encoders shine in this scenario in particular
因为双向信息在这里至关重要。 as bi-directional information is crucial here.
如果我们没有右边的话， If we didn't have the words on the right,
“is”、“Sylvain” 和 “.”， "is", "Sylvain" and the ".",
那么BERT 将能够识别名称的 then there is very little chance
作为正确的词 that BERT would have been able to identify name
的机会就很小。 as the correct word.
为了预测一个掩码单词 The encoder needs to have a good understanding
编码器需要对序列有很好的理解 of the sequence in order to predict a masked word
即使文本在语法上是正确的， as even if the text is grammatically correct,
但不一定符合 it does not necessarily make sense
序列的上下文。 in the context of the sequence.
如前面提到的， As mentioned earlier,
编码器擅长做序列分类。 encoders are good at doing sequence classification.
情感分析是序列分类的一个例子。 Sentiment analysis is an example of sequence classification.
该模型的目的是识别序列的情绪。 The model's aim is to identify the sentiment of a sequence.
它可以从给出的一个序列， It can range from giving a sequence,
做出一颗星到五颗星的评级 a rating from one to five stars
如果进行评论分析 if doing review analysis to giving a positive,
来对一个序列进行积极或消极的评级 or negative rating to a sequence
这就是这里显示的内容。 which is what is shown here.
例如，在这里，给定两个序列， For example, here, given the two sequences,
我们使用模型来计算预测， we use the model to compute a prediction,
并对序列进行分类 and to classify the sequences
在这两个类别中，正面和负面。 among these two classes, positive and negative.
虽然这两个序列非常相似 While the two sequences are very similar
包含相同的词， containing the same words,
意义却完全不同， the meaning is entirely different,
并且编码器模型能够掌握这种差异。 and the encoder model is able to grasp that difference.
（引人注目的结尾） (outro striking)