（图形嘶嘶作响） (graphics whooshing)
- 如何预处理句子对。 - How to pre-process pairs of sentences.
我们已经看到了如何分词化单个句子 We have seen how to tokenize single sentences
并将它们放在一起， and batch them together in the
在 “Batching inputs together” 视频中 "Batching inputs together" video.
如果这段代码对你而言不熟悉， If this code look unfamiliar to you,
请务必再次查看该视频。 be sure to check that video again.
这里将重点关注对句子进行分类的任务。 Here will focus on tasks that classify pair of sentences.
例如，我们可能想要对两个文本是否被释义 For instance, we may want to classify whether two texts
进行分类。 are paraphrased or not.
这是从 Quora Question Pairs 数据集中获取的示例 Here is an example taken from the Quora Question Pairs dataset
它专注于识别重复的问题。 which focuses on identifying duplicate questions.
在第一对中，两个问题是重复的， In the first pair, the two questions are duplicates,
在第二个它们不是。 in the second they are not.
另一个对分类问题是 Another pair classification problem is
当我们想知道两个句子是 when we want to know if two sentences are
逻辑上相关与否， logically related or not,
一个称为自然语言推理或 NLI 的问题。 a problem called natural language inference or NLI.
在这个取自 MultiNLI 数据集的例子中， In this example, taken from the MultiNLI data set,
对于每个可能的标签，我们都有一对句子。 we have a pair of sentences for each possible label.
矛盾，自然的或蕴涵， Contradiction, natural or entailment,
这是第一句话的奇特表达方式 which is a fancy way of saying the first sentence
暗示第二种。 implies the second.
所以分类成对的句子是一个 So classifying pairs of sentences is a problem
值得研究的问题。 worth studying.
事实上，在 GLUE 基准中， In fact, in the GLUE benchmark,
这是文本分类问题的学术界基准 which is an academic benchmark for text classification
10 个数据集中有 8 个是关注 8 of the 10 datasets are focused
使用句子对的任务。 on tasks using pairs of sentences.
这就是为什么像 BERT 这样的模型 That's why models like BERT
通常预先训练有双重目标。 are often pre-trained with a dual objective.
在语言建模目标之上， On top of the language modeling objective,
他们通常有一个与句子对相关的目标。 they often have an objective related to sentence pairs.
例如，在预训练期间 BERT 见到 For instance, during pretraining BERT is shown
成对的句子，必须同时预测 pairs of sentences and must predict both
随机掩蔽的标记值，以及第二个是否 the value of randomly masked tokens, and whether the second
句子是否接着第一个句子。 sentence follow from the first or not.
幸运的是，来自 Transformers 库的分词器 Fortunately, the tokenizer from the Transformers library
有一个很好的 API 来处理成对的句子。 has a nice API to deal with pairs of sentences.
你只需要将它们作为两个参数传递 You just have to pass them as two arguments
到 tokenizer 。 to the tokenizer.
在我们已经研究过的输入 ID On top of the input IDs and the attention mask
和注意掩码之上， we studied already,
它返回一个名为标记类型 ID 的新字段， it returns a new field called token type IDs,
它告诉模型哪些标记属于 which tells the model which tokens belong
第一句话， to the first sentence,
哪些属于第二句。 and which ones belong to the second sentence.
放大一点，这里有一个输入 ID Zooming in a little bit, here has an input IDs
与它们对应的 token 对齐， aligned with the tokens they correspond to,
它们各自的标记类型 ID 和注意掩码。 their respective token type ID and attention mask.
我们可以看到分词器还添加了特殊标记。 We can see the tokenizer also added special tokens.
所以我们有一个 CLS token ，来自第一句话的 token ， So we have a CLS token, the tokens from the first sentence,
一个 SEP 标记，第二句话中的标记， a SEP token, the tokens from the second sentence,
和最终的 SEP 标记。 and a final SEP token.
如果我们有几对句子， If we have several pairs of sentences,
我们可以通过第一句话的传递列表 we can tokenize them together by passing the list
将它们标记在一起，然后是第二句话的列表 of first sentences, then the list of second sentences
以及我们已经研究过的所有关键字参数 and all the keyword arguments we studied already
例如 padding=True。 like padding=True.
放大结果， Zooming in at the result,
我们可以看到分词器如何添加填充 we can see how the tokenizer added padding
到第二对句子使得两个输出的 to the second pair sentences to make the two outputs
长度相同，并正确处理标记类型 ID the same length, and properly dealt with token type IDs
和两个句子的注意力掩码。 and attention masks for the two sentences.
然后就可以通过我们的模型了。 This is then all ready to pass through our model.