（嘶嘶声） (whooshing sound)
（贴纸弹出） (sticker popping)
（嘶嘶声） (whooshing sound)
在我们的很多例子中， In a lot of our examples,
你将看到 DataCollators 一遍又一遍地弹出。 you're going to see DataCollators popping up over and over.
它们用于 PyTorch 和 TensorFlow 工作流程， They're used in both PyTorch and TensorFlow workflows,
甚至在 JAX 中， and maybe even in JAX,
但是没有人真正知道 JAX 中发生了什么。 but no-one really knows what's happening in JAX.
不过，我们确实有一个研究团队正在研究它， We do have a research team working on it though,
所以也许他们很快就会告诉我们。 so maybe they'll tell us soon.
但是回到主题。 But coming back on topic.
什么是数据整理器？ What are data collators?
数据整理器整理数据。 Data collators collate data.
好像和没说一样。 That's not that helpful.
更具体地说，他们整理了一份样本清单 But to be more specific, they put together a list of samples
组成一个单独的小批量训练数据。 into a single training minibatch.
对于某些任务， For some tasks,
数据整理器可以非常简单明了。 the data collator can be very straightforward.
例如，当你进行序列分类时， For example, when you're doing sequence classification,
你真正需要数据整理器做的是 all you really need from your data collator
将你的样本数据填充到相同的长度 is that it pads your samples to the same length
并将它们连接成一个单独的 Tensor。 and concatenates them into a single Tensor.
但对于其他工作流程，数据整理器可能非常复杂 But for other workflows, data collators can be quite complex
因为他们为特定任务 as they handle some of the preprocessing
完成一些预处理操作。 needed for that particular task.
所以，如果你想使用数据整理器， So, if you want to use a data collator,
对于 PyTorch 用户，你通常将数据整理器 for PyTorch users, you usually pass the data collator
传递到你的 Trainer 对象。 to your Trainer object.
在 TensorFlow 中，情况有点不同。 In TensorFlow, it's a bit different.
使用数据整理器的最简单方法 The easiest way to use a data collator
是将它传递给数据集的 to_tf_dataset 方法。 is to pass it to the to_tf_dataset method of your dataset.
这会给你 tensorflow_tf_data.dataset And this will give you a tensorflow_tf_data.dataset
然后你可以将其传递给 model.fit。 that you can then pass to model.fit.
你将在示例中看到这些方法 You'll see these approaches used in the examples
和整个课程的笔记本。 and notebooks throughout this course.
另请注意，我们所有的整理器 Also note that all of our collators
可接受 return_tensors 参数。 take a return_tensors argument.
你可以将其设置为 “pt” 以获取 PyTorch Tensor， You can set this to "pt" to get PyTorch Tensors,
"tf" 获取 TensorFlow Tensor， "tf" to get TensorFlow Tensors,
或 “np” 获取 Numpy 数组。 or "np" to get Numpy arrays.
出于向后兼容的原因， For backward compatibility reasons,
默认值为 “pt”， the default value is "pt",
所以 PyTorch 用户甚至大多数情况下 so PyTorch users don't even have to set this argument
不必设置这个参数。 most of the time.
因此，他们通常完全没有意识到 And so as a result, they're often totally unaware
这个参数的存在。 that this argument even exists.
我们可以从中学到一些东西 We can learn something from this
也就是特权的受益人 which is that the beneficiaries of privilege
往往最无视它的存在。 are often the most blind to its existence.
好吧好吧，扯远了。 But okay, coming back.
让我们看看一些特定的数据整理器是如何工作的。 Let's see how some specific data collators work in action.
虽然再次记住 Although again, remember if none
如果内置数据整理器无法满足你的需求， of the built-in data collators do what you need,
你可以自己实现数据整理器，而且它们通常很短。 you can always write your own and they're often quite short.
所以首先，我们将看到 “基本” 数据整理器。 So first, we'll see the "basic" data collators.
它们是 DefaultDataCollator 和 DataCollatorWithPadding。 These are DefaultDataCollator and DataCollatorWithPadding.
如果在准备训练之前 These are the ones you should use
你的标签很简单 if your labels are straightforward
并且你的数据不需要任何特殊处理 and your data doesn't need any special processing
就可以使用上述的数据整理器 before being ready for training.
请注意，因为不同的模型 Notice that because different models
有不同的填充词元， have different padding tokens,
DataCollatorWithPadding 将需要你模型的 Tokenizer DataCollatorWithPadding will need your model's Tokenizer
所以它知道如何正确填充序列。 so it knows how to pad sequences properly.
默认的数据整理器不需要 Tokenizer 工作， The default data collator doesn't need a Tokenizer to work,
但它会因此抛出错误 but it will as a result throw an error
除非你所有的序列都是相同的长度。 unless all of your sequences are the same length.
所以，你应该意识到这一点。 So, you should be aware of that.
继续前进。 Moving on though.
许多其他数据整理器 A lot of the other data collators
除了基本的两个， aside from the basic two are,
它们通常旨在处理一项特定任务。 they're usually designed to handle one specific task.
所以，我要在这里展示两个整理器。 And so, I'm going to show a couple here.
它们是 DataCollatorForTokenClassification These are DataCollatorForTokenClassification
和 DataCollatorForSeqToSeq。 and DataCollatorForSeqToSeq.
而这些任务需要特殊整理器的原因 And the reason these tasks need special collators
是因为它们的标签长度可变。 is because their labels are variable in length.
在词元分类中，每个词元都有一个标签， In token classification there's one label for each token,
标签的长度 and so the length of the labels
是序列的长度。 is the length of the sequence.
而在 SeqToSeq 中，标签是一系列词元 While in SeqToSeq the labels are a sequence of tokens
它可以是可变长度， that can be variable length,
那可能会和输入序列 that can be very different
的长度不同。 from the length of the input sequence.
所以在这两种情况下，我们通过填充标签 So in both of these cases, we handle collating that batch
处理整理那批数据， by padding the labels as well,
正如你在此示例中看到的那样。 as you can see here in this example.
因此，如果我们想加入可变长度的样本 So, inputs and the labels will need to be padded
到同一个小批量数据， if we want to join samples of variable length
就需要填充输入和标签。 into the same minibatch.
这正是数据整理者所做的 That's exactly what the data collators
而这正是这些数据整理者将为我们做的 and that's exactly what these data collators will do for us
你懂的，为了这个特定的任务。 you know, for this particular task.
那么，还有最后一个 So, there's one final data collator
想在这里与大家分享的数据整理器。 I want to show you as well just in this lecture.
这就是 DataCollatorForLanguageModeling。 And that's the DataCollatorForLanguageModeling.
嗯，它非常重要，首先， So, it's very important, and it's firstly,
因为语言模型是我们这段时间以来 because language models are just so foundational
处理 NLP 所涉及的最基本的概念。 to do for everything we do with NLP these days.
但其次，因为它有两种模式 But secondly, because it has two modes
可以完成两件截然不同的事情。 that do two very different things.
因此，你可以使用 mlm 参数选择所需的模式。 So you choose which mode you want with the mlm argument.
将其设置为 True 以进行屏蔽语言建模， Set it to True for masked language modeling,
并将其设置为 False 以进行因果语言建模。 and set it to False for causal language modeling.
因此，为因果语言建模整理数据 So, collating data for causal language modeling
其实很简单。 is actually quite straightforward.
该模型只是做出预测 The model is just making predictions
接下来是什么词元，所以你的标签 for what token comes next, and so your labels
或多或少只是你输入的副本， are more or less just a copy of your inputs,
整理器会处理你的输入 and the collator will handle that
并确保正确填充输入和标签。 and ensure that the inputs and labels are padded correctly.
但是，当你将 mlm 设置为 True 时， When you set mlm to True though,
你会得到完全不同的效果， you get quite different behavior,
这与任何其他数据整理器不同， that's different from any other data collator,
那是因为将 mlm 设置为 True and that's because setting mlm to True
表示屏蔽语言建模 means masked language modeling
这意味着标签必须是 and that means the labels need to be,
你懂的，输入需要被屏蔽。 you know, the inputs need to be masked.
那么，那看起来像什么？ So, what does that look like?
所以，回想一下，在屏蔽语言建模中， So, recall that in masked language modeling,
该模型没有预测下一个词， the model is not predicting the next word,
相反，我们随机屏蔽掉一些词元 instead we randomly mask out some tokens
模型会同时预测所有这些。 and the model predicts all of them at once.
所以，对于那些被屏蔽的词元 So, it tries to kinda fill in the blanks
它试图填补空白。 for those masked tokens.
但是随机掩蔽的过程出奇地复杂。 But the process of random masking is surprisingly complex.
如果我们遵循原始 BERT 论文中的协议， If we follow the protocol from the original BERT paper,
我们需要用掩码词元替换一些词元， we need to replace some tokens with a masked token,
一些其他带有随机词元的词元， some other tokens with a random token,
然后保持第三组词元不变。 and then keep a third set of tokens unchanged.
好吧，具体细节和我们这么做的原因 Yeah, this is not the lecture
就不在这里和大家详细说明了。 to go into the specifics of that or why we do it.
如果你好奇的话，可以随时查看原始的 You can always check out the original BERT paper
BERT 论文。 if you're curious.
它写得非常好。也很容易理解。 It's well written. It's easy to understand.
这里要知道的主要事情是自己实现起来 The main thing to know here is that it can be a real pain
是一件非常痛苦的事情，而且也非常地复杂。 and quite complex to implement that yourself.
但是当你将 mlm 设置为 True 时，DataCollatorForLanguageModeling But DataCollatorForLanguageModeling will do it for you
会为你完成。 when you set mlm to True.
这是一些数据整理器所完成的 And that's an example of the more intricate
更加复杂的预处理操作。 preprocessing that some of our data collators do.
就是这样！ And that's it!
因此，这涵盖了最常用的数据整理器 So, this covers the most commonly used data collators
以及它们所针对的具体任务。 and the tasks they're used for.
希望现在你会知道何时使用数据整理器 And hopefully, now you'll know when to use data collators
以及该为你的特定任务选择哪一个。 and which one to choose for your specific task.
（嘶嘶声） (whooshing sound)