（嗖嗖） (swoosh)
- [旁白] 嗨，本视频将带领大家了解 - [Narrator] Hi, this is going to be a video
push_to_hub API 适用于 about the push_to_hub API
Tensorflow 和 Keras 的版本。 for Tensorflow and Keras.
那么首先，打开我们的 jupyter。 So, to get started, we'll open up our notebook.
你需要做的第一件事就是 And the first thing you'll need to do is log in to
登录你的 HuggingFace 帐户， your HuggingFace account,
例如调用 notebook_login 函数。 for example with the notebook login function.
所以要使用它，你只需调用函数， So to use that, you simply call the function,
随后会弹出窗口。 the popup will emerge.
你需要输入你的用户名和密码， You will enter your username and password,
这里我使用密码管理器拷贝密码， which I'm going to pull out of my password manager here,
然后就登录了。 and you log in.
接下来的两个单元格只是 The next two cells are just
为训练做好准备。 getting everything ready for training.
所以我们只是要加载一个数据集， So we're just going to load a dataset,
我们要对数据集进行分词， we're going to tokenize that dataset,
然后我们将加载我们的模型并 and then we're going to load our model and compile
使用标准 Adam 优化器进行编译。 it with the standard Adam optimizer.
现在我们会运行这些代码。 So I'm just going to run all of those.
我们等几秒钟， We'll wait a few seconds,
一切都应该为训练做好准备。 and everything should be ready for training.
好的。 Okay.
所以现在我们准备好训练了。 So now we're ready to train.
我将向你展示两种方法 I'm going to show you the two ways
你可以将你的模型推送到 Hub。 you can push your model to the Hub.
所以第一个是 PushToHubCallback。 So the first is with the PushToHubCallback.
所以 Keras 中的回调 So a callback in Keras
也会在训练过程中被频繁调用。 is a function that's called regularly during training.
你可以将其设置为在一定数量的 step 后或者在每个 epoch 被调用， You can set it to be called after a certain number of steps,
甚至只是在训练结束时调用一次。 or every epoch, or even just once at the end of training.
所以 Keras 中有很多回调，例如， So a lot of callbacks in Keras, for example,
在平稳期控制学习率衰减， control learning rate decaying on plateau,
诸如此类的事情。 and things like that.
所以这个回调，默认情况下， So this callback, by default,
在每个 epoch 都会将你的模型保存到 Hub 一次。 will save your model to the Hub once every epoch.
这真的非常有用， And that's really helpful,
特别是如果你的训练时间很长， especially if your training is very long,
因为那意味着你可以从那个存储中恢复， because that means you can resume from that save,
所以你可以实现将你的模型进行自动云存储。 so you get this automatic cloud-saving of your model.
你甚至可以利用 And you can even run inference
该回调所上传的模型存储点(checkpoint) with the checkpoints of your model
进行推断过程。 that have been uploaded by this callback.
这意味着你可以， And that means you can,
你懂的，运行一些测试输入 y'know, run some test inputs
并实际查看你的模型在训练的各个阶段 and actually see how your model works
是如何工作的， at various stages during training,
这是一个非常好的功能。 which is a really nice feature.
所以我们要添加 PushToHubCallback， So we're going to add the PushToHubCallback,
它只需要几个参数。 and it takes just a few arguments.
第一个参数是临时目录 So the first argument is the temporary directory
文件在上传到 Hub 之前 that files are going to be saved to
将被保存到该目录。 before they're uploaded to the Hub.
第二个参数是分词器， The second argument is the tokenizer,
第三个参数在这里 and the third argument here
是关键字参数 hub_model_id。 is the keyword argument hub_model_id.
也就是在 HuggingFace Hub 上 So that's the name it's going to be saved under
它将被保存的名称。 on the HuggingFace Hub.
你还可以上传到组织帐户 You can also upload to an organization account
只需在带有斜杠的仓库名称之前 just by adding the organization name
添加组织名称，就像这样。 before the repository name with a slash, like this.
所以你可能没有权限 So you probably don't have permissions
上传到 HuggingFace 组织， to upload to the HuggingFace organization,
如果你有权限可以上传， if you do please file a bug
请提交 bug 并尽快通知我们。 and let us know extremely urgently.
但如果你确实有权限访问你自己的组织， But if you do have access to your own organization,
那么你可以使用相同的方法 then you can use that same approach
将模型上传到他们的帐户 to upload models to their account
而不是你自己的个人模型集。 instead of to your own personal set of models.
所以，一旦你实现了回调， So, once you've made your callback,
你只需在调用 model.fit 时 you simply add it to the callbacks list
将它添加到回调列表。 when you're calling model.fit.
所有内容均为从那里上传， And everything is uploaded for you from there,
没有什么可担心的。 there's nothing else to worry about.
上传模型的第二种方式， The second way to upload a model, though,
就是调用 model.push_to_hub。 is to call model.push_to_hub.
所以这更像是一种一次性的方法。 So this is more of a once-off method.
在训练期间不会定期调用它。 It's not called regularly during training.
你可以随时手动调用它 You can just call this manually whenever you want to
将模型上传到 hub。 upload a model to the hub.
所以我们建议在训练结束后运行这个， So we recommend running this after the end of training,
只是为了确保你有一条 commit 消息 just to make sure that you have a commit message
保证这是训练结束时 to guarantee that this was the final version
模型的最终版本。 of the model at the end of training.
它只是为了确保，你懂的， And it just makes sure that, you know,
你当前正在使用的是最终训练结束的模型 you're working with the definitive end-of-training model
而不是意外从某个地方拿到的 and not accidentally using a checkpoint
模型的某个存储点的版本。 from somewhere along the way.
所以我要运行这两个单元格。 So I'm going to run both of these cells.
然后我要在这里剪视频， And then I'm going to cut the video here,
只是因为训练需要几分钟。 just because training is going to take a couple of minutes.
所以我会直接跳到最后， So I'll skip forward to the end of that,
当模型全部上传后， when the models have all been uploaded,
我会告诉你 and I'm gonna show you how you can
如何访问 Hub 中的模型， access the models in the Hub,
以及其他能够在那里利用模型做到的事情。 and the other things you can do with them from there.
好的，我们回来了， Okay, we're back,
我们的模型已上传。 and our model was uploaded.
两者都由 PushToHubCallback Both by the PushToHubCallback
以及我们在训练后调用 model.push_to_hub。 and also by our call to model.push_to_hub after training.
所以一切看起来都很好。 So everything's looking good.
那么现在如果我们转到我在 HuggingFace 上的个人资料， So now if we drop over to my profile on HuggingFace,
你只需点击 profile 按钮就可以 and you can get there just by clicking the profile button
在下拉列表中打开。 in the dropdown.
我们可以看到 bert-fine-tuned-cola 模型在这里， We can see that the bert-fine-tuned-cola model is here,
并于 3 分钟前更新。 and was updated 3 minutes ago.
所以它总是在你的列表的顶部， So it'll always be at the top of your list,
因为它们是按最近更新时间排序的。 because they're sorted by how recently they were updated.
我们可以立即开始查询我们的模型。 And we can start querying our model immediately.
所以我们训练的数据集 So the dataset we were training on
是 Glue CoLA 数据集， is the Glue CoLA dataset,
CoLA 是 Corpus of Linguistic Acceptability(语言可接受性语料库) and CoLA is an acronym standing for
的首字母缩写。 the Corpus of Linguistic Acceptability.
所以这意味着正在训练模型用来决定 So what that means is the model is being trained to decide
一个句子在语法或语言上是正确的呢， if a sentence is grammatically or linguistically okay,
还是有问题的呢。 or if there's a problem with it.
例如，我们可以说，“This is a legitimate sentence.” For example, we could say, "This is a legitimate sentence."
并且希望它判断 And hopefully it realizes that
这实际上是一个合理的句子。 this is in fact a legitimate sentence.
所以当你第一次调用模型时， So it might take a couple of seconds for the model to load
加载模型可能需要几秒钟。 when you call it for the first time.
所以我可能会在这里从这个视频中剪掉几秒钟。 So I might cut a couple of seconds out of this video here.
好的，我们回来了。 Okay, we're back.
所以模型加载了，我们得到了一个输出结果， So the model loaded and we got an output,
但这里有一个明显的问题。 but there's an obvious problem here.
这些标签并没有真正告诉我们 So these labels aren't really telling us
模型实际为这个输入的句子 what categories the model has actually assigned
分配了哪些类别。 to this input sentence.
所以如果我们想解决这个问题， So if we want to fix that,
我们要确保模型配置 we want to make sure the model config
针对每个标签类别都有正确的名称， has the correct names for each of the label classes,
然后我们要上传该配置。 and then we want to upload that config.
所以我们可以在这里实现。 So we can do that down here.
要获取标签名称， To get the label names,
我们可以从我们加载的数据集中得到它， we can get that from the dataset we loaded,
它其中包含特性属性。 from the features attribute it has.
然后我们可以创建字典 And then we can create dictionaries
“id2label” 和 “label2id”， "id2label" and "label2id",
并将它们设置到模型配置中。 and just assign them to the model config.
然后我们可以推送我们更新的配置， And then we can just push our updated config,
这将覆盖 Hub 仓库中的现有配置。 and that'll override the existing config in the Hub repo.
所以这已经完成了。 So that's just been done.
那么现在，如果我们回到这里， So now, if we go back here,
我要用一个稍微不同的句子 I'm going to use a slightly different sentence
因为句子的输出有时会被缓存。 because the outputs for sentences are sometimes cached.
所以，如果我们想生成新的结果 And so, if we want to generate new results
我将使用一些稍微不同的东西。 I'm going to use something slightly different.
那么，让我们尝试换一个不正确的句子。 So let's try an incorrect sentence.
所以这不是有效的英语语法 So this is not valid English grammar
希望模型能发现这一点。 and hopefully the model will see that.
它会在这里重新加载， It's going to reload here,
所以在这里稍微快几秒， so I'm going to cut a couple of seconds here,
然后我们会看到模型会返回什么结果。 and then we'll see what the model is going to say.
好的。 Okay.
所以这个模型，它的置信度不是很好， So the model, it's confidence isn't very good,
因为我们还没有真正优化 because of course we didn't really optimize
我们的超参数。 our hyperparameters at all.
但它返回的结果显示这句话 But it has decided that this sentence
不可接受的程度大于可接受的程度。 is more likely to be unacceptable than acceptable.
如果我们更努力地训练 Presumably if we tried a bit harder with training
我们可以获得更低的验证集 loss， we could get a much lower validation loss,
因此模型的预测会更准确。 and therefore the model's predictions would be more precise.
但是让我们再试一次我们原来的句子。 But let's try our original sentence again.
当然，因为缓存的问题， Of course, because of the caching issue,
我们看到原来的答案没有改变。 we're seeing that the original answers are unchanged.
所以让我们尝试一个不同的，有效的句子。 So let's try a different, valid sentence.
所以让我们试试，“This is a valid English sentence”。 So let's try, "This is a valid English sentence".
我们看到现在模型的结果是正确的 And we see that now the model correctly decides
它的可接受度非常高， that it has a very high probability of being acceptable,
并且被拒绝的可能性非常低。 and a very low probability of being unacceptable.
所以你可以使用这个 inference API So you can use this inference API
甚至可用于训练期间上传的存储点， even with the checkpoints that are uploaded during training,
能够看到在每个训练的 epoch 时 so it can be very interesting to see how
不同的样本输入所输出的预测结果 the model's predictions for sample inputs change
也是非常有意思的。 with each epoch of training.
另外，你也可以访问 Also, the model we've uploaded
我们上传的模型 is going to be accessible to you and,
如果公开分享，其它任何人都可以访问。 if it's shared publicly, to anyone else.
所以如果你想加载那个模型， So if you want to load that model,
你或其他任何人所需要做的 all you or anyone else needs to do
就是将它加载到 pipeline， is just to load it in either a pipeline,
或者你可以使用其它方式加载它，例如， or you can just load it with, for example,
TFAutoModelForSequenceClassification。 TFAutoModelForSequenceClassification.
然后对于名称，你只需传入 And then for the name you would just simply pass
想要上传的 repo 的路径即可。 the path to the repo you want to upload.
或者下载，不好意思。 Or to download, excuse me.
所以如果我想再次使用这个模型， So if I want to use this model again,
如果我想从 hub 加载它， if I want to load it from the hub,
仅需运行这一行代码。 I just run this one line of code.
该模型将被下载。 The model will be downloaded.
而且，运气好的话，它会准备好 And, with any luck, it'll be ready to
对不同的数据集进行微调，进行预测， fine-tune on a different dataset, make predictions with,
或者做任何你想做的事情。 or do anything else you wanna do.
以上内容就是关于 So that was a quick overview of how,
在你训练期间或者训练之后 after your training or during your training,
如何将模型上传到 Hub， you can upload models to the Hub,
可以添加存储点， you can checkpoint there,
可以恢复训练， you can resume training from there,
以及从所上传模型 and you can get inference results
获得推理结果。 from the models you've uploaded.
谢谢大家，希望在以后的视频再会。 So thank you, and I hope to see you in a future video.
（嗖嗖） (swoosh)