（空气呼啸） (air whooshing)
- 增强你的 PyTorch 训练循环 - Supercharge your PyTorch training loop
使用 Hugging Face Accelerate with Hugging Face Accelerate.
有多个设置 There are multiple setups
你可以在其上进行训练： on which you can run your training:
它可以在 CPU、GPU、TPU 上， it could be on CPU, GPUs, TPUs,
分布在具有多个设备的一台机器上， distributed on one machine with several devices,
甚至几台机器，通常称为节点 (node) ， or even several machines, often called nodes,
每个都有多个设备。 each with multiple devices.
在这之上，还有新的调整 On top of that, there are new tweaks
使你的训练更快或更有效， to make your training faster or more efficient,
比如混合精度和 DeepSpeed 。 like mixed precision and DeepSpeed.
这每一个设置或训练调整 Each of those setups or training tweaks
都要求你更改训练循环的代码 requires you to change the code of your training loop
以某种方式, 或者学习新的 API。 in one way or another and to learn a new API.
所有这些设置都由 Trainer API 处理， All those setups are handled by the Trainer API,
并且有几个第三方库可以提供帮助。 and there are several third-party libraries that can help.
它们的问题 The problem with those
是他们感觉像个黑盒子 is that they can feel like a black box
并且实现调整可能并不容易 and that it might not be easy to implement the tweak
到你需要的训练循环上。 to the training loop you need.
Accelerate 是专门设计的 Accelerate has been designed specifically
以让你保持对训练循环的完全控制 to let you retain full control over your training loop
并尽可能不打乱。 and be as non-intrusive as possible.
只需四行代码即可添加到你的训练循环中， With just four lines of code to add to your training loop,
这里显示在训练循环视频的例子中， here shown on the example of the training loop video,
Accelerate 将处理所有设置 Accelerate will handle all the setups
和第一张幻灯片中提到的训练调整。 and training tweaks mentioned on the first slide.
只需学习和掌握一个 API It's only one API to learn and master
而不是 10 个不同的。 instead of 10 different ones.
更具体地说，你必须导入和实例化 More specifically, you have to import and instantiate
一个 accelerator 对象， an accelerator object,
这将处理所有必要的代码 that will handle all the necessary code
为你的特定设置。 for your specific setup.
然后你必须发给它模型 Then you have to send it the model,
你正在使用的优化器和数据加载器 optimizer and dataloaders you are using
在 prepare 方法中，这是要记住的主要方法。 in the prepare method, which is the main method to remember.
加速处理设备放置， Accelerate handles device placement,
所以你不需要把你的分批 so you don't need to put your batch
在你使用的特定设备上。 on the specific device you are using.
最后，你必须更换 loss.backward 行 Finally, you have to replace the loss.backward line
成 accelerator.backwardloss， by accelerator.backwardloss,
这就是你所需要的！ and that's all you need!
Accelerate 还处理分布式评估。 Accelerate also handles distributed evaluation.
你仍然可以使用经典的评估循环 You can still use a classic evaluation loop
比如我们在训练循环视频中看到的那个， such as the one we saw in the training loop video,
在这种情况下所有进程 in which case all processes
将进行全面评估。 will perform the full evaluation.
要使用分布式评估， To use a distributed evaluation,
你只需要像这样调整你的评估循环： you just have to adapt your evaluation loop like this:
传递评估数据加载器 pass along the evaluation dataloader
到 accelerator.prepare 方法，比如训练。 to the accelerator.prepare method, like for training.
然后你可以关闭这行 Then you can dismiss the line
将分批放在适当的设备上， that places the batch on the proper device,
在输入你的预测 and just before passing your predictions
和指标的标签之前，使用 accelerator.gather and labels to your metric, use accelerator.gather
来收集预测 to gather together the predictions
和每个进程的标签。 and labels from each process.
分布式训练脚本 A distributed training script
必须在不同的进程中多次启动， has to be launched several times on different processes,
例如，你使用的每个 GPU 一个。 for instance, one per GPU you are using.
你可以使用 PyTorch 工具来做到这一点 You can use the PyTorch tools to do that
如果你熟悉他们， if you are familiar with them,
但 Accelerate 还提供了一个简单的 API but Accelerate also provides an easy API
配置你的设置并启动你的训练脚本。 to configure your setup and launch your training script.
在终端中，运行加速配置 In a terminal, run accelerate config
并回答小问题 and answer the small questionnaire
生成配置文件 to generate a configuration file
含所有相关信息， with all the relevant information,
然后你可以运行加速启动， then you can just run accelerate launch,
然后是训练脚本的路径。 followed by the path to your training script.
在 notebook 中，你可以使用 notebook 启动器函数 In a notebook, you can use the notebook launcher function
开始你的训练。 to launch your training.
（空气呼啸） (air whooshing)