XLM/src/trainer.py (2 lines): - line 468: pred_mask[0] = 0 # TODO: remove - line 577: if False: # AMP checkpoint reloading is buggy, we cannot do that - TODO: fix - https://github.com/NVIDIA/apex/issues/250 XLM/src/model/__init__.py (1 line): - line 147: # TODO: only output when necessary - len(params.clm_steps + params.mlm_steps) > 0