Path	Lines of Code
src/fairseq/fairseq/model_parallel/__init__.py	1
src/fairseq/fairseq/model_parallel/criterions/__init__.py	6
src/fairseq/fairseq/model_parallel/criterions/vocab_parallel_cross_entropy.py	46
src/fairseq/fairseq/model_parallel/megatron_trainer.py	44
src/fairseq/fairseq/model_parallel/models/__init__.py	8
src/fairseq/fairseq/model_parallel/models/roberta/__init__.py	1
src/fairseq/fairseq/model_parallel/models/roberta/model.py	182
src/fairseq/fairseq/model_parallel/models/transformer.py	70
src/fairseq/fairseq/model_parallel/models/transformer_lm.py	70
src/fairseq/fairseq/model_parallel/modules/__init__.py	11
src/fairseq/fairseq/model_parallel/modules/multihead_attention.py	244
src/fairseq/fairseq/model_parallel/modules/transformer_layer.py	54
src/fairseq/fairseq/model_parallel/modules/transformer_sentence_encoder.py	48
src/fairseq/fairseq/model_parallel/modules/transformer_sentence_encoder_layer.py	59