Path	Lines of Code
fairseq/model_parallel/__init__.py	1
fairseq/model_parallel/criterions/__init__.py	6
fairseq/model_parallel/criterions/vocab_parallel_cross_entropy.py	58
fairseq/model_parallel/megatron_trainer.py	55
fairseq/model_parallel/models/__init__.py	12
fairseq/model_parallel/models/pipeline_parallel_transformer/__init__.py	1
fairseq/model_parallel/models/pipeline_parallel_transformer/layers.py	439
fairseq/model_parallel/models/pipeline_parallel_transformer/model.py	625
fairseq/model_parallel/models/roberta/__init__.py	1
fairseq/model_parallel/models/roberta/model.py	159
fairseq/model_parallel/models/transformer.py	77
fairseq/model_parallel/models/transformer_lm.py	135
fairseq/model_parallel/modules/__init__.py	10
fairseq/model_parallel/modules/multihead_attention.py	278
fairseq/model_parallel/modules/transformer_layer.py	51