huggingface / picotron File Size

Intro

File size measurements show the distribution of size of files.
Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.

File Size Overall

Legend:

1001+

501-1000

201-500

101-200

1-100

File Size per Extension

1001+

501-1000

201-500

101-200

1-100

File Size per Logical Decomposition

primary

1001+

501-1000

201-500

101-200

1-100

Longest Files (Top 18)

File	# lines	# units
model.py in picotron	231	26
train.py in root	215	1
checkpoint.py in picotron	212	11
tensor_parallel.py in picotron/tensor_parallel	186	11
extract_metrics.py in root	157	9
submit_slurm_jobs.py in root	157	10
context_parallel.py in picotron/context_parallel	143	9
data.py in picotron	115	6
pipeline_parallel.py in picotron/pipeline_parallel	114	7
create_config.py in root	114	1
bucket.py in picotron/data_parallel	83	11
utils.py in picotron	78	8
data_parallel.py in picotron/data_parallel	76	13
tp_communications.py in picotron/tensor_parallel	73	12
process_group_manager.py in picotron	47	3
cp_communications.py in picotron/context_parallel	45	4
pp_communications.py in picotron/pipeline_parallel	43	2
setup.py in root	10	1

Files With Most Units (Top 18)

File	# lines	# units
model.py in picotron	231	26
data_parallel.py in picotron/data_parallel	76	13
tp_communications.py in picotron/tensor_parallel	73	12
checkpoint.py in picotron	212	11
tensor_parallel.py in picotron/tensor_parallel	186	11
bucket.py in picotron/data_parallel	83	11
submit_slurm_jobs.py in root	157	10
extract_metrics.py in root	157	9
context_parallel.py in picotron/context_parallel	143	9
utils.py in picotron	78	8
pipeline_parallel.py in picotron/pipeline_parallel	114	7
data.py in picotron	115	6
cp_communications.py in picotron/context_parallel	45	4
process_group_manager.py in picotron	47	3
pp_communications.py in picotron/pipeline_parallel	43	2
train.py in root	215	1
create_config.py in root	114	1
setup.py in root	10	1

Files With Long Lines (Top 16)

There are 16 files with lines longer than 120 characters. In total, there are 77 long lines.

File	# lines	# units	# long lines
train.py in root	215	1	15
process_group_manager.py in picotron	47	3	11
pipeline_parallel.py in picotron/pipeline_parallel	114	7	10
cp_communications.py in picotron/context_parallel	45	4	7
create_config.py in root	114	1	5
pp_communications.py in picotron/pipeline_parallel	43	2	4
bucket.py in picotron/data_parallel	83	11	4
model.py in picotron	231	26	4
submit_slurm_jobs.py in root	157	10	4
tensor_parallel.py in picotron/tensor_parallel	186	11	3
data.py in picotron	115	6	3
checkpoint.py in picotron	212	11	2
tp_communications.py in picotron/tensor_parallel	73	12	2
extract_metrics.py in root	157	9	1
context_parallel.py in picotron/context_parallel	143	9	1
data_parallel.py in picotron/data_parallel	76	13	1