huggingface / nanotron File Size

Intro

File size measurements show the distribution of size of files.
Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.

Learn more...

File Size Overall

There are 115 files with 21,887 lines of code.

3 very long files (3,647 lines of code)
9 long files (5,949 lines of code)
24 medium size files (6,999 lines of codeclsfd_ftr_w_mp_ins)
23 small files (3,296 lines of code)
56 very small files (1,996 lines of code)

Legend:

1001+

501-1000

201-500

101-200

1-100

explore: grouped by folders | grouped by size | sunburst | 3D view

File Size per Extension

1001+

501-1000

201-500

101-200

1-100

File Size per Logical Decomposition

primary

1001+

501-1000

201-500

101-200

1-100

Longest Files (Top 50)

File	# lines	# units
ring_attention_lucidrain.py in src/nanotron/nn	1315	47
starcoder2.py in src/nanotron/models	1228	42
trainer.py in src/nanotron	1104	25
llama.py in src/nanotron/models	926	42
qwen.py in src/nanotron/models	810	36
__init__.py in src/nanotron/data/nemo_dataset	690	18
llama3_ring_attention.py in src/nanotron/nn	653	26
decode.py in src/nanotron/generation	649	9
helpers.py in src/nanotron	633	22
tokenized_bytes.py in src/nanotron/data	572	12
config.py in src/nanotron/config	515	26
helpers.cpp in src/nanotron/data/nemo_dataset	501	7
functional.py in src/nanotron/parallel/tensor_parallel	487	13
ring_attention.py in src/nanotron/nn	481	23
p2p.py in src/nanotron/parallel/pipeline_parallel	390	28
scaling_benchmarks.py in scripts	382	6
slurm_launcher.py in root	378	7
evaluation_tasks.py in src/nanotron/eval	345	-
s3_mover.py in src/nanotron/s3_checkpoints	332	24
zero.py in src/nanotron/optim	323	19
weights.py in src/nanotron/serialize	293	7
run_train.py in root	281	3
base.py in src/nanotron/logging	273	19
optimizer.py in src/nanotron/serialize	265	9
dataloader.py in src/nanotron/data	250	6
gradient_accumulator.py in src/nanotron/optim	250	25
samplers.py in src/nanotron/data	245	18
nn.py in src/nanotron/parallel/tensor_parallel	245	13
sanity_checks.py in src/nanotron	241	8
engine.py in src/nanotron/parallel/pipeline_parallel	235	15
indexed_dataset.py in src/nanotron/data/nemo_dataset	225	35
main.py in src/nanotron/serialize	223	2
timers.py in src/nanotron/logging	220	19
base.py in src/nanotron/models	219	33
models_config.py in src/nanotron/config	211	13
flex_attention.py in src/nanotron/nn	205	17
state.py in src/nanotron/parallel/pipeline_parallel	194	30
distributed.py in src/nanotron	179	12
blendable_dataset.py in src/nanotron/data/nemo_dataset	177	10
one_job_runner.py in src/nanotron/eval	175	4
attention.py in src/nanotron/nn	167	7
metrics_logging.py in src/nanotron	166	11
sampler.py in src/nanotron/generation	166	6
nanoset.py in src/nanotron/data	164	7
parametrization.py in src/nanotron/scaling	162	22
clm_collator.py in src/nanotron/data	158	2
rotary.py in src/nanotron/nn	150	6
metadata.py in src/nanotron/serialize	140	13
moe.py in src/nanotron/nn	136	12
parameters.py in src/nanotron/parallel	131	20

Files With Most Units (Top 50)

File	# lines	# units
ring_attention_lucidrain.py in src/nanotron/nn	1315	47
llama.py in src/nanotron/models	926	42
starcoder2.py in src/nanotron/models	1228	42
qwen.py in src/nanotron/models	810	36
indexed_dataset.py in src/nanotron/data/nemo_dataset	225	35
base.py in src/nanotron/models	219	33
state.py in src/nanotron/parallel/pipeline_parallel	194	30
p2p.py in src/nanotron/parallel/pipeline_parallel	390	28
llama3_ring_attention.py in src/nanotron/nn	653	26
config.py in src/nanotron/config	515	26
gradient_accumulator.py in src/nanotron/optim	250	25
trainer.py in src/nanotron	1104	25
s3_mover.py in src/nanotron/s3_checkpoints	332	24
ring_attention.py in src/nanotron/nn	481	23
helpers.py in src/nanotron	633	22
parametrization.py in src/nanotron/scaling	162	22
activations.py in src/nanotron/nn	117	21
parameters.py in src/nanotron/parallel	131	20
base.py in src/nanotron/logging	273	19
timers.py in src/nanotron/logging	220	19
zero.py in src/nanotron/optim	323	19
samplers.py in src/nanotron/data	245	18
__init__.py in src/nanotron/data/nemo_dataset	690	18
flex_attention.py in src/nanotron/nn	205	17
engine.py in src/nanotron/parallel/pipeline_parallel	235	15
random.py in src/nanotron	105	14
models_config.py in src/nanotron/config	211	13
nn.py in src/nanotron/parallel/tensor_parallel	245	13
functional.py in src/nanotron/parallel/tensor_parallel	487	13
metadata.py in src/nanotron/serialize	140	13
tokenized_bytes.py in src/nanotron/data	572	12
moe.py in src/nanotron/nn	136	12
inherit_from_other_optimizer.py in src/nanotron/optim	52	12
utils.py in src/nanotron	103	12
distributed_differentiable_primitives.py in src/nanotron/parallel/tensor_parallel	81	12
distributed.py in src/nanotron	179	12
metrics_logging.py in src/nanotron	166	11
base.py in src/nanotron/optim	126	11
blendable_dataset.py in src/nanotron/data/nemo_dataset	177	10
decode.py in src/nanotron/generation	649	9
optimizer.py in src/nanotron/serialize	265	9
functional.py in src/nanotron/parallel/pipeline_parallel	79	8
sanity_checks.py in src/nanotron	241	8
nanoset.py in src/nanotron/data	164	7
helpers.cpp in src/nanotron/data/nemo_dataset	501	7
attention.py in src/nanotron/nn	167	7
layer_norm.py in src/nanotron/nn	57	7
block.py in src/nanotron/parallel/pipeline_parallel	115	7
context.py in src/nanotron/parallel	124	7
tensor.py in src/nanotron/fp8	58	7

Files With Long Lines (Top 48)

There are 48 files with lines longer than 120 characters. In total, there are 183 long lines.

File	# lines	# units	# long lines
trainer.py in src/nanotron	1104	25	21
config.py in src/nanotron/config	515	26	20
tokenized_bytes.py in src/nanotron/data	572	12	13
scaling_benchmarks.py in scripts	382	6	13
s3_mover.py in src/nanotron/s3_checkpoints	332	24	8
optimizer.py in src/nanotron/serialize	265	9	8
run_train.py in root	281	3	8
llama.py in src/nanotron/models	926	42	6
helpers.py in src/nanotron	633	22	5
zero.py in src/nanotron/optim	323	19	5
models_config.py in src/nanotron/config	211	13	5
blendable_dataset.py in src/nanotron/data/nemo_dataset	177	10	4
starcoder2.py in src/nanotron/models	1228	42	4
qwen.py in src/nanotron/models	810	36	4
metadata.py in src/nanotron/serialize	140	13	4
__init__.py in src/nanotron/data/nemo_dataset	690	18	3
one_job_runner.py in src/nanotron/eval	175	4	3
named_optimizer.py in src/nanotron/optim	57	3	3
p2p.py in src/nanotron/parallel/pipeline_parallel	390	28	3
nn.py in src/nanotron/parallel/tensor_parallel	245	13	3
sanity_checks.py in src/nanotron	241	8	3
main.py in src/nanotron/serialize	223	2	3
preprocess_data.py in tools	87	2	3
clm_collator.py in src/nanotron/data	158	2	2
samplers.py in src/nanotron/data	245	18	2
ring_attention_lucidrain.py in src/nanotron/nn	1315	47	2
decode.py in src/nanotron/generation	649	9	2
block.py in src/nanotron/parallel/pipeline_parallel	115	7	2
random.py in src/nanotron/serialize	25	2	2
sft_processing.py in src/nanotron/data	57	3	1
nanoset.py in src/nanotron/data	164	7	1
processing.py in src/nanotron/data	109	5	1
indexed_dataset.py in src/nanotron/data/nemo_dataset	225	35	1
moe.py in src/nanotron/nn	136	12	1
flex_attention.py in src/nanotron/nn	205	17	1
timers.py in src/nanotron/logging	220	19	1
upload_to_wandb.py in src/nanotron/eval	64	1	1
utils.py in src/nanotron	103	12	1
lighteval_config.py in src/nanotron/config	109	5	1
utils_config.py in src/nanotron/config	99	4	1
engine.py in src/nanotron/parallel/pipeline_parallel	235	15	1
parameters.py in src/nanotron/parallel	131	20	1
context.py in src/nanotron/parallel	124	7	1
base.py in src/nanotron/models	219	33	1
utils.py in src/nanotron/serialize	66	4	1
weights.py in src/nanotron/serialize	293	7	1
weka.py in scripts	71	3	1
slurm_launcher.py in root	378	7	1