huggingface / text-generation-inference Unit Size

Intro

Unit size measurements show the distribution of size of units of code (methods, functions...).
Units are classified in four categories based on their size (lines of code): 1-20 (small units), 20-50 (medium size units), 51-100 (long units), 101+ (very long units).
You should aim at keeping units small (< 20 lines). Long units may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.

Learn more...

Unit Size Overall

There are 2,995 units with 39,132 lines of code in units (39.9% of code).

20 very long units (3,135 lines of code)
69 long units (4,608 lines of code)
452 medium size units (13,445 lines of code)
639 small units (8,871 lines of code)
1,815 very small units (9,073 lines of code)

Legend:

101+

51-100

21-50

11-20

1-10

Unit Size per Extension

101+

51-100

21-50

11-20

1-10

Unit Size per Logical Component

primary logical decomposition

101+

51-100

21-50

11-20

1-10

Alternative Visuals

3D view of all units

Longest Units

Top 50 longest units

Unit	# lines	McCabe index	# params
fn main() in launcher/src/main.rs	275	30	0
fn shard_manager() in launcher/src/main.rs	246	42	33
def concatenate() in server/text_generation_server/models/flash_causal_lm.py	220	21	2
def prepare_for_prefill() in server/text_generation_server/models/flash_causal_lm.py	184	32	1
def concatenate() in backends/gaudi/server/text_generation_server/models/seq2seq_lm.py	178	32	2
def concatenate() in server/text_generation_server/models/seq2seq_lm.py	178	32	2
def generate_token() in server/text_generation_server/models/mamba.py	165	16	2
fn spawn_webserver() in launcher/src/main.rs	164	21	7
def filter() in server/text_generation_server/models/flash_causal_lm.py	157	12	2
def concatenate() in server/text_generation_server/models/causal_lm.py	153	30	2
def filter() in backends/gaudi/server/text_generation_server/models/flash_causal_lm.py	145	9	2
fn image_tokens() in router/src/validation.rs	144	15	4
def cuda_graph_warmup() in server/text_generation_server/models/vlm_causal_lm.py	135	11	4
def cuda_graph_warmup() in server/text_generation_server/models/flash_causal_lm.py	130	11	4
def static() in backends/gaudi/server/text_generation_server/layers/rotary.py	119	18	5
fn next_batch() in backends/v2/src/queue.rs	114	17	5
def static() in server/text_generation_server/layers/rotary.py	111	15	5
fn download_convert_model() in launcher/src/main.rs	108	17	7
fn spawn_shards() in launcher/src/main.rs	108	5	14
def warmup_hpu_graph() in backends/gaudi/server/text_generation_server/models/mllama_causal_lm.py	101	14	2
def warmup_hpu_graph() in backends/gaudi/server/text_generation_server/models/flash_causal_lm.py	100	14	2
def concatenate() in server/text_generation_server/models/mamba.py	95	12	2
fn test_chat_stream_tool_get_weather() in router/src/chat.rs	94	8	0
def filter() in server/text_generation_server/models/idefics_causal_lm.py	92	14	2
fn executor_status_looper() in backends/trtllm/src/looper.rs	90	13	4
def filter() in backends/gaudi/server/text_generation_server/models/seq2seq_lm.py	90	10	2
def filter() in server/text_generation_server/models/seq2seq_lm.py	90	10	2
fn try_into_generate() in router/src/lib.rs	89	4	2
def prefill() in backends/neuron/server/text_generation_server/generator.py	88	27	2
def get_weights_row() in server/text_generation_server/layers/gptq/__init__.py	86	22	3
def filter() in server/text_generation_server/models/causal_lm.py	82	11	2
fn main() in benchmark/src/main.rs	81	4	0
def find_params() in backends/gaudi/server/text_generation_server/layers/gptq/quantize.py	78	21	3
def find_params() in server/text_generation_server/layers/gptq/quantize.py	78	21	3
fn send_responses() in backends/v2/src/backend.rs	76	5	2
fn send_responses() in backends/v3/src/backend.rs	76	5	2
def main() in load_tests/benchmarks.py	75	10	2
def forward() in server/text_generation_server/models/custom_modeling/idefics_modeling.py	75	15	3
fn build_backend() in backends/trtllm/build.rs	74	16	3
def get_weights_row() in backends/gaudi/server/text_generation_server/layers/gptq/__init__.py	74	19	3
def forward() in server/text_generation_server/layers/layernorm.py	70	12	3
fn invariants_hold_on_many_insertions() in backends/v3/src/radix.rs	68	7	1
fn create_event_from_stream_token() in router/src/chat.rs	68	4	7
def _get_quantizer_config() in backends/gaudi/server/text_generation_server/utils/quantization.py	67	14	2
def _get_quantizer_config() in server/text_generation_server/utils/quantization.py	67	14	2
def __init__() in server/text_generation_server/models/custom_modeling/mpt_modeling.py	67	18	4
def get_weights() in server/text_generation_server/layers/gptq/__init__.py	67	15	3
def __init__() in backends/gaudi/server/text_generation_server/models/custom_modeling/flash_llama_modeling.py	66	6	4
fn test_chat_stream_tool_no_tool_simple() in router/src/chat.rs	66	4	0
fn test_chat_stream_usage() in router/src/chat.rs	65	2	0