huggingface / lm-evaluation-harness Conditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 1,356 units with 11,798 lines of code in units (22.9% of code).

1 very complex units (157 lines of code)
1 complex units (157 lines of code)
37 medium complex units (1,791 lines of code)
69 simple units (1,787 lines of code)
1,248 very simple units (7,906 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def process_results() in lm_eval/api/task.py	157	65	3
def cli_evaluate() in lm_eval/__main__.py	157	47	2
def get_train_overlap() in lm_eval/decontamination/decontaminate.py	102	22	3
def _get_task_and_group() in lm_eval/tasks/__init__.py	61	21	2
def apply() in lm_eval/filters/extraction.py	61	20	3
def apply() in lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py	61	20	3
def apply() in lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py	61	20	3
def recreate_metadata_card() in lm_eval/loggers/evaluation_tracker.py	126	19	1
def generate_system_df() in scripts/zeno_visualize.py	26	18	2
def generate_until() in lm_eval/models/openai_completions.py	54	18	3
def main() in scripts/zeno_visualize.py	81	17	0
def parse_math_answer() in lm_eval/tasks/agieval/utils.py	63	17	1
def main() in scripts/write_out.py	47	16	0
def doc_to_target() in lm_eval/api/task.py	38	16	2
def generate_until() in lm_eval/models/openai_completions.py	56	15	3
def process_docs() in lm_eval/tasks/qasper/utils.py	53	15	2
def _sacreformat() in lm_eval/api/metrics.py	12	14	2
def _adjust_config() in lm_eval/evaluator.py	40	14	1
def _detect_batch_size() in lm_eval/models/huggingface.py	69	14	3
def strip_string() in lm_eval/tasks/hendrycks_math/utils.py	30	14	1
def _strip_string() in lm_eval/tasks/agieval/utils.py	30	14	1
def _loglikelihood_tokens() in lm_eval/models/nemo_lm.py	67	13	3
def calculate_score_fullscale() in lm_eval/tasks/eq_bench/utils.py	34	13	2
def make_table() in lm_eval/utils.py	51	13	3
def extract_value() in scripts/regression.py	16	12	5
def get_context() in lm_eval/api/samplers.py	27	12	3
def __getattr__() in lm_eval/api/model.py	43	12	2
def generate_until() in lm_eval/models/neuralmagic.py	51	12	2
def loglikelihood_rolling() in lm_eval/models/neuron_optimum.py	39	12	3
def get_subtask_list() in lm_eval/evaluator_utils.py	38	12	3
def gen_lang_yamls() in lm_eval/tasks/afrimgsm/utils.py	76	12	3
def do_ngrams_in_buckets() in scripts/clean_training_data/generate_13_grams.py	51	11	3
def doc_to_text() in lm_eval/api/task.py	28	11	2
def generate_until() in lm_eval/models/nemo_lm.py	45	11	2
def score_cooked() in lm_eval/tasks/code_x_glue/code-text/bleu.py	38	11	4
def compute_scores() in lm_eval/tasks/coqa/utils.py	15	11	2
def _sanitize_results_dict() in lm_eval/loggers/wandb_logger.py	27	11	1
def _log_results_as_table() in lm_eval/loggers/wandb_logger.py	39	11	1
def log_eval_samples() in lm_eval/loggers/wandb_logger.py	35	11	4
std::vector clean_ngram() in scripts/clean_training_data/janitor_util.cpp	45	10	3
def main() in scripts/regression.py	47	10	0
def _parse_logprobs() in lm_eval/models/vllm_causallms.py	29	10	3
def _model_generate() in lm_eval/models/huggingface.py	18	10	5
def loglikelihood() in lm_eval/models/gguf.py	28	10	3
def gen_lang_yamls() in lm_eval/tasks/translation/utils.py	47	10	2
def gen_lang_yamls() in lm_eval/tasks/mgsm/utils.py	62	10	3
def load_yaml_config() in lm_eval/utils.py	30	10	4
def make_table() in scripts/make_table_results.py	30	9	1
def get_prompt() in lm_eval/prompts/__init__.py	44	9	3
def __post_init__() in lm_eval/api/task.py	32	9	1