openai / simple-evals Conditional Complexity

openai / simple-evals

Conditional Complexity

The distribution of complexity of units (measured with McCabe index).

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

To learn more about conditional complexity and techniques for reducing this type of complexity, Sokrates recommends the following resources:

Simplifying Conditional Expressions, sourcemaking.com
Cyclomatic Complexity, wikipedia.org

Conditional Complexity Overall

There are 100 units with 1,746 lines of code in units (33.0% of code).

1 very complex units (470 lines of code)
0 complex units (0 lines of code)
2 medium complex units (217 lines of code)
11 simple units (321 lines of code)
86 very simple units (738 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def main() in simple_evals.py	470	57	0
def main() in run_multilingual_mmlu.py	122	22	0
def __call__() in healthbench_meta_eval.py	95	17	2
def __call__() in simpleqa_eval.py	54	8	2
def __call__() in sampler/chat_completion_sampler.py	37	8	2
def _compute_stat() in common.py	17	8	2
def __call__() in drop_eval.py	48	8	2
def __call__() in sampler/responses_sampler.py	46	7	2
def get_usage_dict() in healthbench_eval.py	33	7	1
def _match_numbers_if_present() in drop_eval.py	12	7	2
def answer_json_to_strings() in drop_eval.py	20	7	2
def __call__() in sampler/o_chat_completion_sampler.py	30	6	2
def grade_rubric_item() in healthbench_eval.py	18	6	1
def parse_answer() in mgsm_eval.py	6	6	2
def __call__() in sampler/claude_sampler.py	37	5	2
def __call__() in browsecomp_eval.py	39	5	2
def main() in healthbench_eval.py	31	5	0
def _align_bags() in drop_eval.py	11	5	2
def _compute_f1() in drop_eval.py	16	5	2
def __call__() in gpqa_eval.py	38	4	2
def __init__() in mmlu_eval.py	10	4	3
def __call__() in mmlu_eval.py	33	4	2
def normalize_extracted_answer() in common.py	16	4	1
def _remove_punc() in drop_eval.py	5	4	1
def _normalize_answer() in drop_eval.py	8	4	1
def fuzzy_match() in drop_eval.py	6	4	2
def __call__() in mgsm_eval.py	36	4	2
def __init__() in simpleqa_eval.py	11	3	4
def grade_sample() in simpleqa_eval.py	13	3	4
def __init__() in browsecomp_eval.py	11	3	4
def grade_sample() in browsecomp_eval.py	13	3	4
def parse_json_to_dict() in healthbench_eval.py	7	3	1
def __init__() in healthbench_eval.py	12	3	0
def __call__() in healthbench_eval.py	63	3	2
def normalize() in drop_eval.py	7	3	1
def drop_metric() in drop_eval.py	9	3	2
def get_lang_examples() in mgsm_eval.py	11	3	1
def get_all_examples() in mgsm_eval.py	7	3	0
def __init__() in gpqa_eval.py	5	2	0
def decrypt() in browsecomp_eval.py	5	2	2
def __call__() in math_eval.py	22	2	2
def map_with_progress() in common.py	5	2	3
def url_to_fileobj() in common.py	4	2	2
def has_only_user_assistant_messages() in common.py	2	2	1
def _is_number() in drop_eval.py	6	2	1
def _normalize_number() in drop_eval.py	5	2	1
def __init__() in drop_eval.py	18	2	3
def score_mgsm() in mgsm_eval.py	6	2	2
def __init__() in mgsm_eval.py	4	2	0
def __init__() in sampler/chat_completion_sampler.py	6	1	0