openai / preparedness - Conditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 420 units with 3,943 lines of code in units (31.3% of code).

0 very complex units (0 lines of code)
2 complex units (192 lines of code)
5 medium complex units (277 lines of code)
21 simple units (509 lines of code)
392 very simple units (2,965 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def basic_agent_loop() in project/paperbench/paperbench/agents/aisi-basic-agent/_basic_agent_plus.py	103	36	0
def basic_agent_loop() in project/paperbench/paperbench/agents/aisi-basic-agent/_basic_agent_iterative.py	89	27	0
def main() in project/nanoeval/nanoeval/monitor.py	113	16	0
def search_file() in project/paperbench/paperbench/agents/aisi-basic-agent/_file_reader.py	49	16	0
def task_to_alcatraz_config() in project/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py	38	13	2
def check_log() in project/paperbench/paperbench/monitor/monitor.py	47	12	2
def per_paper_results() in project/paperbench/paperbench/metrics.py	30	12	2
def should_include_file() in project/paperbench/paperbench/judge/judge.py	19	10	1
def move_node() in project/paperbench/paperbench/gui/app.py	34	10	0
def sanitize_line() in project/paperbench/paperbench/judge/utils.py	14	9	1
def read_file_chunk() in project/paperbench/paperbench/agents/aisi-basic-agent/_file_reader.py	28	9	0
def __post_init__() in project/paperbench/paperbench/rubric/tasks.py	11	9	1
def setup_logging() in project/nanoeval/nanoeval/library_config.py	37	8	1
def _create_clean_results() in project/nanoeval/nanoeval/evaluation.py	23	8	2
def make_performance_table() in project/paperbench/experiments/judge_eval/judge_eval_perf_tables.py	51	8	2
def _create_tree_structure() in project/paperbench/paperbench/judge/judge.py	21	8	2
def _garbage_collect_net_disk() in project/alcatraz/alcatraz/clusters/local.py	21	7	2
def plot_perf_cost() in project/paperbench/experiments/judge_eval/judge_eval_perf_cost.py	68	7	2
def list_example_ids() in project/paperbench/paperbench/judge/judge_eval/registry.py	11	7	1
def reduce_log() in project/paperbench/paperbench/judge/utils.py	25	7	1
def gather_eval_runs() in project/paperbench/paperbench/nano/utils.py	22	7	2
def reduce_to_category() in project/paperbench/paperbench/rubric/tasks.py	13	7	2
def get_loky_executor() in project/nanoeval/nanoeval/_multiprocessing_utils.py	11	6	1
def file_processor() in project/paperbench/paperbench/nano/utils.py	15	6	3
def get_descendants_with_duplicate_ids() in project/paperbench/paperbench/rubric/tasks.py	5	6	1
def sanitize_request_data() in project/paperbench/paperbench/gui/app.py	8	6	1
def delete_node() in project/paperbench/paperbench/gui/app.py	33	6	0
def move_node_to_parent() in project/paperbench/paperbench/gui/app.py	39	6	0
def wait_for_health() in project/alcatraz/alcatraz/clusters/local.py	17	5	2
def _demux_multiplexed_stream() in project/alcatraz/alcatraz/clusters/_container_proc.py	15	5	1
def valid_ipython_code() in project/nanoeval/nanoeval/solvers/computer_tasks/code_execution_interface.py	10	5	1
def filter() in project/nanoeval/nanoeval/library_config.py	6	5	2
def _validate_concurrency() in project/nanoeval/nanoeval/eval.py	8	5	1
def create_depth_score_plot() in project/paperbench/experiments/judge_max_depth/plot.py	31	5	2
def _get_leaf_node_token_usages() in project/paperbench/paperbench/judge/judge.py	11	5	1
def handle_reasoning_effort() in project/paperbench/paperbench/judge/create_judge.py	12	5	2
def uses_local_config() in project/paperbench/paperbench/nano/utils.py	9	5	1
def build_agent_command() in project/paperbench/paperbench/nano/utils.py	8	5	1
def build_judge_command() in project/paperbench/paperbench/nano/utils.py	18	5	2
def build_agent_command() in project/paperbench/paperbench/agents/run.py	8	5	2
def get_gpu_generation() in project/paperbench/paperbench/agents/aisi-basic-agent/utils.py	16	5	0
def get_parent() in project/paperbench/paperbench/rubric/tasks.py	11	5	2
def _garbage_collect_net() in project/alcatraz/alcatraz/clusters/local.py	16	4	1
def validate_correct_indices() in project/nanoeval/nanoeval/solvers/mcq.py	8	4	1
def _choice_to_answer_group() in project/nanoeval/nanoeval/solvers/mcq.py	5	4	1
def _human_readable_size() in project/nanoeval/nanoeval/_persistent_db.py	9	4	1
def _lock_and_yield_port() in project/nanoeval/nanoeval/_aiomonitor.py	17	4	0
def _load_result() in project/nanoeval/nanoeval/monitor.py	12	4	1
def from_dict() in project/paperbench/paperbench/judge/judge.py	17	4	2
def score_from_children() in project/paperbench/paperbench/judge/judge.py	6	4	1