huggingface / datablations Conditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 39 units with 422 lines of code in units (7.3% of code).

0 very complex units (0 lines of code)
0 complex units (0 lines of code)
2 medium complex units (88 lines of code)
3 simple units (62 lines of code)
34 very simple units (272 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 39 most complex units

Unit	# lines	McCabe index	# params
def main() in utils/hub_sync.py	36	18	0
def main() in utils/merge_generative.py	52	13	0
def get_lr() in training/mup.py	33	9	1
def filter_shards() in filtering/deduplication/filter_oscar_jsonl.py	7	7	1
def get_git_files_by_status() in utils/hub_sync.py	22	6	1
def group_texts() in training/mup.py	10	5	1
def _get_closed_form_lr() in training/mup.py	13	4	1
def get_new_and_modified_files() in utils/hub_sync.py	16	4	1
def hub_config_repo() in utils/hub_sync.py	18	4	2
def params() in utils/flops-params_py.py	15	4	7
def find_all_json() in utils/merge_generative.py	10	4	1
def sort_dict() in utils/merge_generative.py	10	4	1
def get_pairs() in filtering/deduplication/add_dedup_info.py	14	4	1
def get_url() in filtering/deduplication/add_dedup_info.py	6	4	2
def get_hub_data() in utils/hub_sync.py	8	3	0
def output_path() in utils/hf_dataset_subsampling.py	10	3	3
def find_whitespace() in filtering/deduplication/save_rust_format.py	4	3	1
def find_whitespace() in filtering/deduplication/dedup_oscar.py	4	3	1
def get_segmentation() in filtering/deduplication/dedup_oscar.py	10	3	3
def get_untracked_files() in utils/hub_sync.py	4	2	1
def get_modified_files() in utils/hub_sync.py	4	2	1
def run_cmd() in utils/hub_sync.py	13	2	2
def get_size_per_example() in utils/hf_dataset_subsampling.py	4	2	1
def full_flops() in utils/flops-params_py.py	15	2	7
def simple_flops() in utils/flops-params_py.py	4	2	8
def get_bytes() in filtering/deduplication/add_dedup_info.py	7	2	2
def add_duplication_info() in filtering/deduplication/add_dedup_info.py	12	2	2
def get_perplexity() in filtering/deduplication/save_roots_sample.py	4	2	1
def tok() in filtering/deduplication/hf_dataset_to_file.py	7	2	1
def generator_from_dataset() in filtering/deduplication/suffix_dedup.py	3	2	1
def save_dataset() in filtering/deduplication/save_dataset_sample.py	9	2	4
def __init__() in training/mup.py	8	1	0
def get_args() in utils/hub_sync.py	6	1	0
def get_args() in utils/hf_dataset_subsampling.py	11	1	0
def get_total_byte_size() in utils/hf_dataset_subsampling.py	2	1	1
def get_dataset_size() in utils/flops-params_py.py	2	1	8
def get_doc_id() in filtering/deduplication/add_dedup_info.py	4	1	3
def sep() in filtering/deduplication/hf_dataset_to_file.py	4	1	0
def get_args() in filtering/deduplication/suffix_dedup.py	1	1	0