Uber / petastormConditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 423 units with 3,720 lines of code in units (82.5% of code).

0 very complex units (0 lines of code)
0 complex units (0 lines of code)
23 medium complex units (705 lines of code)
50 simple units (1,067 lines of code)
350 very simple units (1,948 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def __init__() in petastorm/reader.py	62	21	20
def _sanitize_field_tf_types() in petastorm/tf_utils.py	21	17	1
def form_ngram() in petastorm/ngram.py	26	16	3
def get_results() in petastorm/workers_pool/process_pool.py	32	16	1
def _numpy_and_codec_from_arrow_type() in petastorm/unischema.py	35	16	1
def _validate_ngram() in petastorm/ngram.py	16	15	5
def generate_petastorm_metadata() in petastorm/etl/petastorm_generate_metadata.py	37	15	5
def __init__() in petastorm/fs_utils.py	54	15	7
def _load_rows_with_predicate() in petastorm/py_dict_reader_worker.py	34	14	5
def _load_rows_into_mem() in petastorm/pytorch.py	37	14	3
def from_arrow_schema() in petastorm/unischema.py	35	14	3
def get_results() in petastorm/workers_pool/thread_pool.py	20	13	1
def process() in petastorm/py_dict_reader_worker.py	27	12	4
def _add_many() in petastorm/reader_impl/pytorch_shuffling_buffer.py	31	12	2
def _sanitize_pytorch_types() in petastorm/pytorch.py	18	12	1
def _iter_impl() in petastorm/pytorch.py	25	12	1
def _ventilate() in petastorm/workers_pool/ventilator.py	17	12	1
def __init__() in petastorm/weighted_sampling_reader.py	21	11	3
def encode() in petastorm/codecs.py	23	11	3
def make_reader() in petastorm/reader.py	76	11	27
def add_to_dataset_metadata() in petastorm/utils.py	25	11	3
def transform_schema() in petastorm/transform.py	20	11	2
def create_schema_view() in petastorm/unischema.py	13	11	2
def read_next() in petastorm/arrow_reader_worker.py	35	10	4
def process() in petastorm/arrow_reader_worker.py	27	10	4
def copy_dataset() in petastorm/tools/copy_dataset.py	28	10	9
def _partition_row_groups() in petastorm/reader.py	15	10	7
def _iter_impl() in petastorm/pytorch.py	28	10	1
def run() in petastorm/workers_pool/thread_pool.py	22	10	1
def hdfs_connect_namenode() in petastorm/hdfs/namenode.py	11	10	4
def match_unischema_fields() in petastorm/unischema.py	19	10	2
def load_row_groups() in petastorm/etl/dataset_metadata.py	25	9	1
def read_next() in petastorm/py_dict_reader_worker.py	17	9	4
def make_batch_reader() in petastorm/reader.py	68	9	26
def get_filesystem_and_path_or_paths() in petastorm/fs_utils.py	19	9	4
def get_results() in petastorm/workers_pool/dummy_pool.py	14	9	1
def _keep_retrying_while_zmq_again() in petastorm/workers_pool/process_pool.py	17	9	3
def dict_to_spark_row() in petastorm/unischema.py	26	9	2
def _convert_precision() in petastorm/spark/spark_dataset_converter.py	17	9	2
def convert_fields() in petastorm/ngram.py	8	8	3
def _split_row_groups() in petastorm/etl/dataset_metadata.py	21	8	1
def build_index() in petastorm/etl/rowgroup_indexers.py	13	8	3
def decimal_friendly_collate() in petastorm/pytorch.py	12	8	1
def decode_row() in petastorm/utils.py	20	8	2
def __init__() in petastorm/workers_pool/ventilator.py	25	8	8
def _worker_bootstrap() in petastorm/workers_pool/process_pool.py	53	8	8
def _normalize_databricks_dbfs_url() in petastorm/spark/spark_dataset_converter.py	13	8	2
def main() in petastorm/benchmark/dummy_reader.py	29	7	3
def _load_rows() in petastorm/arrow_reader_worker.py	23	7	4
def _load_rows_with_predicate() in petastorm/arrow_reader_worker.py	33	7	5