uber / petastorm Conditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 427 units with 3,814 lines of code in units (83.0% of code).

0 very complex units (0 lines of code)
0 complex units (0 lines of code)
23 medium complex units (733 lines of code)
51 simple units (1,089 lines of code)
353 very simple units (1,992 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def __init__() in petastorm/reader.py	64	21	21
def process() in petastorm/arrow_reader_worker.py	41	20	4
def _sanitize_field_tf_types() in petastorm/tf_utils.py	21	17	1
def _numpy_and_codec_from_arrow_type() in petastorm/unischema.py	35	16	1
def form_ngram() in petastorm/ngram.py	26	16	3
def get_results() in petastorm/workers_pool/process_pool.py	32	16	1
def __init__() in petastorm/fs_utils.py	54	15	7
def _validate_ngram() in petastorm/ngram.py	16	15	5
def generate_petastorm_metadata() in petastorm/etl/petastorm_generate_metadata.py	37	15	5
def from_arrow_schema() in petastorm/unischema.py	35	14	3
def _load_rows_with_predicate() in petastorm/py_dict_reader_worker.py	34	14	5
def _load_rows_into_mem() in petastorm/pytorch.py	37	14	3
def _ventilate() in petastorm/workers_pool/ventilator.py	20	13	1
def process() in petastorm/py_dict_reader_worker.py	27	12	4
def _add_many() in petastorm/reader_impl/pytorch_shuffling_buffer.py	31	12	2
def _sanitize_pytorch_types() in petastorm/pytorch.py	18	12	1
def _iter_impl() in petastorm/pytorch.py	25	12	1
def transform_schema() in petastorm/transform.py	20	11	2
def create_schema_view() in petastorm/unischema.py	13	11	2
def make_reader() in petastorm/reader.py	78	11	28
def add_to_dataset_metadata() in petastorm/utils.py	25	11	3
def __init__() in petastorm/weighted_sampling_reader.py	21	11	3
def encode() in petastorm/codecs.py	23	11	3
def hdfs_connect_namenode() in petastorm/hdfs/namenode.py	11	10	4
def match_unischema_fields() in petastorm/unischema.py	19	10	2
def _partition_row_groups() in petastorm/reader.py	15	10	7
def _iter_impl() in petastorm/pytorch.py	28	10	1
def run() in petastorm/workers_pool/thread_pool.py	22	10	1
def copy_dataset() in petastorm/tools/copy_dataset.py	28	10	9
def get_filesystem_and_path_or_paths() in petastorm/fs_utils.py	19	9	4
def dict_to_spark_row() in petastorm/unischema.py	26	9	2
def make_batch_reader() in petastorm/reader.py	70	9	27
def convert_arrow_table_to_numpy_dict() in petastorm/arrow_reader_worker.py	30	9	2
def _load_rows() in petastorm/arrow_reader_worker.py	27	9	4
def _load_rows_with_predicate() in petastorm/arrow_reader_worker.py	38	9	5
def load_row_groups() in petastorm/etl/dataset_metadata.py	25	9	1
def read_next() in petastorm/py_dict_reader_worker.py	17	9	4
def _convert_precision() in petastorm/spark/spark_dataset_converter.py	17	9	2
def _keep_retrying_while_zmq_again() in petastorm/workers_pool/process_pool.py	17	9	3
def get_results() in petastorm/workers_pool/dummy_pool.py	14	9	1
def _read_with_shuffle_row_drop() in petastorm/arrow_reader_worker.py	24	8	5
def decode_row() in petastorm/utils.py	20	8	2
def convert_fields() in petastorm/ngram.py	8	8	3
def build_index() in petastorm/etl/rowgroup_indexers.py	13	8	3
def _split_row_groups() in petastorm/etl/dataset_metadata.py	21	8	1
def _normalize_databricks_dbfs_url() in petastorm/spark/spark_dataset_converter.py	13	8	2
def decimal_friendly_collate() in petastorm/pytorch.py	12	8	1
def _worker_bootstrap() in petastorm/workers_pool/process_pool.py	53	8	8
def get_results() in petastorm/workers_pool/thread_pool.py	27	8	1
def __init__() in petastorm/workers_pool/ventilator.py	26	8	8