apache / datafusion-comet - Conditional Complexity

Intro

Conditional complexity (also called cyclomatic complexity) is a term used to measure the complexity of software. The term refers to the number of possible paths through a program function. A higher value ofter means higher maintenance and testing costs (infosecinstitute.com).
Conditional complexity is calculated by counting all conditions in the program that can affect the execution path (e.g. if statement, loops, switches, and/or operators, try and catch blocks...).
Conditional complexity is measured at the unit level (methods, functions...).
Units are classified in four categories based on the measured McCabe index: 1-5 (simple units), 6-10 (medium complex units), 11-25 (complex units), 26+ (very complex units).

Learn more...

Conditional Complexity Overall

There are 2,064 units with 30,860 lines of code in units (52.4% of code).

6 very complex units (2,390 lines of code)
9 complex units (901 lines of code)
51 medium complex units (2,943 lines of code)
113 simple units (3,682 lines of code)
1,885 very simple units (20,944 lines of code)

Legend:

51+

26-50

11-25

6-10

1-5

Alternative Visuals

3D view of all units

Conditional Complexity per Extension

51+

26-50

11-25

6-10

1-5

Conditional Complexity per Logical Component

primary logical decomposition

51+

26-50

11-25

6-10

1-5

Most Complex Units

Top 50 most complex units

Unit	# lines	McCabe index	# params
def exprToProtoInternal() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	1263	282	3
private def isCometPlan() in spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala	350	118	1
def operator2Proto() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	420	116	2
def generateColumn() in spark/src/main/scala/org/apache/comet/testing/ParquetGenerator.scala	114	62	4
private def createFilterHelper() in spark/src/main/scala/org/apache/comet/parquet/ParquetFilters.scala	152	57	2
public static void checkParquetType() in common/src/main/java/org/apache/comet/parquet/TypeUtil.java	91	52	2
private def createNativeFilter() in spark/src/main/scala/org/apache/comet/parquet/ParquetFilters.scala	122	50	1
private def convertField() in common/src/main/scala/org/apache/spark/sql/comet/parquet/CometSparkToParquetSchemaConverter.scala	133	36	2
def fromArrowType() in common/src/main/scala/org/apache/spark/sql/comet/util/Utils.scala	25	35	1
def windowExprToProto() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	152	34	3
fn criterion_benchmark() in native/core/benches/bit_util.rs	138	30	1
def createValueExpr() in spark/src/main/scala/org/apache/comet/parquet/SourceFilterSerde.scala	67	29	2
def serializeDataType() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	78	28	1
override def doExecuteColumnar() in spark/src/main/scala/org/apache/spark/sql/comet/operators.scala	94	28	0
fn date_parser() in native/spark-expr/src/conversion_funcs/cast.rs	92	28	2
private[sql] def createFieldWriter() in common/src/main/scala/org/apache/spark/sql/comet/execution/arrow/ArrowWriters.scala	40	23	1
override def apply() in spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala	86	23	1
def isSupported() in spark/src/main/scala/org/apache/comet/expressions/CometCast.scala	52	23	4
def generate_changelog() in dev/release/generate-changelog.py	44	23	5
def aggExprToProto() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	36	21	4
private def valueCanMakeFilterOn() in spark/src/main/scala/org/apache/comet/parquet/ParquetFilters.scala	30	21	2
public void init() in common/src/main/java/org/apache/comet/parquet/NativeBatchReader.java	138	20	0
private void init() in common/src/main/java/org/apache/comet/parquet/ConstantColumnReader.java	42	19	1
public static ColumnDescriptor convertToParquet() in common/src/main/java/org/apache/comet/parquet/TypeUtil.java	63	18	1
public void init() in common/src/main/java/org/apache/comet/parquet/BatchReader.java	112	18	0
private ColumnPageReader readAllPages() in common/src/main/java/org/apache/comet/parquet/FileReader.java	150	18	5
private def isCometSink() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	20	17	1
private def generateCompatibilityGuide() in spark/src/main/scala/org/apache/comet/GenerateDocs.scala	46	17	0
private def fetchNext() in spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/NativeBatchDecoderIterator.scala	55	17	0
def prepareJVMShuffleDependency() in spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometShuffleExchangeExec.scala	107	17	6
fn do_cast_string_to_int<() in native/spark-expr/src/conversion_funcs/cast.rs	72	17	4
private def same() in fuzz-testing/src/main/scala/org/apache/comet/fuzz/QueryRunner.scala	15	16	2
def toArrowType() in common/src/main/scala/org/apache/spark/sql/comet/util/Utils.scala	25	16	2
def supportedShuffleDataType() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	23	16	1
private static boolean isDecimalTypeMatched() in common/src/main/java/org/apache/comet/parquet/TypeUtil.java	31	15	2
public DataPage readPage() in common/src/main/java/org/apache/comet/parquet/ColumnPageReader.java	108	15	0
def promote() in spark/src/main/scala/org/apache/spark/sql/comet/DecimalPrecision.scala	56	15	3
fn test_date_trunc_array_fmt_dyn() in native/spark-expr/src/kernels/temporal.rs	97	15	0
fn test_timestamp_trunc_array_fmt_dyn() in native/spark-expr/src/kernels/temporal.rs	129	15	0
fn array_insert() in native/spark-expr/src/array_funcs/array_insert.rs	94	15	4
private def isDynamicPruningFilter() in spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala	45	14	1
private def transformV2Scan() in spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala	52	14	1
private def canCastToString() in spark/src/main/scala/org/apache/comet/expressions/CometCast.scala	38	14	3
def shouldApplySparkToColumnar() in spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala	27	14	2
private def convertibleFiltersHelper() in spark/src/main/scala/org/apache/comet/parquet/ParquetFilters.scala	32	14	2
override protected def doProduce() in spark/src/main/scala/org/apache/spark/sql/comet/CometColumnarToRowExec.scala	78	14	1
public static boolean isEqual() in common/src/main/java/org/apache/comet/parquet/TypeUtil.java	30	13	2
public PageReadStore readNextRowGroup() in common/src/main/java/org/apache/comet/parquet/FileReader.java	35	13	0
private def canCastFromString() in spark/src/main/scala/org/apache/comet/expressions/CometCast.scala	33	13	3
def columnarShuffleSupported() in spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala	38	13	1