• There are 1,929 files with units, with 306,508 lines of code.
• 21 very complex files (McCabe index sum 251+; 31,882 lines of code)
• 54 complex files (McCabe index sum 126-250; 42,225 lines of code)
• 183 medium complex files (McCabe index sum 51-125; 70,399 lines of code)
• 327 slightly complex files (McCabe index sum 26-50; 68,090 lines of code)
• 1,344 simple files (McCabe index sum 1-25; 93,912 lines of code)
• 438 main files have no units (e.g. languages without unit analysis, or files without functions) and are not classified.

The McCabe index of a file grows with its size: per 100 lines tells a large but plain file from a dense one, and most complex unit whether the complexity sits in one unit or is spread over many.

File	# lines	# units	McCabe index (sum)	per 100 lines	most complex unit
Analyzer.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis	2003	109	750	37	32
tests.py in python/pyspark/sql	5232	391	709	14	16
AstBuilder.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser	1809	171	523	29	25
Utils.scala in core/src/main/scala/org/apache/spark/util	1974	149	481	24	16
collectionOperations.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions	3429	119	414	12	11
Cast.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions	1166	36	397	34	105
types.py in python/pyspark/sql	1015	119	381	38	35
TypeCoercion.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis	644	38	374	58	67
HiveInspectors.scala in sql/hive/src/main/scala/org/apache/spark/sql/hive	767	20	334	44	81
Optimizer.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer	1019	58	309	30	36
tests.py in python/pyspark/ml	2180	159	306	14	14
rdd.py in python/pyspark	971	131	301	31	34
DAGScheduler.scala in core/src/main/scala/org/apache/spark/scheduler	1371	71	298	22	65
ScalaReflection.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst	711	23	298	42	77
dataframe.py in python/pyspark/sql	849	103	295	35	27
tests.py in python/pyspark	1903	162	293	15	11
expressions.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer	456	24	284	62	63
__init__.py in python/pyspark/mllib/linalg	742	94	271	37	14
functions.scala in sql/core/src/main/scala/org/apache/spark/sql	1008	249	269	27	4
CodeGenerator.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen	965	61	256	27	29
stringExpressions.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions	1668	92	251	15	20
DateTimeUtils.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util	773	66	244	32	45
UTF8String.java in common/unsafe/src/main/java/org/apache/spark/unsafe/types	951	65	243	26	15
feature.py in python/pyspark/ml	1370	229	237	17	4
tests.py in python/pyspark/streaming	1223	104	235	19	13
__init__.py in python/pyspark/ml/linalg	638	79	232	36	13
SparkSubmit.scala in core/src/main/scala/org/apache/spark/deploy	917	31	222	24	138
BlockManager.scala in core/src/main/scala/org/apache/spark/storage	1172	60	219	19	18
CheckAnalysis.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis	558	8	210	38	147
SparkContext.scala in core/src/main/scala/org/apache/spark	1532	104	207	14	16
Dataset.scala in sql/core/src/main/scala/org/apache/spark/sql	1216	116	203	17	14
functions.py in python/pyspark/sql	936	140	194	21	13
TaskSetManager.scala in core/src/main/scala/org/apache/spark/scheduler	735	35	192	26	24
JdbcUtils.scala in sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc	638	22	184	29	50
tests.py in python/pyspark/mllib	1373	90	182	13	7
objects.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects	1270	46	181	14	35
Client.scala in resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn	1015	35	179	18	22
SessionCatalog.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog	896	85	177	20	8
RDD.scala in core/src/main/scala/org/apache/spark/rdd	981	94	172	18	7
SparkStrategies.scala in sql/core/src/main/scala/org/apache/spark/sql/execution	420	25	171	41	42
LogisticRegression.scala in mllib/src/main/scala/org/apache/spark/ml/classification	968	31	168	17	79
literals.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions	274	14	167	61	41
serializers.py in python/pyspark	444	76	165	37	15
TreeNode.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees	447	25	163	36	35
JsonProtocol.scala in core/src/main/scala/org/apache/spark/util	883	76	162	18	20
JavaTypeInference.scala in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst	353	12	162	46	55
HiveExternalCatalog.scala in sql/hive/src/main/scala/org/apache/spark/sql/hive	923	66	162	18	13
classification.py in python/pyspark/ml	860	117	158	18	9
cloudpickle.py in python/pyspark	647	58	156	24	11
SortMergeJoinExec.scala in sql/core/src/main/scala/org/apache/spark/sql/execution/joins	836	30	150	18	32

• File complexity is the sum of the McCabe indexes (conditional complexity) of the units (methods, functions...) in a file.
• Files are classified in five categories based on that sum: 1-25 (simple files), 26-50 (slightly complex files), 51-125 (medium complex files), 126-250 (complex files), 251+ (very complex files). The default bands are five times the unit complexity bands, as a file is a group of units.
• Only main code files with units are classified. Files without units (languages without unit analysis, or files without functions) have no complexity to measure and are left out rather than counted as simple.
• A very complex file holds many decisions, which often means it has too many responsibilities: a candidate to split, and a place where changes need more testing.