apache / cloudberry-pxf

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

10% duplication:

71,514 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
7,736 duplicated lines

617 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
external-table (31%) --> fdw (30%)	1854	6 file pairs	details...
automation (<1%) --> server (<1%)	208	10 file pairs	details...
concourse (2%) --> dev (10%)	170	3 file pairs	details...
automation (<1%) --> regression (7%)	83	2 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 617 duplicates...

Size	#	Folders	Files	Lines	Code
253	x 2	automation/tinc/main/ext/modgrammar automation/tinc/main/ext/modgrammar	grammar_py2.py grammar_py3.py	14:430 (95%) 15:431 (90%)	view
92	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	55:176 (12%) 57:178 (10%)	view
88	x 2	external-table/src fdw	libchurl.c libchurl.c	142:305 (13%) 168:333 (15%)	view
82	x 2	dev dev	configure_singlecluster.bash init_hadoop.bash	67:148 (45%) 67:148 (43%)	view
62	x 2	external-table/src fdw	libchurl.c libchurl.c	410:535 (9%) 400:525 (10%)	view
57	x 2	dev dev	configure_singlecluster.bash init_hadoop.bash	4:64 (31%) 4:64 (30%)	view
55	x 2	package package	spec pxf-gp5.spec pxf-gp6.spec	8:75 (91%) 11:78 (88%)	view
46	x 2	external-table/src fdw	pxffilters.h pxf_filter.h	40:145 (95%) 40:145 (95%)	view
41	x 2	external-table/src fdw	libchurl.c libchurl.c	1120:1211 (6%) 1030:1121 (7%)	view
39	x 2	dev dev	configure_singlecluster.bash init_hadoop.bash	151:189 (21%) 151:189 (20%)	view
35	x 2	dev dev	configure_singlecluster.bash init_hadoop.bash	114:148 (19%) 30:64 (18%)	view
35	x 2	dev dev	init_hadoop.bash init_hadoop.bash	30:64 (18%) 114:148 (18%)	view
35	x 2	dev dev	configure_singlecluster.bash configure_singlecluster.bash	30:64 (19%) 114:148 (19%)	view
35	x 2	dev dev	configure_singlecluster.bash init_hadoop.bash	30:64 (19%) 114:148 (18%)	view
31	x 2	package package	spec pxf-gp6.spec pxf-gp7.spec	41:78 (50%) 40:77 (50%)	view
31	x 2	external-table/src fdw	libchurl.c libchurl.c	802:848 (4%) 785:832 (5%)	view
31	x 2	package package	spec pxf-gp5.spec pxf-gp7.spec	38:75 (51%) 40:77 (50%)	view
30	x 2	concourse/terraform/ipa-multinode-hadoop concourse/terraform/ipa-multinode-hadoop	main.tf main.tf	80:116 (18%) 146:182 (18%)	view
29	x 2	package package	spec pxf-cbdb1.spec pxf-gp7.spec	45:79 (45%) 43:77 (46%)	view
29	x 2	package package	spec pxf-cbdb1.spec pxf-gp6.spec	45:79 (45%) 44:78 (46%)	view
29	x 2	package package	spec pxf-cbdb1.spec pxf-gp5.spec	45:79 (45%) 41:75 (48%)	view
27	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	1135:1170 (3%) 1229:1264 (3%)	view
26	x 2	concourse/scripts dev	pxf_common.bash configure_singlecluster.bash	344:369 (4%) 121:146 (14%)	view
26	x 2	automation/tincrepo/main...orc_primitive_types/sql automation/tincrepo/main..._types_no_timestamp/sql	query01.sql query01.sql	22:67 (44%) 18:63 (45%)	view
26	x 2	concourse/scripts dev	pxf_common.bash init_hadoop.bash	344:369 (4%) 37:62 (13%)	view
26	x 2	concourse/scripts dev	pxf_common.bash init_hadoop.bash	344:369 (4%) 121:146 (13%)	view
26	x 2	concourse/scripts dev	pxf_common.bash configure_singlecluster.bash	344:369 (4%) 37:62 (14%)	view
24	x 2	automation/src/main/java...automation/datapreparer automation/src/main/java...f/automation/dataschema	CustomAvroRecordPreparer.java CustomWritable.java	202:247 (10%) 190:235 (9%)	view
24	x 2	package package	spec pxf-gp6.spec pxf-gp7.spec	11:40 (38%) 8:37 (38%)	view
24	x 2	package package	spec pxf-gp5.spec pxf-gp7.spec	8:37 (40%) 8:37 (38%)	view
23	x 2	automation/tincrepo/main...orc_primitive_types/sql automation/tincrepo/main..._types_no_timestamp/sql	query01.sql query01.sql	73:121 (38%) 69:117 (40%)	view
22	x 2	regression/sql regression/sql	FilterPushDownTest.sql FilterPushDownTest.sql	12:62 (33%) 65:112 (33%)	view
21	x 2	automation/tincrepo/main...s/ccl/connectivity/odbc automation/tincrepo/main...s/ccl/connectivity/odbc	TestOdbcSelectSQL.c TestOdbcTransactionCommit.c	29:55 (55%) 38:64 (47%)	view
21	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	1258:1292 (2%) 1374:1408 (2%)	view
21	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	234:259 (2%) 244:268 (2%)	view
20	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	740:781 (2%) 817:859 (2%)	view
20	x 2	automation/tincrepo/main...s/ccl/connectivity/jdbc automation/tincrepo/main...s/ccl/connectivity/jdbc	TestJDBCTransactionCommit.java TestJDBCTransactionRollback.java	9:34 (39%) 9:34 (39%)	view
19	x 2	automation/tinc/main/ext...tils/gppylib/operations automation/tinc/main/ext...tils/gppylib/operations	restore.py restore.py	823:841 (2%) 858:876 (2%)	view
19	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	1393:1419 (2%) 1529:1557 (2%)	view
19	x 2	external-table external-table	pxf--1.0.sql pxf--2.0.sql	5:28 (100%) 5:28 (100%)	view
19	x 2	automation/tincrepo/main...s/ccl/connectivity/odbc automation/tincrepo/main...s/ccl/connectivity/odbc	TestOdbcTransactionCommit.c TestOdbcTransactionRollback.c	5:29 (43%) 5:29 (43%)	view
19	x 2	automation/src/main/java...f/automation/testplugin server/pxf-diagnostic/sr...reenplum/pxf/diagnostic	UserDataVerifyAccessor.java UserDataVerifyAccessor.java	21:55 (61%) 23:54 (59%)	view
18	x 2	external-table/src fdw	libchurl.c libchurl.c	374:397 (2%) 362:385 (3%)	view
18	x 2	regression/sql regression/sql	FDW_FilterPushDownTest.sql FDW_FilterPushDownTest.sql	22:55 (23%) 80:113 (23%)	view
17	x 2	regression/sql regression/sql	FDW_HiveSmokeTest.sql HiveSmokeTest.sql	3:19 (45%) 2:18 (51%)	view
17	x 2	automation/tincrepo/main...checkFilterPushDown/sql regression/sql	query01.sql FilterPushDownTest.sql	2:34 (100%) 68:99 (25%)	view
17	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	1347:1379 (2%) 1480:1513 (2%)	view
17	x 2	automation/tincrepo/main...checkFilterPushDown/sql regression/sql	query01.sql FilterPushDownTest.sql	2:34 (100%) 15:46 (25%)	view
17	x 2	external-table/src fdw	pxffilters.c pxf_filter.c	974:1011 (2%) 1057:1094 (2%)	view
17	x 2	automation/tincrepo/main...s/ccl/connectivity/jdbc automation/tincrepo/main...s/ccl/connectivity/jdbc	TestJDBCSelectSQL.java TestJDBCTransactionCommit.java	31:53 (38%) 39:61 (33%)	view

Duplicated Units

The list of top 8 duplicated units.

See data for all 8 unit duplicates...

Size	#	Folders	Files	Lines	Code
107	x 2	automation/tinc/main/ext/modgrammar automation/tinc/main/ext/modgrammar	grammar_py3.py grammar_py2.py	0:0 0:0	view
9	x 2	automation/src/main/java...f/automation/testplugin server/pxf-diagnostic/sr...reenplum/pxf/diagnostic	FilterVerifyFragmenter.java FilterVerifyFragmenter.java	24:40 25:41	view
9	x 2	automation/tinc/main/ext/suds/umx automation/tinc/main/ext/suds/mx	__init__.py __init__.py	0:0 0:0	view
8	x 2	automation/src/main/java...f/automation/testplugin server/pxf-diagnostic/sr...reenplum/pxf/diagnostic	UserDataVerifyAccessor.java UserDataVerifyAccessor.java	33:48 32:47	view
7	x 2	automation/src/main/java...f/automation/dataschema automation/src/main/java...f/automation/dataschema	CustomWritableWithChar.java CustomWritableWithCircle.java	65:74 68:77	view
7	x 2	automation/src/main/java...f/automation/testplugin automation/src/main/java...f/automation/testplugin	HiveDataFragmenterWithFilter.java HiveInputFormatFragmenterWithFilter.java	21:33 21:33	view
7	x 2	automation/src/main/java...f/automation/testplugin server/pxf-hbase/src/mai...nplum/pxf/plugins/hbase	HBaseAccessorWithFilter.java HBaseAccessor.java	185:196 234:246	view
6	x 2	automation/src/main/java...n/structures/tables/pxf automation/src/main/java...structures/tables/basic	ExternalTable.java Table.java	65:75 127:137	view