mozilla / telemetry-airflow

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

26% duplication:

8,453 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
2,231 duplicated lines

435 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 435 duplicates...

Size	#	Folders	Files	Lines	Code
59	x 2	dags dags	eam_slack_channels.py eam_workday_everfi_integration.py	57:119 (55%) 57:119 (46%)	view
53	x 2	dags dags	eam_slack_channels.py eam_workday_netsuite.py	57:112 (49%) 64:119 (32%)	view
53	x 2	dags dags	eam_workday_everfi_integration.py eam_workday_netsuite.py	57:112 (41%) 64:119 (32%)	view
50	x 2	dags dags	catalyst.py jetstream.py	55:108 (59%) 83:136 (46%)	view
46	x 2	dags dags	glam_fenix.py glam_fog.py	111:166 (28%) 95:153 (29%)	view
28	x 2	utils utils	dataproc.py dataproc.py	513:541 (5%) 672:700 (5%)	view
24	x 2	dags dags	glam_fenix.py glam_fenix_release.py	171:199 (14%) 68:96 (25%)	view
20	x 2	dags dags	glam_fenix.py glam_fog.py	201:220 (12%) 191:210 (12%)	view
19	x 2	dags dags	glam_fenix.py glam_fog.py	22:47 (11%) 13:38 (12%)	view
18	x 2	utils utils	dataproc.py dataproc.py	428:445 (3%) 581:598 (3%)	view
18	x 2	dags dags	glam_glean_imports.py glam_glean_imports.py	67:86 (16%) 101:119 (16%)	view
17	x 2	dags dags	glam_fenix.py glam_fog.py	179:199 (10%) 167:187 (10%)	view
17	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	715:764 (2%) 789:830 (2%)	view
17	x 2	dags dags	glam_fenix_release.py glam_fog.py	76:96 (18%) 167:187 (10%)	view
16	x 2	utils utils	dataproc.py dataproc.py	362:377 (3%) 672:687 (3%)	view
16	x 2	dags dags	glam_fenix.py glam_fog_release.py	181:201 (9%) 91:111 (15%)	view
16	x 2	utils utils	dataproc.py dataproc.py	362:377 (3%) 513:528 (3%)	view
16	x 2	dags dags	glam_fenix_release.py glam_fog_release.py	56:74 (17%) 61:79 (15%)	view
16	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	910:945 (2%) 962:1002 (2%)	view
16	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	559:634 (2%) 788:828 (2%)	view
16	x 2	dags dags	glam_fenix.py glam_fog.py	90:108 (9%) 76:94 (10%)	view
15	x 2	dags dags	glam_fog.py glam_fog_release.py	169:187 (9%) 91:109 (14%)	view
15	x 2	dags dags	fxci_metric_export.py fxci_pulse_export.py	15:31 (27%) 18:34 (27%)	view
15	x 2	dags dags	probe_scraper.py probe_scraper.py	188:202 (5%) 259:273 (5%)	view
15	x 2	dags dags	glam_fenix_release.py glam_fog_release.py	78:96 (15%) 91:109 (14%)	view
15	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	560:634 (2%) 715:762 (2%)	view
15	x 2	dags dags	bqetl_backfill.py operational_monitoring_backfill.py	62:76 (9%) 38:52 (23%)	view
14	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	684:697 (2%) 818:831 (2%)	view
14	x 2	dags dags	glam_fenix.py glam_fog.py	77:90 (8%) 59:72 (8%)	view
13	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	684:696 (2%) 752:764 (2%)	view
13	x 2	dags dags	mad_server.py mad_server.py	102:114 (11%) 129:141 (11%)	view
13	x 2	utils utils	dataproc.py dataproc.py	449:507 (2%) 602:668 (2%)	view
13	x 2	dags dags	probe_scraper.py probe_scraper.py	148:160 (4%) 226:238 (4%)	view
13	x 2	dags dags	eam_workday_everfi_integration.py eam_workday_netsuite.py	32:55 (10%) 39:62 (7%)	view
12	x 2	utils utils	dataproc.py dataproc.py	29:40 (2%) 585:596 (2%)	view
12	x 2	dags dags	bqetl_backfill_complete.py bqetl_backfill_initiate.py	12:26 (16%) 12:26 (14%)	view
12	x 2	utils utils	dataproc.py dataproc.py	29:40 (2%) 432:443 (2%)	view
12	x 2	dags dags	bqetl_backfill_complete.py bqetl_backfill_initiate.py	28:39 (16%) 28:39 (14%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	231:241 (1%) 253:263 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	818:828 (1%) 934:944 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	624:634 (1%) 934:944 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	624:634 (1%) 866:876 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	818:828 (1%) 991:1001 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	684:694 (1%) 866:876 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	231:241 (1%) 275:285 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	684:694 (1%) 991:1001 (1%)	view
11	x 2	utils/patched utils/patched	dataproc_hook.py dataproc_hook.py	684:694 (1%) 934:944 (1%)	view
11	x 2	dags dags	probe_scraper.py probe_scraper.py	224:234 (3%) 256:266 (3%)	view
11	x 2	dags dags	glam_fenix.py glam_fog_release.py	31:42 (6%) 21:32 (10%)	view
11	x 2	utils utils	dataproc.py dataproc.py	386:399 (2%) 533:546 (2%)	view