huggingface / bench_cluster Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

21% duplication:

1,458 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
320 duplicated lines

35 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 35 longest duplicates.

See data for all 35 duplicates...

Size	#	Folders	Files	Lines	Code
19	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py broadcast.py	41:63 (24%) 40:62 (24%)	view
16	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py p2p.py	104:120 (17%) 107:123 (16%)	view
16	x 2	bench_cluster/communication bench_cluster/communication	broadcast.py p2p.py	85:101 (20%) 107:123 (16%)	view
16	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py broadcast.py	104:120 (17%) 85:101 (20%)	view
15	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_reduce.py	50:69 (16%) 44:61 (19%)	view
15	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py broadcast.py	50:69 (16%) 43:60 (19%)	view
13	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_reduce.py	108:121 (14%) 88:101 (16%)	view
13	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py broadcast.py	90:103 (16%) 88:101 (16%)	view
13	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py p2p.py	90:103 (16%) 110:123 (13%)	view
13	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py all_to_all.py	90:103 (16%) 107:120 (13%)	view
12	x 2	bench_cluster/communication bench_cluster/communication	p2p.py p2p.py	14:25 (12%) 31:42 (12%)	view
11	x 2	bench_cluster bench_cluster	create_configs.py create_configs.py	193:206 (4%) 292:305 (4%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py broadcast.py	110:121 (12%) 88:99 (14%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py broadcast.py	42:55 (11%) 43:56 (14%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py p2p.py	110:121 (12%) 110:121 (11%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_to_all.py	110:121 (12%) 107:118 (11%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_to_all.py	50:64 (12%) 42:55 (11%)	view
11	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py all_to_all.py	44:57 (14%) 42:55 (11%)	view
10	x 2	bench_cluster/communication bench_cluster/communication	broadcast.py p2p.py	59:68 (12%) 81:90 (10%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py all_to_all.py	78:85 (10%) 81:88 (8%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_to_all.py	97:105 (8%) 81:88 (8%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_reduce.py	97:105 (8%) 78:85 (10%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py p2p.py	72:79 (8%) 83:90 (8%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py all_to_all.py	72:79 (8%) 60:67 (8%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py p2p.py	60:67 (8%) 83:90 (8%)	view
8	x 2	bench_cluster bench_cluster	create_configs.py create_configs.py	159:167 (3%) 228:236 (3%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py broadcast.py	60:67 (8%) 61:68 (10%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_gather.py broadcast.py	72:79 (8%) 61:68 (10%)	view
8	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py broadcast.py	75:82 (10%) 74:81 (10%)	view
7	x 2	bench_cluster bench_cluster	create_configs.py create_configs.py	149:155 (3%) 219:225 (3%)	view
6	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py broadcast.py	18:25 (7%) 18:25 (7%)	view
6	x 2	bench_cluster bench_cluster	create_configs.py create_configs.py	170:176 (2%) 252:258 (2%)	view
6	x 2	bench_cluster/communication bench_cluster/communication	broadcast.py p2p.py	77:82 (7%) 99:104 (6%)	view
6	x 2	bench_cluster/communication bench_cluster/communication	all_reduce.py all_to_all.py	18:25 (7%) 18:25 (6%)	view
6	x 2	bench_cluster/communication bench_cluster/communication	all_to_all.py broadcast.py	18:25 (6%) 18:25 (7%)	view