alibaba / TorchEasyRec

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

24% duplication:

25,212 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
6,269 duplicated lines

917 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 917 duplicates...

Size	#	Folders	Files	Lines	Code
61	x 2	tzrec/features tzrec/features	custom_feature.py lookup_feature.py	44:119 (35%) 42:117 (29%)	view
38	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_attention.py triton_hstu_attention.py	498:535 (2%) 592:629 (2%)	view
34	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_attention.py triton_hstu_attention.py	542:575 (2%) 646:679 (2%)	view
34	x 2	tzrec tzrec/tools/tdm	main.py retrieval.py	1055:1094 (3%) 183:221 (9%)	view
34	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_attention.py triton_hstu_attention.py	929:962 (2%) 964:997 (2%)	view
33	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_attention.py triton_hstu_attention.py	1297:1329 (2%) 1334:1366 (2%)	view
32	x 2	tzrec/tools/tdm tzrec/tools/tdm	cluster_tree.py init_tree.py	37:68 (34%) 37:68 (36%)	view
30	x 2	tzrec/modules tzrec/modules	embedding.py embedding.py	425:457 (2%) 924:956 (2%)	view
27	x 2	tzrec/models tzrec/models	dbmtl.py dc2vr.py	51:80 (24%) 46:75 (24%)	view
27	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	466:497 (5%) 649:680 (5%)	view
27	x 2	tzrec tzrec/tools/tdm	predict.py retrieval.py	16:42 (30%) 409:435 (7%)	view
26	x 2	tzrec/ops/triton tzrec/ops/triton	triton_layer_norm.py triton_layer_norm.py	37:65 (3%) 83:112 (3%)	view
25	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	273:303 (4%) 653:683 (4%)	view
25	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_linear.py triton_hstu_linear.py	207:231 (2%) 717:741 (2%)	view
25	x 2	tzrec/modules tzrec/modules	hstu.py hstu.py	188:212 (7%) 432:456 (7%)	view
25	x 2	tzrec/protos tzrec/protos	proto feature.proto feature.proto	242:268 (3%) 322:348 (3%)	view
24	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	376:405 (4%) 553:582 (4%)	view
23	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_linear.py triton_layer_norm.py	283:308 (1%) 267:292 (3%)	view
23	x 2	tzrec/ops/triton tzrec/ops/triton	triton_jagged_tensors.py triton_jagged_tensors.py	62:84 (3%) 120:142 (3%)	view
23	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	273:300 (4%) 470:497 (4%)	view
22	x 2	tzrec/features tzrec/features	custom_feature.py lookup_feature.py	123:154 (12%) 121:152 (10%)	view
22	x 2	tzrec/datasets tzrec/datasets	sampler.py sampler.py	716:737 (3%) 907:928 (3%)	view
21	x 2	tzrec/datasets tzrec/datasets	sampler.py sampler.py	436:456 (3%) 625:645 (3%)	view
21	x 2	tzrec/models tzrec/models	dbmtl.py dc2vr.py	111:143 (19%) 110:142 (19%)	view
20	x 2	tzrec/datasets tzrec/datasets	sampler.py sampler.py	526:545 (2%) 622:641 (2%)	view
20	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	421:449 (3%) 601:629 (3%)	view
19	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	193:214 (3%) 553:574 (3%)	view
19	x 2	tzrec/datasets tzrec/datasets	data_parser.py data_parser.py	468:486 (3%) 608:626 (3%)	view
19	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	193:214 (3%) 376:397 (3%)	view
19	x 2	tzrec/features tzrec/features	custom_feature.py overlap_feature.py	123:151 (11%) 72:100 (24%)	view
19	x 2	tzrec/ops tzrec/ops/triton	hstu_compute.py triton_hstu_preprocess_and_attention.py	133:151 (10%) 41:59 (6%)	view
19	x 2	tzrec/features tzrec/features	lookup_feature.py overlap_feature.py	121:149 (9%) 72:100 (24%)	view
19	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_linear.py triton_hstu_linear.py	110:128 (1%) 627:645 (1%)	view
18	x 2	tzrec/features tzrec/features	lookup_feature.py match_feature.py	50:75 (8%) 52:77 (10%)	view
18	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_linear.py triton_hstu_linear.py	417:434 (1%) 907:924 (1%)	view
18	x 2	tzrec/models tzrec/models	multi_tower.py multi_tower_din.py	35:53 (48%) 36:54 (33%)	view
18	x 2	tzrec/features tzrec/features	custom_feature.py match_feature.py	52:77 (10%) 52:77 (10%)	view
17	x 2	tzrec/datasets tzrec/datasets	sampler.py sampler.py	436:452 (2%) 529:545 (2%)	view
17	x 2	tzrec/features tzrec/features	custom_feature.py id_feature.py	93:111 (10%) 75:93 (12%)	view
17	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_linear.py triton_layer_norm.py	261:281 (1%) 245:265 (2%)	view
17	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	175:191 (3%) 535:551 (3%)	view
17	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	358:374 (3%) 535:551 (3%)	view
17	x 2	tzrec tzrec	main.py main.py	446:462 (1%) 468:484 (1%)	view
17	x 2	tzrec/features tzrec/features	expr_feature.py overlap_feature.py	40:63 (23%) 40:63 (21%)	view
17	x 2	tzrec/features tzrec/features	sequence_feature.py sequence_feature.py	175:191 (3%) 358:374 (3%)	view
17	x 2	tzrec/ops/triton tzrec/ops/triton	triton_hstu_attention.py triton_hstu_attention.py	414:430 (1%) 449:465 (1%)	view
17	x 2	tzrec tzrec/tools/tdm	predict.py retrieval.py	59:75 (19%) 440:456 (4%)	view
17	x 2	tzrec/features tzrec/features	id_feature.py lookup_feature.py	75:93 (12%) 91:109 (8%)	view
16	x 2	tzrec tzrec	main.py main.py	469:484 (1%) 505:520 (1%)	view
16	x 2	tzrec/features tzrec/features	combo_feature.py id_feature.py	122:137 (17%) 164:179 (11%)	view