huggingface / setfit Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

33% duplication:

11,354 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
3,789 duplicated lines

919 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
scripts (<1%) --> src (<1%)	52	4 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 919 duplicates...

Size	#	Folders	Files	Lines	Code
52	x 2	scripts/adapet/ADAPET/src/data scripts/adapet/ADAPET/src/data	CBReader.py RTEReader.py	90:166 (41%) 93:170 (33%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	872:922 (3%) 1170:1220 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 1170:1220 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1042:1092 (3%) 1094:1144 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	872:922 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 1094:1144 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 1042:1092 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 1170:1220 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	924:974 (3%) 1170:1220 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 1094:1144 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	924:974 (3%) 1042:1092 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 1042:1092 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	872:922 (3%) 1094:1144 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	872:922 (3%) 1042:1092 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	924:974 (3%) 1094:1144 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1170:1220 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 872:922 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 924:974 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	657:707 (3%) 709:759 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	924:974 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 924:974 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	872:922 (3%) 924:974 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1042:1092 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1094:1144 (3%) 1170:1220 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	709:759 (3%) 872:922 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1094:1144 (3%) 1222:1272 (3%)	view
51	x 2	notebooks notebooks	multilabel_HoC.ipynb multilabel_HoC.ipynb	1042:1092 (3%) 1170:1220 (3%)	view
38	x 2	notebooks notebooks	text-classification.ipynb text-classification_multilabel.ipynb	41:78 (4%) 39:76 (4%)	view
31	x 2	scripts/transformers scripts/transformers	run_fewshot.py run_fewshot_multilingual.py	92:128 (25%) 117:153 (22%)	view
29	x 2	notebooks notebooks	text-classification.ipynb text-classification_multilabel.ipynb	741:769 (3%) 869:897 (3%)	view
26	x 2	notebooks notebooks	text-classification.ipynb text-classification_multilabel.ipynb	462:487 (3%) 544:569 (2%)	view
25	x 2	notebooks notebooks	text-classification.ipynb text-classification_multilabel.ipynb	359:383 (3%) 441:465 (2%)	view
25	x 2	scripts/transformers scripts/transformers	run_fewshot.py run_fewshot_multilingual.py	55:90 (20%) 80:115 (18%)	view
23	x 2	notebooks notebooks	text-classification.ipynb text-classification_multilabel.ipynb	287:309 (2%) 368:390 (2%)	view
22	x 2	scripts/adapet/ADAPET/src/data scripts/adapet/ADAPET/src/data	CBReader.py RTEReader.py	60:88 (17%) 61:88 (14%)	view
22	x 2	scripts/transformers scripts/transformers	run_full.py run_full_multilingual.py	91:115 (19%) 110:134 (17%)	view
22	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_zeroshot.py	64:93 (16%) 59:88 (17%)	view
22	x 2	notebooks notebooks	openvino_inference.ipynb text-classification.ipynb	17:38 (9%) 17:38 (2%)	view
21	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_zeroshot.py	129:151 (16%) 126:148 (16%)	view
21	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_zeroshot.py	154:177 (16%) 149:172 (16%)	view
21	x 2	notebooks notebooks	onnx_model_export.ipynb text-classification_multilabel.ipynb	207:227 (9%) 866:886 (2%)	view
21	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_zeroshot.py	36:56 (16%) 35:55 (16%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot_multilingual.py run_fewshot_multilingual.py	182:202 (9%) 252:272 (9%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot_multilingual.py run_fewshot_multilingual.py	215:235 (9%) 252:272 (9%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot_multilingual.py run_fewshot_multilingual.py	182:202 (9%) 215:235 (9%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot_multilingual.py run_zeroshot.py	36:54 (9%) 37:55 (15%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_fewshot_distillation.py	40:58 (14%) 53:71 (7%)	view
19	x 2	scripts/setfit scripts/setfit	run_fewshot.py run_fewshot_multilingual.py	38:56 (14%) 36:54 (9%)	view