huggingface / autotrain-advanced Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

44% duplication:

14,855 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
6,627 duplicated lines

1,552 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
colabs (25%) --> notebooks (22%)	162	6 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 1,552 duplicates...

Size	#	Folders	Files	Lines	Code
78	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/image_regression	__main__.py __main__.py	111:197 (39%) 97:183 (41%)	view
68	x 2	src/autotrain src/autotrain	commands.py commands.py	359:427 (15%) 435:504 (15%)	view
66	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/text_classification	__main__.py __main__.py	139:212 (33%) 134:207 (33%)	view
53	x 2	src/autotrain/trainers/text_regression src/autotrain/trainers/token_classification	__main__.py __main__.py	126:184 (28%) 132:190 (27%)	view
53	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/token_classification	__main__.py __main__.py	139:197 (26%) 132:190 (27%)	view
53	x 2	src/autotrain/trainers/image_regression src/autotrain/trainers/text_regression	__main__.py __main__.py	125:183 (28%) 126:184 (28%)	view
53	x 2	src/autotrain/trainers/text_classification src/autotrain/trainers/token_classification	__main__.py __main__.py	134:192 (27%) 132:190 (27%)	view
53	x 2	src/autotrain/trainers/image_regression src/autotrain/trainers/text_classification	__main__.py __main__.py	125:183 (28%) 134:192 (27%)	view
53	x 2	src/autotrain/trainers/image_regression src/autotrain/trainers/token_classification	__main__.py __main__.py	125:183 (28%) 132:190 (27%)	view
53	x 2	src/autotrain/trainers/text_classification src/autotrain/trainers/text_regression	__main__.py __main__.py	134:192 (27%) 126:184 (28%)	view
53	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/text_regression	__main__.py __main__.py	139:197 (26%) 126:184 (28%)	view
52	x 2	src/autotrain/trainers/clm src/autotrain/trainers/vlm	utils.py utils.py	654:760 (9%) 127:183 (23%)	view
49	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/text_classification	utils.py utils.py	10:122 (47%) 8:115 (60%)	view
48	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/image_regression	__main__.py __main__.py	148:200 (22%) 125:177 (25%)	view
48	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/text_regression	__main__.py __main__.py	148:200 (22%) 126:178 (25%)	view
48	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/image_classification	__main__.py __main__.py	148:200 (22%) 139:191 (24%)	view
48	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/text_classification	__main__.py __main__.py	148:200 (22%) 134:186 (24%)	view
48	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/token_classification	__main__.py __main__.py	148:200 (22%) 132:184 (25%)	view
45	x 2	notebooks notebooks	text_classification.ipynb text_regression.ipynb	74:118 (38%) 74:118 (38%)	view
44	x 2	src/autotrain/trainers/text_classification src/autotrain/trainers/token_classification	__main__.py __main__.py	45:90 (22%) 46:91 (23%)	view
44	x 2	src/autotrain/trainers/seq2seq src/autotrain/trainers/text_regression	__main__.py __main__.py	50:95 (20%) 45:90 (23%)	view
44	x 2	src/autotrain/trainers/sent_transformers src/autotrain/trainers/text_classification	__main__.py __main__.py	42:87 (21%) 45:90 (22%)	view
44	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/text_regression	__main__.py __main__.py	47:92 (20%) 45:90 (23%)	view
44	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/sent_transformers	__main__.py __main__.py	47:92 (20%) 42:87 (21%)	view
44	x 2	src/autotrain/trainers/text_classification src/autotrain/trainers/text_regression	__main__.py __main__.py	45:90 (22%) 45:90 (23%)	view
44	x 2	src/autotrain/trainers/seq2seq src/autotrain/trainers/text_classification	__main__.py __main__.py	50:95 (20%) 45:90 (22%)	view
44	x 2	src/autotrain/trainers/text_regression src/autotrain/trainers/token_classification	__main__.py __main__.py	45:90 (23%) 46:91 (23%)	view
44	x 2	src/autotrain/trainers/seq2seq src/autotrain/trainers/token_classification	__main__.py __main__.py	50:95 (20%) 46:91 (23%)	view
44	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/text_classification	__main__.py __main__.py	47:92 (20%) 45:90 (22%)	view
44	x 2	src/autotrain/trainers/sent_transformers src/autotrain/trainers/text_regression	__main__.py __main__.py	42:87 (21%) 45:90 (23%)	view
44	x 2	src/autotrain/trainers/sent_transformers src/autotrain/trainers/seq2seq	__main__.py __main__.py	42:87 (21%) 50:95 (20%)	view
44	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/token_classification	__main__.py __main__.py	47:92 (20%) 46:91 (23%)	view
44	x 2	src/autotrain/trainers/e...tive_question_answering src/autotrain/trainers/seq2seq	__main__.py __main__.py	47:92 (20%) 50:95 (20%)	view
44	x 2	src/autotrain/trainers/sent_transformers src/autotrain/trainers/token_classification	__main__.py __main__.py	42:87 (21%) 46:91 (23%)	view
43	x 2	src/autotrain/trainers/image_regression src/autotrain/trainers/object_detection	__main__.py __main__.py	121:168 (23%) 124:171 (22%)	view
43	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/object_detection	__main__.py __main__.py	135:182 (21%) 124:171 (22%)	view
42	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/object_detection	__main__.py __main__.py	44:87 (21%) 45:88 (21%)	view
42	x 2	src/autotrain/trainers/image_regression src/autotrain/trainers/object_detection	__main__.py __main__.py	44:87 (22%) 45:88 (21%)	view
42	x 2	src/autotrain/trainers/image_classification src/autotrain/trainers/image_regression	__main__.py __main__.py	44:87 (21%) 44:87 (22%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	110:155 (6%) 420:465 (6%)	view
41	x 2	notebooks notebooks	llm_finetuning.ipynb text_classification.ipynb	80:120 (34%) 78:118 (34%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	320:365 (6%) 420:465 (6%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	209:254 (6%) 420:465 (6%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	209:254 (6%) 320:365 (6%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	110:155 (6%) 320:365 (6%)	view
41	x 2	src/autotrain src/autotrain	dataset.py dataset.py	110:155 (6%) 209:254 (6%)	view
41	x 2	notebooks notebooks	llm_finetuning.ipynb text_regression.ipynb	80:120 (34%) 78:118 (34%)	view
41	x 2	src/autotrain/preprocessor src/autotrain/preprocessor	vision.py vlm.py	438:495 (11%) 71:128 (29%)	view
40	x 2	src/autotrain/trainers/object_detection src/autotrain/trainers/text_classification	__main__.py __main__.py	128:171 (20%) 134:177 (20%)	view
40	x 2	src/autotrain/trainers/object_detection src/autotrain/trainers/token_classification	__main__.py __main__.py	128:171 (20%) 132:175 (20%)	view