huggingface / pyspark_huggingface Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Duplication Overall

1,032 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
155 duplicated lines

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 21 longest duplicates.

Size	#	Folders	Files	Lines	Code
13	x 2	ROOT ROOT	demo.ipynb demo.ipynb	476:488 (2%) 522:534 (2%)	view
12	x 2	pyspark_huggingface pyspark_huggingface	huggingface_sink.py huggingface_source.py	76:87 (5%) 93:104 (15%)	view
9	x 2	ROOT ROOT	demo.ipynb demo.ipynb	126:134 (1%) 285:293 (1%)	view
9	x 2	ROOT ROOT	demo.ipynb demo.ipynb	462:470 (1%) 508:516 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	347:353 (1%) 510:516 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	454:460 (1%) 500:506 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	153:159 (1%) 311:317 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	153:159 (1%) 500:506 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	100:106 (1%) 344:350 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	311:317 (1%) 454:460 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	153:159 (1%) 454:460 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	311:317 (1%) 500:506 (1%)	view
7	x 2	ROOT ROOT	demo.ipynb demo.ipynb	347:353 (1%) 464:470 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	126:131 (1%) 401:406 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	314:319 (1%) 526:531 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	314:319 (1%) 480:485 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	285:290 (1%) 401:406 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	126:131 (1%) 221:226 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	221:226 (1%) 285:290 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	221:226 (1%) 401:406 (1%)	view
6	x 2	ROOT ROOT	demo.ipynb demo.ipynb	163:168 (1%) 318:323 (1%)	view