huggingface / chug

Duplication

Places in code with 6 or more lines that are exactly the same.

Longest Duplicates

The list of 36 longest duplicates.

See data for all 36 duplicates...

Size	#	Folders	Files	Lines	Code
21	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_doc_read.py pipeline_doc_vqa.py	39:62 (47%) 99:122 (28%)	view
18	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_gtparse.py pipeline_image_text.py	72:93 (26%) 70:90 (27%)	view
17	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_gtparse.py pipeline_image_text.py	47:65 (25%) 45:63 (26%)	view
14	x 2	src/chug/image src/chug/image	build_transforms_doc.py build_transforms_doc.py	15:33 (5%) 53:70 (5%)	view
12	x 2	src/chug/wds src/chug/wds	filters.py filters.py	76:87 (11%) 112:123 (11%)	view
12	x 2	src/chug/doc src/chug/task_pipeline	__init__.py pipeline_doc_vqa.py	2:13 (100%) 11:22 (16%)	view
12	x 2	src/chug/image src/chug/image	build_transforms_doc.py build_transforms_doc.py	35:49 (4%) 160:174 (4%)	view
11	x 2	src/chug/image src/chug/image	transforms_alb.py transforms_alb.py	37:49 (13%) 57:69 (13%)	view
10	x 2	src/chug/image src/chug/image	transforms_torch.py transforms_torch.py	50:60 (9%) 72:82 (9%)	view
10	x 2	src/chug/doc src/chug/doc	doc_read_processor.py doc_vqa_processor.py	21:30 (8%) 25:34 (11%)	view
9	x 2	src/chug/wds src/chug/wds	filters.py filters.py	56:64 (8%) 112:120 (8%)	view
9	x 2	src/chug/hfds src/chug/wds	loader.py loader.py	39:47 (7%) 22:30 (10%)	view
9	x 2	src/chug/wds src/chug/wds	filters.py filters.py	56:64 (8%) 76:84 (8%)	view
8	x 2	src/chug src/chug	loader.py loader.py	38:45 (6%) 49:56 (6%)	view
7	x 2	src/chug/image src/chug/image	build_transforms_image.py build_transforms_image.py	17:23 (4%) 87:93 (4%)	view
7	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_doc_vqa.py pipeline_gtparse.py	116:122 (9%) 86:93 (10%)	view
7	x 2	src/chug/doc src/chug/doc	doc_processor.py doc_processor.py	124:130 (4%) 147:153 (4%)	view
7	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_doc_read.py pipeline_gtparse.py	56:62 (15%) 86:93 (10%)	view
7	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_gtparse.py pipeline_image_text.py	26:32 (10%) 23:29 (10%)	view
7	x 2	src/chug/doc src/chug/doc	doc_processor.py doc_read_processor.py	70:76 (4%) 20:26 (5%)	view
7	x 2	src/chug/wds src/chug/wds	filters.py filters.py	58:64 (6%) 94:100 (6%)	view
7	x 2	src/chug/wds src/chug/wds	filters.py filters.py	94:100 (6%) 114:120 (6%)	view
7	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_doc_read.py pipeline_image_text.py	56:62 (15%) 83:90 (10%)	view
7	x 2	src/chug/image src/chug/image	build_transforms_image.py build_transforms_image.py	38:46 (4%) 109:117 (4%)	view
7	x 2	src/chug/doc src/chug/doc	doc_processor.py doc_read_processor.py	63:69 (4%) 12:18 (5%)	view
7	x 2	src/chug/image src/chug/image	build_transforms_doc.py build_transforms_doc.py	15:22 (2%) 178:188 (2%)	view
7	x 2	src/chug/image src/chug/image	build_transforms_doc.py build_transforms_doc.py	53:60 (2%) 178:188 (2%)	view
7	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_doc_vqa.py pipeline_image_text.py	116:122 (9%) 83:90 (10%)	view
7	x 2	src/chug src/chug	loader.py loader.py	10:16 (5%) 134:140 (5%)	view
7	x 2	src/chug/wds src/chug/wds	filters.py filters.py	78:84 (6%) 94:100 (6%)	view
6	x 2	src/chug/doc src/chug/doc	doc_processor.py doc_vqa_processor.py	71:76 (3%) 25:30 (7%)	view
6	x 2	src/chug/text src/chug/text	tokenization.py tokenization.py	68:73 (6%) 104:109 (6%)	view
6	x 2	src/chug/common src/chug/common	config.py config.py	68:73 (2%) 78:83 (2%)	view
6	x 2	src/chug/task_pipeline src/chug/task_pipeline	pipeline_gtparse.py pipeline_image_text.py	36:45 (8%) 33:43 (9%)	view
6	x 2	src/chug/hfds src/chug/hfds	loader.py loader.py	99:105 (4%) 152:158 (4%)	view
6	x 2	src/chug/wds src/chug/wds	decode.py decode.py	69:74 (2%) 146:151 (2%)	view