azure / gpt-rag-ingestion

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

16% duplication:

3,974 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
648 duplicated lines

55 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 55 duplicates...

Size	#	Folders	Files	Lines	Code
55	x 2	tools tools	doc_intelligence.py doc_intelligence.py	176:242 (18%) 325:391 (18%)	view
23	x 2	ROOT ROOT	function_app.py run_sharepoint.py	21:44 (14%) 45:68 (54%)	view
20	x 2	ROOT ROOT	setup.py setup.py	569:588 (1%) 704:723 (1%)	view
16	x 2	ROOT ROOT	setup.py setup.py	609:624 (1%) 771:786 (1%)	view
15	x 2	tools tools	blob.py blob.py	43:64 (9%) 125:147 (9%)	view
14	x 2	tools tools	doc_intelligence.py doc_intelligence.py	159:172 (4%) 280:293 (4%)	view
12	x 2	ROOT ROOT	setup.py setup.py	642:653 (1%) 665:676 (1%)	view
12	x 2	tools tools	doc_intelligence.py doc_intelligence.py	142:157 (4%) 263:278 (4%)	view
11	x 2	ROOT ROOT	setup.py setup.py	569:579 (<1%) 630:640 (<1%)	view
11	x 2	ROOT ROOT	setup.py setup.py	689:699 (<1%) 777:787 (<1%)	view
11	x 2	ROOT ROOT	setup.py setup.py	630:640 (<1%) 704:714 (<1%)	view
10	x 2	ROOT ROOT	setup.py setup.py	615:624 (<1%) 689:698 (<1%)	view
9	x 2	ROOT ROOT	setup.py setup.py	590:598 (<1%) 734:742 (<1%)	view
9	x 2	connectors/sharepoint connectors/sharepoint	sharepoint_deleted_files_purger.py sharepoint_files_indexer.py	37:47 (4%) 43:53 (4%)	view
9	x 2	chunking/chunkers chunking/chunkers	langchain_chunker.py transcription_chunker.py	125:135 (14%) 103:112 (18%)	view
8	x 2	ROOT ROOT	setup.py setup.py	546:553 (<1%) 615:622 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	716:723 (<1%) 753:760 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	725:732 (<1%) 753:760 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	716:723 (<1%) 744:751 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	581:588 (<1%) 753:760 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	716:723 (<1%) 725:732 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	609:616 (<1%) 762:769 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	581:588 (<1%) 744:751 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	581:588 (<1%) 725:732 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	546:553 (<1%) 689:696 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	744:751 (<1%) 753:760 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	546:553 (<1%) 777:784 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	1124:1131 (<1%) 1150:1157 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	762:769 (<1%) 771:778 (<1%)	view
8	x 2	ROOT ROOT	setup.py setup.py	725:732 (<1%) 744:751 (<1%)	view
7	x 2	ROOT ROOT	setup.py setup.py	1104:1110 (<1%) 1155:1161 (<1%)	view
7	x 2	ROOT ROOT	setup.py setup.py	557:563 (<1%) 692:698 (<1%)	view
7	x 2	ROOT ROOT	setup.py setup.py	557:563 (<1%) 780:786 (<1%)	view
7	x 2	chunking/chunkers chunking/chunkers	base_chunker.py doc_analysis_chunker.py	276:285 (7%) 311:321 (5%)	view
7	x 2	ROOT ROOT	setup.py setup.py	557:563 (<1%) 618:624 (<1%)	view
7	x 2	chunking/chunkers chunking/chunkers	doc_analysis_chunker.py multimodal_chunker.py	132:139 (5%) 129:136 (2%)	view
7	x 2	chunking/chunkers chunking/chunkers	doc_analysis_chunker.py multimodal_chunker.py	142:148 (5%) 139:146 (2%)	view
7	x 2	ROOT ROOT	setup.py setup.py	491:497 (<1%) 499:505 (<1%)	view
7	x 2	ROOT ROOT	setup.py setup.py	420:426 (<1%) 428:434 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	1134:1139 (<1%) 1164:1169 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	1105:1110 (<1%) 1138:1143 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	602:607 (<1%) 773:778 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	1138:1143 (<1%) 1156:1161 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	457:462 (<1%) 648:653 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	464:469 (<1%) 671:676 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	1045:1050 (<1%) 1099:1104 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	528:533 (<1%) 535:540 (<1%)	view
6	x 2	ROOT ROOT	setup.py setup.py	602:607 (<1%) 764:769 (<1%)	view
6	x 2	chunking/chunkers chunking/chunkers	nl2sql_chunker.py transcription_chunker.py	30:42 (18%) 46:58 (12%)	view
6	x 2	ROOT ROOT	setup.py setup.py	602:607 (<1%) 611:616 (<1%)	view