openai / chatgpt-retrieval-plugin

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

16% duplication:

3,894 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
661 duplicated lines

107 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
local_server (68%) --> server (68%)	164	1 file pair	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 107 duplicates...

Size	#	Folders	Files	Lines	Code
36	x 2	local_server server	main.py main.py	63:102 (33%) 47:86 (30%)	view
35	x 2	local_server server	main.py main.py	107:147 (32%) 115:155 (29%)	view
23	x 2	scripts/process_json scripts/process_jsonl	process_json.py process_jsonl.py	113:139 (23%) 111:137 (24%)	view
23	x 2	scripts/process_json scripts/process_jsonl	process_json.py process_jsonl.py	27:56 (23%) 27:56 (24%)	view
23	x 2	scripts/process_jsonl scripts/process_zip	process_jsonl.py process_zip.py	111:137 (24%) 118:144 (23%)	view
23	x 2	scripts/process_json scripts/process_zip	process_json.py process_zip.py	113:139 (23%) 118:144 (23%)	view
15	x 2	datastore/providers datastore/providers	weaviate_datastore.py weaviate_datastore.py	199:213 (5%) 220:234 (5%)	view
15	x 2	scripts/process_json scripts/process_jsonl	process_json.py process_jsonl.py	60:83 (15%) 59:82 (15%)	view
10	x 2	scripts/process_json scripts/process_jsonl	process_json.py process_jsonl.py	85:99 (10%) 84:98 (10%)	view
10	x 2	server server	main.py main.py	94:103 (8%) 113:122 (8%)	view
10	x 2	datastore/providers datastore/providers	analyticdb_datastore.py pgvector_datastore.py	226:235 (4%) 132:143 (8%)	view
9	x 2	services services	extract_metadata.py pii_detection.py	13:31 (40%) 7:24 (60%)	view
9	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py pinecone_datastore.py	113:127 (3%) 179:191 (4%)	view
8	x 2	datastore/providers datastore/providers	azurecosmosdb_datastore.py milvus_datastore.py	268:280 (4%) 477:493 (1%)	view
8	x 2	datastore/providers datastore/providers	azuresearch_datastore.py llama_datastore.py	19:26 (2%) 7:14 (5%)	view
8	x 2	datastore/providers datastore/providers	pgvector_datastore.py pinecone_datastore.py	155:167 (6%) 179:190 (4%)	view
8	x 2	datastore/providers datastore/providers	postgres_datastore.py postgres_datastore.py	53:60 (8%) 61:68 (8%)	view
8	x 2	datastore/providers datastore/providers	chroma_datastore.py pinecone_datastore.py	19:26 (4%) 10:17 (4%)	view
8	x 2	local_server server	main.py main.py	107:114 (7%) 96:103 (6%)	view
8	x 2	datastore/providers datastore/providers	chroma_datastore.py weaviate_datastore.py	19:26 (4%) 14:21 (2%)	view
8	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py pgvector_datastore.py	113:126 (2%) 155:167 (6%)	view
8	x 2	datastore/providers datastore/providers	pinecone_datastore.py weaviate_datastore.py	10:17 (4%) 14:21 (2%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py mongodb_atlas_datastore.py	71:86 (3%) 77:89 (4%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py mongodb_atlas_datastore.py	220:231 (3%) 159:173 (4%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py pinecone_datastore.py	220:231 (3%) 179:189 (3%)	view
7	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py weaviate_datastore.py	113:125 (2%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py mongodb_atlas_datastore.py	113:125 (2%) 159:173 (4%)	view
7	x 2	datastore/providers datastore/providers	pinecone_datastore.py weaviate_datastore.py	179:189 (3%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	llama_datastore.py pgvector_datastore.py	185:195 (4%) 155:166 (5%)	view
7	x 2	datastore/providers datastore/providers	pgvector_datastore.py weaviate_datastore.py	155:166 (5%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	azurecosmosdb_datastore.py mongodb_atlas_datastore.py	270:280 (3%) 159:173 (4%)	view
7	x 2	datastore/providers datastore/providers	analyticdb_datastore.py pgvector_datastore.py	17:23 (3%) 9:15 (5%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py weaviate_datastore.py	220:231 (3%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	azurecosmosdb_datastore.py weaviate_datastore.py	270:280 (3%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	mongodb_atlas_datastore.py pinecone_datastore.py	159:173 (4%) 179:189 (3%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py pgvector_datastore.py	220:231 (3%) 155:166 (5%)	view
7	x 2	datastore/providers datastore/providers	llama_datastore.py mongodb_atlas_datastore.py	185:195 (4%) 159:173 (4%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py llama_datastore.py	220:231 (3%) 185:195 (4%)	view
7	x 2	datastore/providers datastore/providers	llama_datastore.py pinecone_datastore.py	185:195 (4%) 179:189 (3%)	view
7	x 2	datastore/providers datastore/providers	milvus_datastore.py weaviate_datastore.py	479:493 (1%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	milvus_datastore.py pinecone_datastore.py	479:493 (1%) 179:189 (3%)	view
7	x 2	datastore/providers datastore/providers	chroma_datastore.py milvus_datastore.py	220:231 (3%) 479:493 (1%)	view
7	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py llama_datastore.py	113:125 (2%) 185:195 (4%)	view
7	x 2	datastore/providers datastore/providers	mongodb_atlas_datastore.py pgvector_datastore.py	159:173 (4%) 155:166 (5%)	view
7	x 2	datastore/providers datastore/providers	milvus_datastore.py pgvector_datastore.py	479:493 (1%) 155:166 (5%)	view
7	x 2	datastore/providers datastore/providers	mongodb_atlas_datastore.py weaviate_datastore.py	159:173 (4%) 264:275 (2%)	view
7	x 2	local_server server	main.py main.py	9:15 (6%) 10:16 (5%)	view
7	x 2	datastore/providers datastore/providers	elasticsearch_datastore.py qdrant_datastore.py	111:118 (2%) 103:110 (3%)	view
7	x 2	datastore/providers datastore/providers	llama_datastore.py weaviate_datastore.py	185:195 (4%) 264:275 (2%)	view
7	x 2	datastore/providers datastore/providers	azurecosmosdb_datastore.py pinecone_datastore.py	270:280 (3%) 179:189 (3%)	view