aws / sagemaker-core - Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

45% duplication:

9,430 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
4,336 duplicated lines

6,928 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 6,928 duplicates...

Size	#	Folders	Files	Lines	Code
760	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker-core-llama-3-8B.ipynb	174:933 (79%) 162:921 (80%)	view
135	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker_core_overview.ipynb	148:282 (23%) 134:268 (22%)	view
59	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	258:316 (7%) 374:432 (9%)	view
58	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	197:254 (7%) 225:282 (9%)	view
58	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	197:254 (7%) 211:268 (9%)	view
56	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker_core_overview.ipynb	91:146 (9%) 77:132 (9%)	view
55	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker-core-llama-3-8B.ipynb	17:71 (5%) 14:68 (5%)	view
53	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	139:191 (6%) 157:209 (8%)	view
53	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	139:191 (6%) 171:223 (9%)	view
53	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	734:786 (6%) 541:593 (8%)	view
41	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	195:235 (4%) 854:894 (4%)	view
40	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker-core-llama-3-8B.ipynb	127:166 (4%) 116:155 (4%)	view
39	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	99:137 (4%) 131:169 (6%)	view
26	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb track_local_pytorch_experiment.ipynb	84:109 (4%) 27:52 (4%)	view
25	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker-core-llama-3-8B.ipynb	81:105 (2%) 75:99 (2%)	view
25	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	52:76 (3%) 84:108 (4%)	view
25	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb track_local_pytorch_experiment.ipynb	52:76 (3%) 27:51 (4%)	view
25	x 2	example_notebooks example_notebooks	sagemaker_core_overview.ipynb track_local_pytorch_experiment.ipynb	571:595 (4%) 516:540 (4%)	view
23	x 2	example_notebooks example_notebooks	get_started.ipynb track_local_pytorch_experiment.ipynb	685:707 (3%) 516:538 (4%)	view
23	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb track_local_pytorch_experiment.ipynb	764:786 (2%) 516:538 (4%)	view
23	x 2	example_notebooks example_notebooks	get_started.ipynb sagemaker_core_overview.ipynb	685:707 (3%) 571:593 (3%)	view
23	x 2	example_notebooks example_notebooks	get_started.ipynb inference_and_resource_chaining.ipynb	685:707 (3%) 764:786 (2%)	view
22	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	116:137 (2%) 134:155 (3%)	view
22	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker_core_overview.ipynb	541:562 (3%) 554:575 (3%)	view
22	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	747:768 (2%) 541:562 (3%)	view
19	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb track_local_pytorch_experiment.ipynb	564:582 (3%) 522:540 (3%)	view
19	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker_core_overview.ipynb	564:582 (3%) 577:595 (3%)	view
19	x 2	example_notebooks example_notebooks	sagemaker_core_overview.ipynb track_local_pytorch_experiment.ipynb	77:95 (3%) 34:52 (3%)	view
19	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker-core-llama-3-8B-speculative...	770:788 (2%) 935:953 (1%)	view
18	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb inference_and_resource_chaining.ipynb	395:412 (2%) 596:613 (2%)	view
18	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	59:76 (2%) 77:94 (3%)	view
17	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... track_local_pytorch_experiment.ipynb	935:951 (1%) 522:538 (3%)	view
17	x 2	example_notebooks example_notebooks	get_started.ipynb intelligent_defaults_and_logging.ipynb	691:707 (2%) 564:580 (2%)	view
17	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker_core_overview.ipynb	935:951 (1%) 577:593 (2%)	view
17	x 2	example_notebooks example_notebooks	sagemaker-core-llama-3-8B-speculative... sagemaker-core-llama-3-8B.ipynb	937:953 (1%) 925:941 (1%)	view
17	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	770:786 (2%) 564:580 (2%)	view
17	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker-core-llama-3-8B-speculative...	564:580 (2%) 935:951 (1%)	view
17	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker-core-llama-3-8B.ipynb	772:788 (2%) 925:941 (1%)	view
17	x 2	example_notebooks example_notebooks	get_started.ipynb sagemaker-core-llama-3-8B-speculative...	691:707 (2%) 935:951 (1%)	view
16	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker_core_overview.ipynb	99:114 (2%) 117:132 (2%)	view
16	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	573:588 (1%) 670:685 (1%)	view
16	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	516:531 (1%) 670:685 (1%)	view
16	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	516:531 (1%) 573:588 (1%)	view
15	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	314:328 (1%) 474:488 (1%)	view
15	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker-core-llama-3-8B-speculative...	753:767 (1%) 312:326 (1%)	view
15	x 2	src/sagemaker_core/main src/sagemaker_core/main	config_schema.py config_schema.py	164:178 (1%) 671:685 (1%)	view
15	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb sagemaker-core-llama-3-8B.ipynb	753:767 (1%) 300:314 (1%)	view
15	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker-core-llama-3-8B.ipynb	547:561 (2%) 300:314 (1%)	view
15	x 2	example_notebooks example_notebooks	intelligent_defaults_and_logging.ipynb sagemaker-core-llama-3-8B.ipynb	566:580 (2%) 925:939 (1%)	view
15	x 2	example_notebooks example_notebooks	inference_and_resource_chaining.ipynb intelligent_defaults_and_logging.ipynb	287:301 (1%) 431:445 (2%)	view