GoogleCloudPlatform / scientific-computing-examples

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

48% duplication:

5,538 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
2,671 duplicated lines

411 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
llama2-finetuning-slurm (82%) --> qwiklabs (91%)	767	11 file pairs	details...
fsi (38%) --> hcls (44%)	331	4 file pairs	details...
fsi (24%) --> llama2-finetuning-slurm (22%)	210	2 file pairs	details...
fsi (24%) --> qwiklabs (25%)	210	2 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 411 duplicates...

Size	#	Folders	Files	Lines	Code
142	x 2	fsi/MonteCarloMultiGPU fsi/MonteCarloMultiGPU	montecarlo_toolkit.yaml montecarlo_toolkit_dws.yaml	25:199 (74%) 25:198 (66%)	view
118	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	135:252 (24%) 131:248 (19%)	view
103	x 2	llama2-finetuning-slurm qwiklabs/llama2-finetuning-slurm	hpc-slurm-llama2.yaml hpc-slurm-llama2.yaml	27:143 (33%) 28:144 (39%)	view
85	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	15:99 (17%) 15:99 (14%)	view
64	x 2	fsi/MonteCarloMultiGPU llama2-finetuning-slurm	montecarlo_toolkit.yaml hpc-slurm-llama2.yaml	96:159 (33%) 52:115 (20%)	view
64	x 2	fsi/MonteCarloMultiGPU qwiklabs/llama2-finetuning-slurm	montecarlo_toolkit_dws.yaml hpc-slurm-llama2.yaml	95:158 (29%) 53:116 (24%)	view
64	x 2	fsi/MonteCarloMultiGPU llama2-finetuning-slurm	montecarlo_toolkit_dws.yaml hpc-slurm-llama2.yaml	95:158 (29%) 52:115 (20%)	view
64	x 2	fsi/MonteCarloMultiGPU qwiklabs/llama2-finetuning-slurm	montecarlo_toolkit.yaml hpc-slurm-llama2.yaml	96:159 (33%) 53:116 (24%)	view
57	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	fine-tune.py fine-tune.py	23:101 (100%) 23:101 (100%)	view
39	x 2	fluxfw-gcp/img fluxfw-gcp/img	hcl compute.pkr.hcl login.pkr.hcl	1:47 (66%) 1:47 (66%)	view
37	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	main.tf main.tf	1:46 (100%) 1:46 (100%)	view
36	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	versions.tf versions.tf	1:37 (100%) 1:37 (100%)	view
34	x 2	fsi/MonteCarloMultiGPU fsi/MonteCarloMultiGPU	montecarlo_toolkit.yaml montecarlo_toolkit_dws.yaml	201:237 (17%) 224:260 (15%)	view
28	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit.yaml namd-slurm.yaml	207:237 (14%) 173:203 (16%)	view
28	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	446:473 (5%) 565:592 (4%)	view
28	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit_dws.yaml namd-slurm.yaml	230:260 (13%) 173:203 (16%)	view
24	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	variables.tf variables.tf	1:26 (100%) 1:26 (100%)	view
23	x 2	hcls/namd-on-slurm hcls/namd-on-slurm	namd-slurm-qwiklab.yaml namd-slurm.yaml	38:61 (11%) 36:59 (13%)	view
21	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	275:295 (4%) 385:405 (3%)	view
21	x 2	llama2-finetuning-slurm qwiklabs/llama2-finetuning-slurm	hpc-slurm-llama2.yaml hpc-slurm-llama2.yaml	286:308 (6%) 201:223 (8%)	view
20	x 2	llama2-finetuning-slurm qwiklabs/llama2-finetuning-slurm	hpc-slurm-llama2.yaml hpc-slurm-llama2.yaml	262:283 (6%) 177:198 (7%)	view
20	x 2	fluxfw-gcp/img fluxfw-gcp/img	hcl login.pkr.hcl manager.pkr.hcl	22:47 (33%) 18:43 (35%)	view
20	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	113:132 (4%) 111:130 (3%)	view
20	x 2	llama2-finetuning-slurm qwiklabs/llama2-finetuning-slurm	hpc-slurm-llama2.yaml hpc-slurm-llama2.yaml	311:331 (6%) 226:246 (7%)	view
20	x 2	fluxfw-gcp/img fluxfw-gcp/img	hcl compute.pkr.hcl manager.pkr.hcl	22:47 (33%) 18:43 (35%)	view
20	x 2	llama2-finetuning-slurm qwiklabs/llama2-finetuning-slurm	hpc-slurm-llama2.yaml hpc-slurm-llama2.yaml	218:244 (6%) 147:173 (7%)	view
19	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	310:328 (4%) 291:309 (3%)	view
18	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	427:444 (3%) 545:562 (3%)	view
17	x 2	fluxfw-gcp/img fluxfw-gcp/img	hcl compute.pkr.hcl manager.pkr.hcl	1:18 (28%) 1:18 (30%)	view
17	x 2	fluxfw-gcp/img fluxfw-gcp/img	hcl login.pkr.hcl manager.pkr.hcl	1:18 (28%) 1:18 (30%)	view
16	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit_dws.yaml namd-slurm-qwiklab.yaml	42:57 (7%) 41:56 (7%)	view
16	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit.yaml namd-slurm-qwiklab.yaml	42:58 (8%) 41:56 (7%)	view
16	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit_dws.yaml namd-slurm.yaml	42:57 (7%) 39:54 (9%)	view
16	x 2	fluxfw-gcp/img fluxfw-gcp/img	loginbuild.yaml managerbuild.yaml	15:30 (47%) 15:30 (51%)	view
16	x 2	fsi/MonteCarloMultiGPU hcls/namd-on-slurm	montecarlo_toolkit.yaml namd-slurm.yaml	42:58 (8%) 39:54 (9%)	view
15	x 2	notebooks notebooks	batch_create_a2_vm.ipynb batch_hello_world.ipynb	380:394 (3%) 428:442 (2%)	view
15	x 2	llama2-finetuning-slurm/files llama2-finetuning-slurm/files	main.tf versions.tf	1:15 (40%) 1:15 (41%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	versions.tf variables.tf	1:15 (41%) 1:15 (62%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	main.tf variables.tf	1:15 (40%) 1:15 (62%)	view
15	x 2	llama2-finetuning-slurm/files llama2-finetuning-slurm/files	variables.tf versions.tf	1:15 (62%) 1:15 (41%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	main.tf versions.tf	1:15 (40%) 1:15 (41%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	versions.tf main.tf	1:15 (41%) 1:15 (40%)	view
15	x 2	qwiklabs/llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	main.tf variables.tf	1:15 (40%) 1:15 (62%)	view
15	x 2	llama2-finetuning-slurm/files llama2-finetuning-slurm/files	main.tf variables.tf	1:15 (40%) 1:15 (62%)	view
15	x 2	qwiklabs/llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	main.tf versions.tf	1:15 (40%) 1:15 (41%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	variables.tf main.tf	1:15 (62%) 1:15 (40%)	view
15	x 2	qwiklabs/llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	variables.tf versions.tf	1:15 (62%) 1:15 (41%)	view
15	x 2	llama2-finetuning-slurm/files qwiklabs/llama2-finetuning-slurm/files	variables.tf versions.tf	1:15 (62%) 1:15 (41%)	view
14	x 2	fluxfw-gcp/img fluxfw-gcp/img	computebuild.yaml managerbuild.yaml	15:28 (37%) 15:28 (45%)	view
14	x 2	fluxfw-gcp/img fluxfw-gcp/img	computebuild.yaml loginbuild.yaml	30:43 (37%) 28:41 (41%)	view