facebookresearch / CompilerGym
Source Code Overview

Analysis scope, overview of main, test, generated, deployment, build, and other code.

Source Code Analysis Scope
Files includes and excluded from analyses
ps
txt
cc
cmake
patch
tpl
ll
in
ini
clang-format
proto
  • 28 extensions are included in analyses: py, ps, txt, yaml, js, cc, rst, md, cmake, h, scss, sh, gitignore, cpp, c, patch, json, css, ipynb, hpp, tpl, ll, bzl, html, in, ini, clang-format, proto
  • 8 criteria are used to exclude files from analysis:
    • exclude files with path like ".*/[.][a-zA-Z0-9_]+.*" (Hidden files and folders) (23 files).
    • exclude files with path like ".*/bin/.*" (Binaries for distribution) (12 files).
    • exclude files with path like ".*/docs/.*" (Documentation) (30 files).
    • exclude files with path like ".*/(3rd|[Tt]hird)[-_]?[Pp]arty/.*" (Dependencies) (133 files).
    • exclude files with path like ".*/extern(al)?/.*" (Dependencies) (11 files).
    • exclude files with path like ".*/git[-]history[.]txt" (Git history) (1 file).
    • exclude files with path like ".*/git[-][a-zA-Z0-9_]+[.]txt" (Git data exports for sokrates analyses) (0 files).
    • exclude files with path like ".*[.]txt" (Text files) (52 files).
Overview of Analyzed Files
Basic stats on analyzed files
Intro
For analysis purposes we separate files in scope into several categories: main, test, generated, deployment and build, and other.

  • The main category contains all manually created source code files that are being used in the production.
  • Files in the main category are used as input for other analyses: logical decomposition, concerns, duplication, file size, unit size, and conditional complexity.
  • Test source code files are used only for testing of the product. These files are normally not deployed to production.
  • Build and deployment source code files are used to configure or support build and deployment process.
  • Generated source code files are automatically generated files that have not been manually changed after generation.
  • While a source code folder may contain a number of files, we are primarily interested in the source code files that are being written and maintained by developers.
  • Files containing binaries, documentation, or third-party libraries, for instance, are excluded from analysis. The exception are third-party libraries that have been changed by developers.

main19687 LOC (16%) 212 files
test8725 LOC (7%) 135 files
generated41413 LOC (34%) 1 files
build and deployment31 LOC (<1%) 3 files
other51908 LOC (42%) 142 files
Main Code
All manually created or maintained source code that defines logic of the product that is run in a production environment.
cmake
cc
proto
Explore:   circles  |  sunburst
  • The following criteria are used to filter files:
    • files with paths like ".*".
  • 212 files match defined criteria (19,687 lines of code, 100.0% vs. main code):
    • 114 *.py files (10,360 lines of code)
    • 31 *.js files (3,741 lines of code)
    • 21 *.cmake files (1,975 lines of code)
    • 15 *.cc files (1,624 lines of code)
    • 19 *.h files (925 lines of code)
    • 8 *.scss files (675 lines of code)
    • 1 *.proto files (351 lines of code)
    • 1 *.html files (21 lines of code)
    • 2 *.css files (15 lines of code)
  • " *.py" is biggest, containing 52.62% of code.
  • " *.css" is smallest, containing 0.08% of code.


*.py10360 LOC (52%) 114 files
*.js3741 LOC (19%) 31 files
*.cmake1975 LOC (10%) 21 files
*.cc1624 LOC (8%) 15 files
*.h925 LOC (4%) 19 files
*.scss675 LOC (3%) 8 files
*.proto351 LOC (1%) 1 files
*.html21 LOC (<1%) 1 files
*.css15 LOC (<1%) 2 files
Test Code
Used only for testing of the product. Normally not deployed in a production environment.
cc
cmake
ll
Explore:   circles  |  sunburst
  • The following criteria are used to filter files:
    • files with paths like ".*/[Tt]ests/.*".
    • files with paths like ".*_test[.].*".
    • files with paths like ".*/test_.*".
    • files with paths like ".*[.][Tt]est[.].*".
    • files with paths like ".*[.]test[.].*".
    • files with paths like ".*_tests[.].*".
  • 135 files match defined criteria (8,725 lines of code, 44.3% vs. main code):
    • 121 *.py files (7,985 lines of code)
    • 7 *.cc files (314 lines of code)
    • 3 *.cmake files (254 lines of code)
    • 1 *.ll files (139 lines of code)
    • 1 *.h files (17 lines of code)
    • 1 *.sh files (11 lines of code)
    • 1 *.js files (5 lines of code)
  • " *.py" is biggest, containing 91.52% of code.
  • " *.js" is smallest, containing 0.06% of code.


*.py7985 LOC (91%) 121 files
*.cc314 LOC (3%) 7 files
*.cmake254 LOC (2%) 3 files
*.ll139 LOC (1%) 1 files
*.h17 LOC (<1%) 1 files
*.sh11 LOC (<1%) 1 files
*.js5 LOC (<1%) 1 files
Generated Code
Automatically generated files, not manually changed after generation.
Explore:   circles  |  sunburst
  • The following criteria are used to filter files:
    • files with paths like ".*/package[-]lock[.]json".
    • files with paths like ".*[.](ps|eps|pfa)".
  • 1 file matches defined criteria (41,413 lines of code, 210.4% vs. main code). All matches are in *.json files.


*.json41413 LOC (100%) 1 files
Build and Deployment Code
Source code used to configure or support build and deployment process.
Explore:   circles  |  sunburst
  • The following criteria are used to filter files:
    • files with paths like ".*[.]git[a-z]+".
    • files with paths like ".*/[.]gitignore".
    • files with paths like ".*[.]sh".
    • files with paths like ".*/package[-]lock[.]json".
    • files with paths like ".*/package[.]json".
  • 3 files match defined criteria (31 lines of code, 0.2% vs. main code). All matches are in *.sh files.


*.sh31 LOC (100%) 3 files
Other Code
cc
ini
in
Explore:   circles  |  sunburst
  • The following criteria are used to filter files:
    • files with paths like ".*/[.]gitignore".
    • files with paths like ".*[.]md".
    • files with paths like ".*[.]txt".
    • files with paths like ".*/INSTALL[.][a-z0-9]+".
    • files with paths like ".*/README[.][a-z0-9]+".
    • files with paths like ".*[.](rst|rest|resttxt|rsttxt)".
    • files with paths like ".*[.]json".
    • files with paths like ".*/[Ee]xamples/.*".
    • files with paths like ".*[.]ini".
    • files with paths like ".*[.]patch".
    • files with paths like ".*/LICENSE[.][a-z0-9]+".
    • files with paths like ".*[.](xml|xsd|robot|sql|pgsql|dashboard|profile|ipynb|raml|avsc|al)".
  • 142 files match defined criteria (51,908 lines of code, 263.7% vs. main code):
    • 3 *.json files (41,477 lines of code)
    • 82 *.py files (6,646 lines of code)
    • 21 *.md files (1,728 lines of code)
    • 2 *.ipynb files (1,017 lines of code)
    • 4 *.cc files (574 lines of code)
    • 22 *.yaml files (295 lines of code)
    • 5 *.c files (152 lines of code)
    • 1 *.ini files (13 lines of code)
    • 1 *.sh files (5 lines of code)
    • 1 *.in files (1 lines of code)
  • " *.json" is biggest, containing 79.9% of code.
  • " *.in" is smallest, containing 0% of code.


*.json41477 LOC (79%) 3 files
*.py6646 LOC (12%) 82 files
*.md1728 LOC (3%) 21 files
*.ipynb1017 LOC (1%) 2 files
*.cc574 LOC (1%) 4 files
*.yaml295 LOC (<1%) 22 files
*.c152 LOC (<1%) 5 files
*.ini13 LOC (<1%) 1 files
*.sh5 LOC (<1%) 1 files
*.in1 LOC (<1%) 1 files
Analyzers
Info about analyzers used for source code examinations.
  • *.py files are analyzed with PythonAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • Unit size analysis
    • Conditional complexity analysis
    • Basic heuristic dependency analysis
  • *.js files are analyzed with JavaScriptAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • Unit size analysis
    • Conditional complexity analysis
    • No dependency analysis
  • *.cmake files are analyzed with DefaultLanguageAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Basic code cleaning (empty lines removed for LOC calculations and duplication calculations)
    • No unit size analysis
    • No conditional complexity analysis
    • No dependency analysis
  • *.cc files are analyzed with CppAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • Unit size analysis
    • Conditional complexity analysis
    • Advanced heuristic dependency analysis
  • *.h files are analyzed with CppAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • Unit size analysis
    • Conditional complexity analysis
    • Advanced heuristic dependency analysis
  • *.scss files are analyzed with ScssAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • No unit size analysis
    • No conditional complexity analysis
    • No dependency analysis
  • *.proto files are analyzed with DefaultLanguageAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Basic code cleaning (empty lines removed for LOC calculations and duplication calculations)
    • No unit size analysis
    • No conditional complexity analysis
    • No dependency analysis
  • *.html files are analyzed with HtmlAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • Unit size analysis
    • Conditional complexity analysis
    • Advanced heuristic dependency analysis
  • *.css files are analyzed with CssAnalyzer:
    • All basic standard analyses supported (source code overview, duplication, file size, concerns, findings, metrics, controls)
    • Advanced code cleaning (empty lines and comments removed for LOC calculations, additional cleaning for duplication calculations)
    • No unit size analysis
    • No conditional complexity analysis
    • No dependency analysis


2022-01-25 20:05