awslabs / sagemaker-debugger
File Size

The distribution of size of files (measured in lines of code).

Intro
  • File size measurements show the distribution of size of files.
  • Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
  • It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.
Learn more...
File Size Overall
  • There are 143 files with 15,478 lines of code.
    • 1 very long files (1,003 lines of code)
    • 2 long files (1,248 lines of code)
    • 19 medium size files (5,815 lines of codeclsfd_ftr_w_mp_ins)
    • 24 small files (3,535 lines of code)
    • 97 very small files (3,877 lines of code)
6% | 8% | 37% | 22% | 25%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: zoomable circles | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
py6% | 8% | 39% | 23% | 21%
proto0% | 0% | 0% | 20% | 79%
yml0% | 0% | 0% | 0% | 100%
cfg0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
smdebug/tensorflow46% | 0% | 31% | 13% | 9%
smdebug/core0% | 13% | 38% | 19% | 28%
smdebug/trials0% | 71% | 0% | 0% | 28%
smdebug/profiler0% | 0% | 44% | 38% | 17%
smdebug/xgboost0% | 0% | 93% | 0% | 6%
smdebug/pytorch0% | 0% | 80% | 0% | 19%
smdebug/mxnet0% | 0% | 0% | 55% | 44%
smdebug/rules0% | 0% | 0% | 27% | 72%
smdebug0% | 0% | 0% | 94% | 5%
config0% | 0% | 0% | 0% | 100%
ROOT0% | 0% | 0% | 0% | 100%
smdebug/analysis0% | 0% | 0% | 0% | 100%
Longest Files (Top 50)
File# lines# units
keras.py
in smdebug/tensorflow
 1003 75
hook.py
in smdebug/core
 746 64
trial.py
in smdebug/trials
 502 43
hook.py
in smdebug/pytorch
 473 34
profiler_data_to_pandas.py
in smdebug/profiler/analysis/utils
 403 10
utils.py
in smdebug/core
 403 41
base_hook.py
in smdebug/tensorflow
 397 29
timeline_charts.py
in smdebug/profiler/analysis/notebook_utils
 370 9
pandas_data_analysis.py
in smdebug/profiler/analysis/utils
 327 12
index_reader.py
in smdebug/core
 310 25
trace_event_file_parser.py
in smdebug/profiler
 304 26
profiler_config_parser.py
in smdebug/profiler
 296 17
tensor.py
in smdebug/core
 288 35
session.py
in smdebug/tensorflow
 283 24
timeline_file_writer.py
in smdebug/core/tfevent
 280 20
_crc32c.py
in smdebug/core/tfrecord
 275 3
merge_timelines.py
in smdebug/profiler/analysis/utils
 258 5
utils.py
in smdebug/xgboost
 250 15
hook.py
in smdebug/xgboost
 248 26
locations.py
in smdebug/core
 226 30
s3handler.py
in smdebug/core/access_layer
 216 12
collection.py
in smdebug/core
 208 19
save_config.py
in smdebug/core
 196 20
json_config.py
in smdebug/core
 190 9
metrics_reader_base.py
in smdebug/profiler
 188 17
python_profile_analysis.py
in smdebug/profiler/analysis
 188 17
hook.py
in smdebug/mxnet
 188 16
utils.py
in smdebug/profiler
 187 20
utils.py
in smdebug/tensorflow
 167 9
heatmap.py
in smdebug/profiler/analysis/notebook_utils
 163 3
profiler_config.py
in smdebug/profiler
 161 14
writer.py
in smdebug/core
 160 18
python_profiler.py
in smdebug/profiler
 147 17
pytorch_dataloader_analysis.py
in smdebug/profiler/analysis/utils
 147 7
tf_profiler_parser.py
in smdebug/profiler
 146 13
state_store.py
in smdebug/core
 145 8
algorithm_metrics_reader.py
in smdebug/profiler
 140 12
metrics_histogram.py
in smdebug/profiler/analysis/notebook_utils
 127 7
message_action.py
in smdebug/rules/action
 119 6
reduction_config.py
in smdebug/core
 118 8
collection.py
in smdebug/tensorflow
 118 20
proto
summary.proto
in smdebug/core/tfevent/proto
 111 -
utils.py
in smdebug/core/access_layer
 111 6
event_file_writer.py
in smdebug/core/tfevent
 107 12
step_histogram.py
in smdebug/profiler/analysis/notebook_utils
 106 8
exceptions.py
in smdebug
 105 26
python_profile_analysis_utils.py
in smdebug/profiler/analysis/utils
 99 14
step_timeline_chart.py
in smdebug/profiler/analysis/notebook_utils
 97 3
python_stats_reader.py
in smdebug/profiler/analysis
 96 8
system_metrics_reader.py
in smdebug/profiler
 93 11
Files With Most Units (Top 20)
File# lines# units
keras.py
in smdebug/tensorflow
 1003 75
hook.py
in smdebug/core
 746 64
trial.py
in smdebug/trials
 502 43
utils.py
in smdebug/core
 403 41
tensor.py
in smdebug/core
 288 35
hook.py
in smdebug/pytorch
 473 34
locations.py
in smdebug/core
 226 30
base_hook.py
in smdebug/tensorflow
 397 29
hook.py
in smdebug/xgboost
 248 26
trace_event_file_parser.py
in smdebug/profiler
 304 26
exceptions.py
in smdebug
 105 26
index_reader.py
in smdebug/core
 310 25
session.py
in smdebug/tensorflow
 283 24
utils.py
in smdebug/profiler
 187 20
timeline_file_writer.py
in smdebug/core/tfevent
 280 20
save_config.py
in smdebug/core
 196 20
collection.py
in smdebug/tensorflow
 118 20
collection.py
in smdebug/core
 208 19
writer.py
in smdebug/core
 160 18
metrics_reader_base.py
in smdebug/profiler
 188 17
Files With Long Lines (Top 20)

There are 33 files with lines longer than 120 characters. In total, there are 57 long lines.

File# lines# units# long lines
buildspec_xgboost_1_2_1.yml
in config
 48 - 7
buildspec.yml
in config
 51 - 5
buildspec_tensorflow_2_4.yml
in config
 51 - 5
buildspec_tensorflow_2_3.yml
in config
 49 - 3
state_store.py
in smdebug/core
 145 8 3
compile_protobuf.py
in root
 61 7 2
buildspec_zero_code_change.yml
in config
 33 - 2
buildspec_build_wheel.yml
in config
 36 - 2
message_action.py
in smdebug/rules/action
 119 6 2
proto
types.proto
in smdebug/core/tfevent/proto
 60 - 2
trial.py
in smdebug/trials
 502 43 2
algorithm_metrics_reader.py
in smdebug/profiler
 140 12 1
merge_timelines.py
in smdebug/profiler/analysis/utils
 258 5 1
pytorch_dataloader_analysis.py
in smdebug/profiler/analysis/utils
 147 7 1
training_job.py
in smdebug/profiler/analysis/notebook_utils
 78 8 1
metrics_histogram.py
in smdebug/profiler/analysis/notebook_utils
 127 7 1
timeline_charts.py
in smdebug/profiler/analysis/notebook_utils
 370 9 1
proto
resource_handle.proto
in smdebug/core/tfevent/proto
 24 - 1
proto
versions.proto
in smdebug/core/tfevent/proto
 28 - 1
proto
summary.proto
in smdebug/core/tfevent/proto
 111 - 1