aws-samples / aws-glue-samples
File Age

File age measurements show the distribution of file ages (days since the first commit) and the recency of file updates (days since the latest commit).

Summary
  • Number of files: 59
  • Daily file updates (only one update per file and date counted): 131
  • First update: 2017-05-21
  • Latest update: 2022-01-04
  • Days between first and latest update: 1690 (241 weeks, estimated 1205 working days)
  • Active days (at least one file change): 65
  • Data:
File Change History Overall
File Age Distribution Overall
Days since first update
  • There are 59 files with 4,268 lines of code in files.
    • 41 files that are 366+ days old (3,445 lines of code)
    • 7 files that are 181-365 days old (343 lines of code)
    • 11 files that are 91-180 days old (480 lines of code)
    • 0 files that are 31-90 days old (0 lines of code)
    • 0 files that are 1-30 days old (0 lines of code)
80% | 8% | 11% | 0% | 0%
Legend:
366+
181-365
91-180
31-90
1-30
Latest Change Distribution Overall
Days since last update
  • There are 59 files with 4,268 lines of code in files.
    • 30 files have been last changed 366+ days ago (1,653 lines of code)
    • 16 files have been last changed 181-365 days ago (991 lines of code)
    • 13 files have been last changed 91-180 days ago (1,624 lines of code)
    • 0 files have been last changed 31-90 days ago (0 lines of code)
    • 0 files have been last changed 1-30 days ago (0 lines of code)
38% | 23% | 38% | 0% | 0%
Legend:
366+
181-365
91-180
31-90
1-30
File Change History per File Extension
scala, md, py, java, xml, ipynb, txt, sh, json, yaml
File Age Distribution per Extension
Days since first update
366+
181-365
91-180
31-90
1-30
py100% | 0% | 0% | 0% | 0%
scala57% | 10% | 31% | 0% | 0%
java100% | 0% | 0% | 0% | 0%
yaml50% | 50% | 0% | 0% | 0%
Latest Change Distribution per Extension
Days since last update
366+
181-365
91-180
31-90
1-30
scala40% | 27% | 31% | 0% | 0%
py29% | 6% | 64% | 0% | 0%
java53% | 46% | 0% | 0% | 0%
yaml50% | 50% | 0% | 0% | 0%
File Change History per Logical Decomposition
primary
primary (file age distribution)
Days since first update
366+
181-365
91-180
31-90
1-30
utilities/Hive_metastore_migration/src100% | 0% | 0% | 0% | 0%
GlueCustomConnectors/localValidation100% | 0% | 0% | 0% | 0%
GlueCustomConnectors/development/Spark52% | 0% | 47% | 0% | 0%
utilities/Crawler_undo_redo/src100% | 0% | 0% | 0% | 0%
GlueCustomConnectors/development/Athena100% | 0% | 0% | 0% | 0%
utilities/sagemaker_notebook_automation50% | 50% | 0% | 0% | 0%
GlueCustomConnectors/gluescripts/withoutConnection63% | 36% | 0% | 0% | 0%
GlueCustomConnectors/gluescripts/withConnection63% | 36% | 0% | 0% | 0%
GlueCustomConnectors/glueJobValidation100% | 0% | 0% | 0% | 0%
utilities/use_only_IAM_access_controls100% | 0% | 0% | 0% | 0%
primary (latest change distribution)
Days since last update
366+
181-365
91-180
31-90
1-30
GlueCustomConnectors/localValidation100% | 0% | 0% | 0% | 0%
utilities/Crawler_undo_redo/src100% | 0% | 0% | 0% | 0%
utilities/Hive_metastore_migration/src15% | 0% | 84% | 0% | 0%
GlueCustomConnectors/development/Athena100% | 0% | 0% | 0% | 0%
utilities/sagemaker_notebook_automation50% | 50% | 0% | 0% | 0%
GlueCustomConnectors/glueJobValidation100% | 0% | 0% | 0% | 0%
GlueCustomConnectors/gluescripts/withoutConnection38% | 61% | 0% | 0% | 0%
GlueCustomConnectors/gluescripts/withConnection39% | 60% | 0% | 0% | 0%
GlueCustomConnectors/development/Spark0% | 52% | 47% | 0% | 0%
utilities/use_only_IAM_access_controls0% | 0% | 100% | 0% | 0%
Oldest Files (Top 20)
File# lines# unitslast modified
(days ago)
created
(days ago)
# changes
hive_metastore_migration.py
in utilities/Hive_metastore_migration/src
1051 92 174 1632 14
export_from_datacatalog.py
in utilities/Hive_metastore_migration/src
96 6 1029 1632 5
import_into_datacatalog.py
in utilities/Hive_metastore_migration/src
95 5 1029 1632 5
crawler_undo.py
in utilities/Crawler_undo_redo/src
80 4 634 1632 4
scripts_utils.py
in utilities/Crawler_undo_redo/src
74 8 634 1632 4
crawler_redo_from_backup.py
in utilities/Crawler_undo_redo/src
46 3 634 1632 4
__init__.py
in utilities/Hive_metastore_migration/src
1 - 1029 1632 3
__init__.py
in utilities/Crawler_undo_redo/src
1 - 1029 1632 3
update_permission.py
in utilities/use_only_IAM_access_controls
93 - 174 824 9
glue_sagemaker_notebook.yaml
in utilities/sagemaker_notebook_automation
177 - 553 553 1
ExampleMetadataHandler.java
in GlueCustomConnectors/development/Athena/src/main/java/com/amazonaws/athena/connectors/example
106 7 407 409 2
ExampleRecordHandler.java
in GlueCustomConnectors/development/Athena/src/main/java/com/amazonaws/athena/connectors/example
84 4 407 409 2
SparkConnectorMySQL.scala
in GlueCustomConnectors/development/Spark
199 17 323 407 2
SparkConnectorCSV.java
in GlueCustomConnectors/development/Spark
162 14 323 407 2
MinimalSparkConnector.java
in GlueCustomConnectors/development/Spark
120 14 323 407 2
DbtableQueryTest.scala
in GlueCustomConnectors/localValidation
112 2 407 407 1
glue_job_validation_update.py
in GlueCustomConnectors/glueJobValidation
96 - 407 407 1
DataSinkTest.scala
in GlueCustomConnectors/localValidation
74 4 407 407 1
DataTypeMappingTest.scala
in GlueCustomConnectors/localValidation
54 2 407 407 1
JDBCUrlTest.scala
in GlueCustomConnectors/localValidation
50 2 407 407 1
Files Not Recently Changed (Top 20)
File# lines# unitslast modified
(days ago)
created
(days ago)
# changes
__init__.py
in utilities/Crawler_undo_redo/src
1 - 1029 1632 3
__init__.py
in utilities/Hive_metastore_migration/src
1 - 1029 1632 3
import_into_datacatalog.py
in utilities/Hive_metastore_migration/src
95 5 1029 1632 5
export_from_datacatalog.py
in utilities/Hive_metastore_migration/src
96 6 1029 1632 5
crawler_redo_from_backup.py
in utilities/Crawler_undo_redo/src
46 3 634 1632 4
scripts_utils.py
in utilities/Crawler_undo_redo/src
74 8 634 1632 4
crawler_undo.py
in utilities/Crawler_undo_redo/src
80 4 634 1632 4
glue_sagemaker_notebook.yaml
in utilities/sagemaker_notebook_automation
177 - 553 553 1
athena-script-example.py
in GlueCustomConnectors/gluescripts/withoutConnection
18 - 407 407 1
athena-script-example.py
in GlueCustomConnectors/gluescripts/withConnection
18 - 407 407 1
AthenaCloudwatch.java
in GlueCustomConnectors/gluescripts/withoutConnection
21 1 407 407 1
AthenaCloudwatch.java
in GlueCustomConnectors/gluescripts/withConnection
21 1 407 407 1
SparkSnowflake.java
in GlueCustomConnectors/gluescripts/withConnection
23 1 407 407 1
SparkSnowflake.java
in GlueCustomConnectors/gluescripts/withoutConnection
24 1 407 407 1
JDBCSalesforce.java
in GlueCustomConnectors/gluescripts/withConnection
24 1 407 407 1
JDBCSalesforce.java
in GlueCustomConnectors/gluescripts/withoutConnection
27 1 407 407 1
DataSchemaTest.scala
in GlueCustomConnectors/localValidation
44 2 407 407 1
FilterPredicateTest.scala
in GlueCustomConnectors/localValidation
46 2 407 407 1
SecretsManagerTest.scala
in GlueCustomConnectors/localValidation
47 2 407 407 1
CatalogConnectionTest.scala
in GlueCustomConnectors/localValidation
48 2 407 407 1
Most Recently Created Files (Top 20)
File# lines# unitslast modified
(days ago)
created
(days ago)
# changes
TPCDSUtils.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
64 4 105 106 2
GlueJobValidationDataSchemaTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
62 1 105 106 2
TPCDSScan.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
59 2 105 106 2
GlueJobValidationDataPartitioningTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
57 1 105 106 2
TPCDSPartitionReader.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
45 2 105 106 2
GlueJobValidationDataSourceTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
26 1 105 106 2
TPCDSTable.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
21 1 105 106 2
TPCDSInputPartition.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
20 - 105 106 2
TPCDSSource.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
19 1 105 106 2
TPCDSPartitionReaderFactory.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
10 1 105 106 2
MinimalSpark3Connector.scala
in GlueCustomConnectors/development/Spark/glue-3.0
97 17 155 166 2
JDBCSalesforce.scala
in GlueCustomConnectors/gluescripts/withoutConnection
37 1 323 323 1
JDBCSalesforce.scala
in GlueCustomConnectors/gluescripts/withConnection
32 1 323 323 1
SparkSnowflake.scala
in GlueCustomConnectors/gluescripts/withoutConnection
28 1 323 323 1
SparkSnowflake.scala
in GlueCustomConnectors/gluescripts/withConnection
27 1 323 323 1
AthenaCloudwatch.scala
in GlueCustomConnectors/gluescripts/withConnection
21 1 323 323 1
AthenaCloudwatch.scala
in GlueCustomConnectors/gluescripts/withoutConnection
21 1 323 323 1
glue_sagemaker_notebook_cn.yaml
in utilities/sagemaker_notebook_automation
177 - 363 364 2
SparkConnectorMySQL.scala
in GlueCustomConnectors/development/Spark
199 17 323 407 2
SparkConnectorCSV.java
in GlueCustomConnectors/development/Spark
162 14 323 407 2
Most Recently Changed Files (Top 20)
File# lines# unitslast modified
(days ago)
created
(days ago)
# changes
TPCDSUtils.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
64 4 105 106 2
GlueJobValidationDataSchemaTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
62 1 105 106 2
TPCDSScan.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
59 2 105 106 2
GlueJobValidationDataPartitioningTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
57 1 105 106 2
TPCDSPartitionReader.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
45 2 105 106 2
GlueJobValidationDataSourceTest.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/jobvalidation/scala
26 1 105 106 2
TPCDSTable.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
21 1 105 106 2
TPCDSInputPartition.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
20 - 105 106 2
TPCDSSource.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
19 1 105 106 2
TPCDSPartitionReaderFactory.scala
in GlueCustomConnectors/development/Spark/glue-3.0/tpcds-custom-connector-for-glue3.0/src/main/scala/com/amazonaws/services/glue/marketplace/connector...
10 1 105 106 2
MinimalSpark3Connector.scala
in GlueCustomConnectors/development/Spark/glue-3.0
97 17 155 166 2
hive_metastore_migration.py
in utilities/Hive_metastore_migration/src
1051 92 174 1632 14
update_permission.py
in utilities/use_only_IAM_access_controls
93 - 174 824 9
SparkConnectorMySQL.scala
in GlueCustomConnectors/development/Spark
199 17 323 407 2
SparkConnectorCSV.java
in GlueCustomConnectors/development/Spark
162 14 323 407 2
MinimalSparkConnector.java
in GlueCustomConnectors/development/Spark
120 14 323 407 2
JDBCSalesforce.scala
in GlueCustomConnectors/gluescripts/withoutConnection
37 1 323 323 1
jdbc-script-example.py
in GlueCustomConnectors/gluescripts/withoutConnection
34 - 323 407 2
JDBCSalesforce.scala
in GlueCustomConnectors/gluescripts/withConnection
32 1 323 323 1
jdbc-script-example.py
in GlueCustomConnectors/gluescripts/withConnection
30 - 323 407 2