openai / evals

	github actions make docker
Standard analyses:
		Main Code: 41,930 LOC (1499 files) = PY (71%) + YAML (19%) + JSONL (4%) + IPYNB (3%) + HTML (<1%) + JS (<1%) + TOML (<1%) + IN (<1%) Secondary code: Test: 2,508 LOC (50); Generated: 0 LOC (0); Build & Deploy: 794 LOC (31); Other: 4,961 LOC (76);
		Duplication: 13%
		File Size: 5% long (>1000 LOC), 66% short (<= 200 LOC)
		Logical Component Decomposition: primary (3 components)
		2 years, 1 month old 99% of code older than 365 days 99% of code not updated in the past 365 days
		0% of code updated more than 50 times Also see temporal dependencies for files frequently changed in same commits.
		Goals: Keep the system simple and easy to change (4)
Experimental analyses (less reliable heuristic analyses):
		Unit Size: 0% long (>100 LOC), 59% short (<= 10 LOC)
		Conditional Complexity: 1% complex (McCabe index > 50), 67% simple (McCabe index <= 5)
		Static Component Dependencies: primary (3 components)

	Features of interest: TODOs 42 files

All Analysis Reports

Source Code Overview

Components

Temporal Dependencies

Duplication

File Size

File Age & Freshness

File Change Frequency

Commits

Contributors

Unit Size*

Conditional Complexity*

Features of Interest

All Metrics

Goals & Controls

Component Dependencies*

Trend

Notes & Findings

Overall Activity Per Year

Latest commit date: 2024-12-18

Reference analysis date: 2025-05-04

0

commits
(30 days)

0

contributors
(30 days)

0	47	636
0	19	436
2025	2024	2023

Activity Per File Extension

commits

	2025	2024	2023
		34	127	py
		23	491	yaml
jsonl		16	454	jsonl
		13	19	toml
		2		html
		1	9	ipynb
in			4	in
			2	js

contributors

	2025	2024	2023
		12	46	py
		10	401	yaml
jsonl		8	393	jsonl
		7	13	toml
		2		html
		1	8	ipynb
in			3	in
			2	js

Visual Code Explorers

MAIN (1499)	Circles	Sunburst
TEST (50)	Circles	Sunburst
BUILD AND DEPLOYMENT (31)	Circles	Sunburst
GENERATED (0)	Circles	Sunburst
OTHER (76)	Circles	Sunburst

File Visualizations

File size views:

Duplication views:

File age views:

	files grouped by age category files grouped by folder
	files grouped by freshness category files grouped by folder

File change frequency views:

Contributors per file views:

Contributor Visualizations

Contributor dependency views:

past 30 days: graphviz | 2D graph | 2D graph (with files) | 2D graph (with shared files only) | 3D graph | 3D graph (with files) | 3D graph (with shared files only)
past 3 months: graphviz | 2D graph | 2D graph (with files) | 2D graph (with shared files only) | 3D graph | 3D graph (with files) | 3D graph (with shared files only)
past 6 months: graphviz | 2D graph | 2D graph (with files) | 2D graph (with shared files only) | 3D graph | 3D graph (with files) | 3D graph (with shared files only)
past year: graphviz | 2D graph | 2D graph (with files) | 2D graph (with shared files only) | 3D graph | 3D graph (with files) | 3D graph (with shared files only)

Components and Dependencies Visualizations

Components

Temporal Dependencies

Duplication

Commits Racing Charts

30 days

3 months

6 months

PRIMARY (3)

Bubble Chart

Tree Map

graphviz

2D

3D

graphviz

2D

3D

graphviz

2D

3D

Duplication Graph

All Time

12 Months

File Dependencies Visualizations

Temporal dependencies among files:

past 30 days: no dependencies
past 3 months: no dependencies
past 6 months: no dependencies

Units Visualizations

Unit size and conditional complexity views:

Lists of Files Per Scope

Analysis Results

CONFIGURATION: JSON
ALL ANALYSIS RESULTS: JSON
DUPLICATES: TXT | JSON
UNITS: TXT | JSON
CONTRIBUTORS: TXT | JSON
LOGICAL DECOMPOSITIONS: JSON
CONCERNS: JSON
CONTROLS: TXT
ALL METRICS: TXT

Zipped Files

GIT HISTORY: ZIP
ALL FILES IN ALL ANALYSIS SCOPES: ZIP

Generative AI tools, like ChatGPT or Gemini, can help you explore and discuss various aspects of source code repositories using simple prompts and file uploads. Sokrates provides you with curated data that you can use to analyze your source code further.

Example Prompt 1: Repository Evolution Analyzer (based on git history)

Try on: OpenAI ChatGPT | Claude.ai Chat | Google Gemini

Files to upload: git-history.zip

Prompt:

You are an experienced software architect tasked with analyzing Git commit history.
Context:
- Repository Name: openai / evals
- Repository Description: Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
- Data Provided: A plain text file containing Git commit records. Source code is not available.
- Analysis Basis: Your analysis must be based solely on inferring context from the provided file paths, file names, file extensions, commit timestamps, and author information within the commit records.

Input Format:
The commit records are provided in the following format per line:
[YYYY-MM-DD] [author email] [commit hash] [file path changed]

Requested Insights:

Please analyze the provided commit history and generate a report covering the following sections:

1. Management Summary
- Provide a concise, high-level overview suitable for non-technical stakeholders.
- Summarize the repository's likely purpose, main development phases observed, and key takeaways about the team structure/activity.

2. Functional Overview
- Based only on file/module/class names, infer and describe the primary functions and capabilities of the software in this repository.
- What services or features does the codebase seem to support?

3. Key Business/Domain Concepts
- Identify and list core business or domain concepts reflected in the file and directory names (e.g., reporting, analytics, data export, user management, sourcecode analysis, landscape, visualization).

4. Inferred Architecture & Design
- Infer potential architectural patterns (e.g., modular, layered, microservices) based on directory structures and file naming conventions.
- Highlight notable design choices evident from the structure, such as separation of concerns (e.g., common module), use of configuration files, or templating engines.

5. Inferred Technology Stack
- List the programming languages, frameworks, libraries, and tools likely used, based on file extensions (.java, .py, .js, .html, .xml, .yml, pom.xml, dockerfile), configuration files, and common directory structures (e.g., /src/main/java).
- Categorize these technologies where possible (e.g., Backend, Frontend, Build, Infrastructure).

6. Logical Components / Modules
- Identify the major logical components or modules suggested by the directory structure (e.g., codeanalyzer, reports, cli, common, web).
- Briefly describe the inferred purpose of each major component.

7. Evolution by Year
- Summarize the apparent focus of development activity for each year present in the commit history.
- Assign a thematic title to each year (e.g., "Year 1: Foundation & Core Analysis", "Year 2: Reporting & Visualization Expansion").

8. Trends Over Time
Identify and describe significant trends observed across the years, such as:
- Shifts in technology usage (e.g., introduction of Docker).
- Changes in module activity (e.g., increased focus on landscape analysis).
- Signs of project maturity, expansion into new areas, or periods of focused maintenance.
9. Team Dynamics
- Analyze contributor patterns based on author emails and commit frequency:
 - Identify core maintainer(s) vs. occasional contributors.
 - Describe observable contribution trends (e.g., stable team, high turnover, reliance on a single developer).
 - Note any appearance of automated contributors (e.g., Dependabot).
10. Risk Analysis:
- Based on the findings in all previous sections (Team Dynamics, Technology, Architecture, Evolution), identify potential key risks for the project.
- Examples: Key person dependency, technology obsolescence, maintainability challenges due to complexity, potential scalability bottlenecks, bus factor.

Input File:
You will receive the commit records in a plain text file. Please parse this file to perform your analysis.

Example Prompt 2: File name conventions

Try on: OpenAI ChatGPT | Claude.ai Chat | Google Gemini

Files to upload: files.json

Prompt:

Example Prompt 3: Technology analyzer (based of file paths)

Try on: OpenAI ChatGPT | Claude.ai Chat | Google Gemini

Files to upload: files.json

Prompt:

Analyze the provided list of file paths to identify as many technologies, frameworks, tools, programming languages, cloud services, and architectural patterns used in the project as possible.

**Input:**
Please analyze the following list of file structures (provided as a list of strings or a JSON object containing file paths and optionally extensions/metadata):

*(Example: A JSON array like the one previously provided, or a simple list like `["src/server.ts", "Dockerfile", "tests/e2e/spec.cy.js", "infra/main.tf"]`)*

**Analysis Instructions:**
When performing the analysis, please consider the following:

1. **File Extensions:** Identify languages and configuration types (e.g., `.ts`, `.js`, `.py`, `.java`, `.tf`, `.yml`, `.hcl`, `.css`, `.html`).
2. **Directory Structure & Naming:** Look for conventional directory names (`src`, `tests`, `infra`, `docs`, `cicd`) and names indicating specific components or frameworks (`packages/bff`, `features`, `controllers`, `services`, `cypress`).
3. **Specific Filenames:** Identify standard configuration or definition files (e.g., `package.json`, `pom.xml`, `docker-compose.yml`, `Dockerfile`, `Jenkinsfile`, `webpack.config.js`, `tsconfig.json`, `requirements.txt`, `terragrunt.hcl`, `lint-staged.config.js`, `commitlint.config.js`).
4. **Keywords in Paths:** Look for names of known technologies, tools, frameworks, or cloud providers within the paths (e.g., `aws`, `azure`, `lambda`, `sqs`, `react`, `nest`, `k6`, `datadog`, `pm2`).

**Output:**
Please present the findings as a list of identified technologies. If possible, group them into relevant categories such as:

* Programming Languages
* Frameworks & Runtimes
* Testing Tools
* Infrastructure as Code / Cloud Providers
* Containerization & Orchestration
* Build Tools / Bundlers
* Databases (if identifiable)
* Monitoring & Logging
* CI/CD & Dev Tools
* Architectural Patterns