openai / frontier-evals
File Churn

File change frequency (churn) shows the distribution of file updates (days with at least one commit).

Overview
File Change Frequency Overall
  • There are 265 files with 27,760 lines of code.
    • 0 files changed more than 100 times (0 lines of code)
    • 0 files changed 51-100 times (0 lines of code)
    • 0 files changed 21-50 times (0 lines of code)
    • 7 files changed 6-20 times (1,780 lines of code)
    • 258 files changed 1-5 times (25,980 lines of code)
0% | 0% | 0% | 6% | 93%
Legend:
101+
51-100
21-50
6-20
1-5

explore: grouped by folders | grouped by update frequency | data
Contributors Count Frequency Overall
  • There are 265 files with 27,760 lines of code.
    • 0 files changed by more than 25 contributors (0 lines of code)
    • 0 files changed by 11-25 contributors (0 lines of code)
    • 0 files changed by 6-10 contributors (0 lines of code)
    • 16 files changed by 2-5 contributors (4,370 lines of code)
    • 249 files changed by 1 contributor (23,390 lines of code)
0% | 0% | 0% | 15% | 84%
Legend:
26+
11-25
6-10
2-5
1

explore: grouped by folders | grouped by contributors count | data
File Change Frequency per File Extension
md, txt, py, json, patch, yaml, sh, rs, diff, toml, gitignore, js, ts, gitattributes, html, dockerignore, css, tsx, ini, dockerfile, gitmodules
File Change Frequency per Extension
The number of recorded file updates
101+
51-100
21-50
6-20
1-5
py0% | 0% | 0% | 7% | 92%
yaml0% | 0% | 0% | 3% | 96%
rs0% | 0% | 0% | 0% | 100%
js0% | 0% | 0% | 0% | 100%
tsx0% | 0% | 0% | 0% | 100%
toml0% | 0% | 0% | 0% | 100%
css0% | 0% | 0% | 0% | 0%
html0% | 0% | 0% | 0% | 0%
File Change Frequency per Logical Decomposition
primary
primary (file change frequency)
The number of recorded file updates
101+
51-100
21-50
6-20
1-5
project0% | 0% | 0% | 6% | 93%
ROOT0% | 0% | 0% | 100% | 0%
Most Frequently Changed Files (Top 50)

See data for all files...

File# lines# unitscreatedlast modified# changes
(days)		# contributorsline
churn		first
contributor		latest
contributor
eval.py
in project/paperbench/paperbench/nano
 284 7 2025-04-04 2025-12-06 12 3 +479 / -1290 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench/nano
 68 3 2025-04-04 2025-12-01 8 2 +237 / -271 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench
 171 21 2025-04-08 2025-12-01 8 2 +206 / -119 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
alcatraz.py
in project/paperbench/paperbench/infra
 250 - 2025-04-08 2025-12-01 8 3 +381 / -149 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
43 - 2025-07-17 2026-04-21 6 3 +51 / -21 giulio.starace@gmail.com 146999853+mcgrew-oai@users....
task.py
in project/paperbench/paperbench/nano
 401 7 2025-07-17 2025-12-01 6 1 +769 / -279 giulio.starace@gmail.com giulio.starace@gmail.com
simple.py
in project/paperbench/paperbench/judge
 563 10 2025-07-17 2025-12-01 6 1 +1018 / -308 giulio.starace@gmail.com giulio.starace@gmail.com
run_judge.py
in project/paperbench/paperbench/scripts
 97 1 2025-07-15 2025-12-01 5 1 +89 / -195 giulio.starace@gmail.com giulio.starace@gmail.com
structs.py
in project/paperbench/paperbench/nano
 180 6 2025-07-17 2025-12-01 5 1 +294 / -61 giulio.starace@gmail.com giulio.starace@gmail.com
grade.py
in project/paperbench/paperbench
 184 3 2025-07-17 2025-12-01 5 1 +265 / -39 giulio.starace@gmail.com giulio.starace@gmail.com
run_monitor.py
in project/paperbench/paperbench/scripts
 187 2 2025-04-08 2025-12-01 5 2 +112 / -70 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
run_judge_eval.py
in project/paperbench/paperbench/scripts
 247 3 2025-07-15 2025-12-01 5 1 +214 / -148 giulio.starace@gmail.com giulio.starace@gmail.com
eval.py
in project/swelancer/swelancer
 548 8 2025-07-17 2025-12-02 5 2 +676 / -9 giulio.starace@gmail.com giulio.starace@gmail.com
entrypoint.py
in project/paperbench/paperbench/nano
 19 - 2025-04-08 2025-07-18 4 2 +15 / -22 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/dummy
 64 1 2025-07-17 2025-12-01 4 1 +313 / -233 giulio.starace@gmail.com giulio.starace@gmail.com
monitor.py
in project/paperbench/paperbench/monitor
 132 11 2025-07-15 2025-12-01 4 1 +53 / -19 giulio.starace@gmail.com giulio.starace@gmail.com
alcatraz_computer_interface.py
in project/common/nanoeval_alcatraz/nanoeval_alcatraz
 155 2 2025-09-24 2025-12-02 4 2 +262 / -35 simon@openai.com giulio.starace@gmail.com
metrics.py
in project/paperbench/paperbench
 231 11 2025-04-04 2025-07-17 4 2 +116 / -41 giulio.starace@gmail.com giulio.starace@gmail.com
tasks.py
in project/paperbench/paperbench/rubric
 285 29 2025-07-15 2025-12-01 4 1 +118 / -58 giulio.starace@gmail.com giulio.starace@gmail.com
main.py
in project/paperbench/paperbench/solvers/dummy
 12 - 2025-07-17 2025-12-01 3 1 +48 / -29 giulio.starace@gmail.com giulio.starace@gmail.com
run_swelancer.py
in project/swelancer/swelancer
 30 - 2025-07-17 2025-08-28 3 1 +50 / -12 giulio.starace@gmail.com giulio.starace@gmail.com
logging.py
in project/paperbench/paperbench/nano
 41 3 2025-07-15 2025-07-18 3 1 +76 / -4 giulio.starace@gmail.com giulio.starace@gmail.com
create_judge.py
in project/paperbench/paperbench/judge
 43 3 2025-07-15 2025-07-17 3 1 +23 / -38 giulio.starace@gmail.com giulio.starace@gmail.com
task_to_alcatraz_config.py
in project/common/nanoeval_alcatraz/nanoeval_alcatraz
 67 1 2025-09-24 2025-12-02 3 2 +88 / -4 simon@openai.com giulio.starace@gmail.com
registry.py
in project/paperbench/paperbench/judge/judge_eval
 73 7 2025-07-15 2025-12-01 3 1 +16 / -12 giulio.starace@gmail.com giulio.starace@gmail.com
paper_registry.py
in project/paperbench/paperbench
 76 5 2025-07-15 2025-12-01 3 1 +13 / -6 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/direct_submission
 95 7 2025-07-18 2025-12-01 3 1 +310 / -156 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench/judge
 117 5 2025-07-15 2025-07-18 3 1 +103 / -37 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/swelancer/swelancer/solvers/dummy
 124 1 2025-07-17 2025-11-28 3 1 +181 / -20 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/common/preparedness_turn_completer/preparedness_turn_completer
 125 5 2025-09-24 2025-11-28 3 3 +157 / -1 simon@openai.com giulio.starace@gmail.com
base.py
in project/paperbench/paperbench/solvers
 178 1 2025-07-18 2025-12-01 3 1 +352 / -116 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/swelancer/swelancer/solvers/swelancer_agent
 291 4 2025-07-17 2025-11-28 3 1 +402 / -26 giulio.starace@gmail.com giulio.starace@gmail.com
config.yaml
in project/evmbench/audits/2026-01-tempo-feeamm
 12 - 2026-02-18 2026-03-04 2 1 +14 / -1 justin.wang@c-openai.com justin.wang@c-openai.com
config.yaml
in project/evmbench/audits/2026-01-tempo-mpp-streams
 14 - 2026-02-18 2026-03-04 2 1 +24 / -8 justin.wang@c-openai.com justin.wang@c-openai.com
config.yaml
in project/evmbench/audits/2026-01-tempo-stablecoin-dex
 23 - 2026-02-18 2026-03-04 2 1 +48 / -11 justin.wang@c-openai.com justin.wang@c-openai.com
custom_logging.py
in project/swelancer/swelancer/utils
 48 5 2025-07-17 2025-07-18 2 1 +90 / -2 giulio.starace@gmail.com giulio.starace@gmail.com
download_data.py
in project/paperbench/paperbench/judge/judge_eval
 70 4 2025-07-17 2025-10-31 2 1 +2 / -3 giulio.starace@gmail.com giulio.starace@gmail.com
evaluate.py
in project/paperbench/paperbench/judge/judge_eval
 102 6 2025-07-15 2025-07-17 2 1 +108 / -51 giulio.starace@gmail.com giulio.starace@gmail.com
oai_completions_turn_completer.py
in project/common/preparedness_turn_completer/preparedness_turn_completer
 118 5 2025-09-24 2025-11-28 2 2 +150 / -8 simon@openai.com giulio.starace@gmail.com
completer.py
in project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer
 131 5 2025-09-24 2025-11-28 2 2 +172 / -10 simon@openai.com giulio.starace@gmail.com
base.py
in project/paperbench/paperbench/judge
 143 3 2025-07-17 2025-07-18 2 1 +211 / -15 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/basicagent
 280 4 2025-12-01 2025-12-02 2 1 +341 / -1 giulio.starace@gmail.com giulio.starace@gmail.com
converters.py
in project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer
 383 18 2025-09-24 2025-11-28 2 2 +448 / -13 simon@openai.com giulio.starace@gmail.com
local.py
in project/common/alcatraz/alcatraz/clusters
 1590 15 2025-09-24 2025-12-02 2 2 +2427 / -336 simon@openai.com giulio.starace@gmail.com
__init__.py
in project/evmbench/evmbench/experiments
 1 - 2026-02-18 2026-02-18 1 1 +1 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
__init__.py
in project/evmbench/evmbench/nano
 1 - 2026-02-18 2026-02-18 1 1 - justin.wang@c-openai.com justin.wang@c-openai.com
__init__.py
in project/evmbench/evmbench/agents
 1 - 2026-02-18 2026-02-18 1 1 - justin.wang@c-openai.com justin.wang@c-openai.com
__init__.py
in project/swelancer/swelancer
 1 - 2025-07-17 2025-07-17 1 1 - giulio.starace@gmail.com giulio.starace@gmail.com
__init__.py
in project/common/alcatraz/alcatraz
 1 - 2025-12-02 2025-12-02 1 1 - giulio.starace@gmail.com giulio.starace@gmail.com
__init__.py
in project/common/alcatraz/alcatraz/clusters
 1 - 2025-12-02 2025-12-02 1 1 - giulio.starace@gmail.com giulio.starace@gmail.com
Files With Most Churn (Top 50)
Based on the number of lines added and deleted across all commits.

See data for all files...

File# lines# unitscreatedlast modified# changes
(days)		# contributorsline
churn		first
contributor		latest
contributor
local.py
in project/common/alcatraz/alcatraz/clusters
 1590 15 2025-09-24 2025-12-02 2 2 +2427 / -336 simon@openai.com giulio.starace@gmail.com
eval.py
in project/paperbench/paperbench/nano
 284 7 2025-04-04 2025-12-06 12 3 +479 / -1290 giulio.starace@gmail.com giulio.starace@gmail.com
simple.py
in project/paperbench/paperbench/judge
 563 10 2025-07-17 2025-12-01 6 1 +1018 / -308 giulio.starace@gmail.com giulio.starace@gmail.com
stats.py
in project/evmbench/scripts
 1115 45 2026-02-18 2026-02-18 1 1 +1273 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
replay.py
in project/swelancer/runtime_scripts
 726 24 2025-07-17 2025-07-17 1 1 +1086 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
task.py
in project/paperbench/paperbench/nano
 401 7 2025-07-17 2025-12-01 6 1 +769 / -279 giulio.starace@gmail.com giulio.starace@gmail.com
setup.rs
in project/evmbench/ploit/crates/core/src
 450 9 2026-02-18 2026-02-18 1 1 +721 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
eval.py
in project/swelancer/swelancer
 548 8 2025-07-17 2025-12-02 5 2 +676 / -9 giulio.starace@gmail.com giulio.starace@gmail.com
task.py
in project/evmbench/evmbench/nano
 510 5 2026-02-18 2026-02-18 1 1 +610 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
patch.py
in project/evmbench/evmbench/nano/grade
 477 7 2026-02-18 2026-02-18 1 1 +600 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
_executor_worker.py
in project/common/nanoeval/nanoeval
 376 8 2025-09-24 2025-09-24 1 1 +585 / -0 simon@openai.com simon@openai.com
gateway.py
in project/evmbench/evmbench/nano
 330 3 2026-02-18 2026-02-18 1 1 +553 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
solver.py
in project/paperbench/paperbench/solvers/dummy
 64 1 2025-07-17 2025-12-01 4 1 +313 / -233 giulio.starace@gmail.com giulio.starace@gmail.com
alcatraz.py
in project/paperbench/paperbench/infra
 250 - 2025-04-08 2025-12-01 8 3 +381 / -149 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
solver.py
in project/evmbench/evmbench/nano
 443 2 2026-02-18 2026-02-18 1 1 +523 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
utils.py
in project/paperbench/paperbench/nano
 68 3 2025-04-04 2025-12-01 8 2 +237 / -271 giulio.starace@gmail.com giulio.starace@gmail.com
apply_patch.py
in project/paperbench/paperbench/solvers
 359 24 2025-12-01 2025-12-01 1 1 +489 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
local.rs
in project/evmbench/ploit/crates/exec/src
 347 4 2026-02-18 2026-02-18 1 1 +479 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
base.py
in project/paperbench/paperbench/solvers
 178 1 2025-07-18 2025-12-01 3 1 +352 / -116 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/direct_submission
 95 7 2025-07-18 2025-12-01 3 1 +310 / -156 giulio.starace@gmail.com giulio.starace@gmail.com
transactions.rs
in project/evmbench/ploit/crates/core/src
 311 6 2026-02-18 2026-02-18 1 1 +466 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
converters.py
in project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer
 383 18 2025-09-24 2025-11-28 2 2 +448 / -13 simon@openai.com giulio.starace@gmail.com
solver.py
in project/swelancer/swelancer/solvers/swelancer_agent
 291 4 2025-07-17 2025-11-28 3 1 +402 / -26 giulio.starace@gmail.com giulio.starace@gmail.com
evaluation.py
in project/common/nanoeval/nanoeval
 285 1 2025-09-24 2025-09-24 1 1 +412 / -0 simon@openai.com simon@openai.com
alcatraz.py
in project/evmbench/evmbench
 292 2 2026-02-18 2026-02-18 1 1 +403 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
solver.py
in project/common/nanoeval/nanoeval/solvers/computer_tasks
 287 11 2025-09-24 2025-09-24 1 1 +400 / -0 simon@openai.com simon@openai.com
code_execution_interface.py
in project/common/nanoeval/nanoeval/solvers/computer_tasks
 190 7 2025-09-24 2025-09-24 1 1 +383 / -0 simon@openai.com simon@openai.com
utils.py
in project/paperbench/paperbench/solvers/basicagent
 268 5 2025-12-01 2025-12-01 1 1 +382 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
smoke_test_agent_clis.py
in project/evmbench/scripts
 299 7 2026-02-18 2026-02-18 1 1 +370 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
run_judge_eval.py
in project/paperbench/paperbench/scripts
 247 3 2025-07-15 2025-12-01 5 1 +214 / -148 giulio.starace@gmail.com giulio.starace@gmail.com
structs.py
in project/paperbench/paperbench/nano
 180 6 2025-07-17 2025-12-01 5 1 +294 / -61 giulio.starace@gmail.com giulio.starace@gmail.com
config.rs
in project/evmbench/ploit/crates/config/src
 220 6 2026-02-18 2026-02-18 1 1 +351 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
reproduce.py
in project/paperbench/paperbench
 259 2 2025-12-01 2025-12-01 1 1 +343 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
manager.rs
in project/evmbench/ploit/crates/config/src
 278 4 2026-02-18 2026-02-18 1 1 +342 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
solver.py
in project/paperbench/paperbench/solvers/basicagent
 280 4 2025-12-01 2025-12-02 2 1 +341 / -1 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench
 171 21 2025-04-08 2025-12-01 8 2 +206 / -119 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
result.rs
in project/evmbench/ploit/crates/grader/src
 238 10 2026-02-18 2026-02-18 1 1 +319 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
veto.py
in project/evmbench/evmbench/ploit
 245 2 2026-02-18 2026-02-18 1 1 +316 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
eval.py
in project/common/nanoeval/nanoeval
 183 11 2025-09-24 2025-09-24 1 1 +310 / -0 simon@openai.com simon@openai.com
basic.py
in project/paperbench/paperbench/solvers/basicagent/tools
 248 10 2025-12-01 2025-12-01 1 1 +309 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
grade.py
in project/paperbench/paperbench
 184 3 2025-07-17 2025-12-01 5 1 +265 / -39 giulio.starace@gmail.com giulio.starace@gmail.com
alcatraz_computer_interface.py
in project/common/nanoeval_alcatraz/nanoeval_alcatraz
 155 2 2025-09-24 2025-12-02 4 2 +262 / -35 simon@openai.com giulio.starace@gmail.com
server.rs
in project/evmbench/veto/crates/core/src
 256 6 2026-02-18 2026-02-18 1 1 +296 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
run_judge.py
in project/paperbench/paperbench/scripts
 97 1 2025-07-15 2025-12-01 5 1 +89 / -195 giulio.starace@gmail.com giulio.starace@gmail.com
count_disclosures.py
in project/evmbench/scripts
 238 8 2026-02-18 2026-02-18 1 1 +273 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
monitor.py
in project/common/nanoeval/nanoeval
 179 5 2025-09-24 2025-09-24 1 1 +265 / -0 simon@openai.com simon@openai.com
generate_incorrect_findings.py
in project/evmbench/evmbench/experiments
 135 - 2026-02-18 2026-02-18 1 1 +262 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
library_config.py
in project/common/nanoeval/nanoeval
 173 25 2025-09-24 2025-09-24 1 1 +255 / -0 simon@openai.com simon@openai.com
script.rs
in project/evmbench/ploit/crates/grader/src
 182 6 2026-02-18 2026-02-18 1 1 +255 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
intense_debug_logging.py
in project/swelancer/runtime_utils
 129 7 2025-07-17 2025-07-17 1 1 +253 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
Files With Most Contributors (Top 50)
Based on the number of unique email addresses found in commits.

See data for all files...

File# lines# unitscreatedlast modified# changes
(days)		# contributorsline
churn		first
contributor		latest
contributor
eval.py
in project/paperbench/paperbench/nano
 284 7 2025-04-04 2025-12-06 12 3 +479 / -1290 giulio.starace@gmail.com giulio.starace@gmail.com
alcatraz.py
in project/paperbench/paperbench/infra
 250 - 2025-04-08 2025-12-01 8 3 +381 / -149 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
43 - 2025-07-17 2026-04-21 6 3 +51 / -21 giulio.starace@gmail.com 146999853+mcgrew-oai@users....
utils.py
in project/common/preparedness_turn_completer/preparedness_turn_completer
 125 5 2025-09-24 2025-11-28 3 3 +157 / -1 simon@openai.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench
 171 21 2025-04-08 2025-12-01 8 2 +206 / -119 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench/nano
 68 3 2025-04-04 2025-12-01 8 2 +237 / -271 giulio.starace@gmail.com giulio.starace@gmail.com
run_monitor.py
in project/paperbench/paperbench/scripts
 187 2 2025-04-08 2025-12-01 5 2 +112 / -70 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
eval.py
in project/swelancer/swelancer
 548 8 2025-07-17 2025-12-02 5 2 +676 / -9 giulio.starace@gmail.com giulio.starace@gmail.com
entrypoint.py
in project/paperbench/paperbench/nano
 19 - 2025-04-08 2025-07-18 4 2 +15 / -22 oliver.jaffe@hotmail.co.uk giulio.starace@gmail.com
metrics.py
in project/paperbench/paperbench
 231 11 2025-04-04 2025-07-17 4 2 +116 / -41 giulio.starace@gmail.com giulio.starace@gmail.com
alcatraz_computer_interface.py
in project/common/nanoeval_alcatraz/nanoeval_alcatraz
 155 2 2025-09-24 2025-12-02 4 2 +262 / -35 simon@openai.com giulio.starace@gmail.com
task_to_alcatraz_config.py
in project/common/nanoeval_alcatraz/nanoeval_alcatraz
 67 1 2025-09-24 2025-12-02 3 2 +88 / -4 simon@openai.com giulio.starace@gmail.com
completer.py
in project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer
 131 5 2025-09-24 2025-11-28 2 2 +172 / -10 simon@openai.com giulio.starace@gmail.com
converters.py
in project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer
 383 18 2025-09-24 2025-11-28 2 2 +448 / -13 simon@openai.com giulio.starace@gmail.com
oai_completions_turn_completer.py
in project/common/preparedness_turn_completer/preparedness_turn_completer
 118 5 2025-09-24 2025-11-28 2 2 +150 / -8 simon@openai.com giulio.starace@gmail.com
local.py
in project/common/alcatraz/alcatraz/clusters
 1590 15 2025-09-24 2025-12-02 2 2 +2427 / -336 simon@openai.com giulio.starace@gmail.com
simple.py
in project/paperbench/paperbench/judge
 563 10 2025-07-17 2025-12-01 6 1 +1018 / -308 giulio.starace@gmail.com giulio.starace@gmail.com
task.py
in project/paperbench/paperbench/nano
 401 7 2025-07-17 2025-12-01 6 1 +769 / -279 giulio.starace@gmail.com giulio.starace@gmail.com
grade.py
in project/paperbench/paperbench
 184 3 2025-07-17 2025-12-01 5 1 +265 / -39 giulio.starace@gmail.com giulio.starace@gmail.com
run_judge.py
in project/paperbench/paperbench/scripts
 97 1 2025-07-15 2025-12-01 5 1 +89 / -195 giulio.starace@gmail.com giulio.starace@gmail.com
run_judge_eval.py
in project/paperbench/paperbench/scripts
 247 3 2025-07-15 2025-12-01 5 1 +214 / -148 giulio.starace@gmail.com giulio.starace@gmail.com
structs.py
in project/paperbench/paperbench/nano
 180 6 2025-07-17 2025-12-01 5 1 +294 / -61 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/dummy
 64 1 2025-07-17 2025-12-01 4 1 +313 / -233 giulio.starace@gmail.com giulio.starace@gmail.com
monitor.py
in project/paperbench/paperbench/monitor
 132 11 2025-07-15 2025-12-01 4 1 +53 / -19 giulio.starace@gmail.com giulio.starace@gmail.com
tasks.py
in project/paperbench/paperbench/rubric
 285 29 2025-07-15 2025-12-01 4 1 +118 / -58 giulio.starace@gmail.com giulio.starace@gmail.com
base.py
in project/paperbench/paperbench/solvers
 178 1 2025-07-18 2025-12-01 3 1 +352 / -116 giulio.starace@gmail.com giulio.starace@gmail.com
main.py
in project/paperbench/paperbench/solvers/dummy
 12 - 2025-07-17 2025-12-01 3 1 +48 / -29 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/direct_submission
 95 7 2025-07-18 2025-12-01 3 1 +310 / -156 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench/judge
 117 5 2025-07-15 2025-07-18 3 1 +103 / -37 giulio.starace@gmail.com giulio.starace@gmail.com
create_judge.py
in project/paperbench/paperbench/judge
 43 3 2025-07-15 2025-07-17 3 1 +23 / -38 giulio.starace@gmail.com giulio.starace@gmail.com
registry.py
in project/paperbench/paperbench/judge/judge_eval
 73 7 2025-07-15 2025-12-01 3 1 +16 / -12 giulio.starace@gmail.com giulio.starace@gmail.com
logging.py
in project/paperbench/paperbench/nano
 41 3 2025-07-15 2025-07-18 3 1 +76 / -4 giulio.starace@gmail.com giulio.starace@gmail.com
paper_registry.py
in project/paperbench/paperbench
 76 5 2025-07-15 2025-12-01 3 1 +13 / -6 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/swelancer/swelancer/solvers/dummy
 124 1 2025-07-17 2025-11-28 3 1 +181 / -20 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/swelancer/swelancer/solvers/swelancer_agent
 291 4 2025-07-17 2025-11-28 3 1 +402 / -26 giulio.starace@gmail.com giulio.starace@gmail.com
run_swelancer.py
in project/swelancer/swelancer
 30 - 2025-07-17 2025-08-28 3 1 +50 / -12 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/paperbench/paperbench/solvers/basicagent
 280 4 2025-12-01 2025-12-02 2 1 +341 / -1 giulio.starace@gmail.com giulio.starace@gmail.com
base.py
in project/paperbench/paperbench/judge
 143 3 2025-07-17 2025-07-18 2 1 +211 / -15 giulio.starace@gmail.com giulio.starace@gmail.com
evaluate.py
in project/paperbench/paperbench/judge/judge_eval
 102 6 2025-07-15 2025-07-17 2 1 +108 / -51 giulio.starace@gmail.com giulio.starace@gmail.com
download_data.py
in project/paperbench/paperbench/judge/judge_eval
 70 4 2025-07-17 2025-10-31 2 1 +2 / -3 giulio.starace@gmail.com giulio.starace@gmail.com
custom_logging.py
in project/swelancer/swelancer/utils
 48 5 2025-07-17 2025-07-18 2 1 +90 / -2 giulio.starace@gmail.com giulio.starace@gmail.com
config.yaml
in project/evmbench/audits/2026-01-tempo-feeamm
 12 - 2026-02-18 2026-03-04 2 1 +14 / -1 justin.wang@c-openai.com justin.wang@c-openai.com
config.yaml
in project/evmbench/audits/2026-01-tempo-stablecoin-dex
 23 - 2026-02-18 2026-03-04 2 1 +48 / -11 justin.wang@c-openai.com justin.wang@c-openai.com
config.yaml
in project/evmbench/audits/2026-01-tempo-mpp-streams
 14 - 2026-02-18 2026-03-04 2 1 +24 / -8 justin.wang@c-openai.com justin.wang@c-openai.com
config.yaml
in project/paperbench/data/papers/stochastic-interpolants
 2 - 2025-04-08 2025-04-08 1 1 +2 / -0 129281094+james-aung@users.... 129281094+james-aung@users....
utils.py
in project/paperbench/paperbench/solvers
 149 5 2025-12-01 2025-12-01 1 1 +198 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
upload.py
in project/paperbench/paperbench/solvers
 124 - 2025-12-01 2025-12-01 1 1 +142 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
utils.py
in project/paperbench/paperbench/solvers/basicagent
 268 5 2025-12-01 2025-12-01 1 1 +382 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
completer.py
in project/paperbench/paperbench/solvers/basicagent
 68 5 2025-12-01 2025-12-01 1 1 +110 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
base.py
in project/paperbench/paperbench/solvers/basicagent/tools
 21 2 2025-12-01 2025-12-01 1 1 +53 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
Files With Least Contributors (Top 50)
Based on the number of unique email addresses found in commits.

See data for all files...

File# lines# unitscreatedlast modified# changes
(days)		# contributorsline
churn		first
contributor		latest
contributor
script.js
in project/paperbench/paperbench/gui/static
 345 18
styles.css
in project/paperbench/paperbench/gui/static
 100 -
rubric.html
in project/paperbench/paperbench/gui/templates
 38 -
macros.html
in project/paperbench/paperbench/gui/templates
 33 -
constants.py
in project/paperbench/paperbench/judge
 15 2
base.html
in project/paperbench/paperbench/gui/templates
 15 -
config.yaml
in project/paperbench/data/papers/ftrl
 2 -
config.yaml
in project/paperbench/data/papers/lbcs
 2 -
config.yaml
in project/paperbench/data/papers/pinn
 2 -
config.yaml
in project/paperbench/data/papers/sequential-neural-score-estimation
 2 -
config.yaml
in project/paperbench/data/papers/fre
 2 -
config.yaml
in project/paperbench/data/papers/adaptive-pruning
 2 -
config.yaml
in project/paperbench/data/papers/test-time-model-adaptation
 2 -
config.yaml
in project/paperbench/data/papers/semantic-self-consistency
 2 -
config.yaml
in project/paperbench/data/papers/what-will-my-model-forget
 2 -
config.yaml
in project/paperbench/data/papers/bam
 2 -
config.yaml
in project/paperbench/data/papers/stay-on-topic-with-classifier-free-guidance
 2 -
config.yaml
in project/paperbench/data/papers/self-composing-policies
 2 -
config.yaml
in project/paperbench/data/papers/all-in-one
 2 -
config.yaml
in project/paperbench/data/papers/bbox
 2 -
config.yaml
in project/paperbench/data/papers/robust-clip
 2 -
config.yaml
in project/paperbench/data/papers/self-expansion
 2 -
config.yaml
in project/paperbench/data/papers/sample-specific-masks
 2 -
config.yaml
in project/paperbench/data/papers/lca-on-the-line
 2 -
config.yaml
in project/paperbench/data/papers/sapg
 2 -
config.yaml
in project/paperbench/data/papers/mechanistic-understanding
 2 -
config.yaml
in project/paperbench/data/papers/rice
 2 -
config.yaml
in project/paperbench/data/papers/bridging-data-gaps
 2 -
__init__.py
in project/paperbench/paperbench/rubric
 1 -
stats.py
in project/evmbench/scripts
 1115 45 2026-02-18 2026-02-18 1 1 +1273 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
replay.py
in project/swelancer/runtime_scripts
 726 24 2025-07-17 2025-07-17 1 1 +1086 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
simple.py
in project/paperbench/paperbench/judge
 563 10 2025-07-17 2025-12-01 6 1 +1018 / -308 giulio.starace@gmail.com giulio.starace@gmail.com
task.py
in project/evmbench/evmbench/nano
 510 5 2026-02-18 2026-02-18 1 1 +610 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
patch.py
in project/evmbench/evmbench/nano/grade
 477 7 2026-02-18 2026-02-18 1 1 +600 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
setup.rs
in project/evmbench/ploit/crates/core/src
 450 9 2026-02-18 2026-02-18 1 1 +721 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
solver.py
in project/evmbench/evmbench/nano
 443 2 2026-02-18 2026-02-18 1 1 +523 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
app.py
in project/paperbench/paperbench/gui
 420 14 2025-07-17 2025-07-17 1 1 +17 / -16 giulio.starace@gmail.com giulio.starace@gmail.com
task.py
in project/paperbench/paperbench/nano
 401 7 2025-07-17 2025-12-01 6 1 +769 / -279 giulio.starace@gmail.com giulio.starace@gmail.com
_executor_worker.py
in project/common/nanoeval/nanoeval
 376 8 2025-09-24 2025-09-24 1 1 +585 / -0 simon@openai.com simon@openai.com
apply_patch.py
in project/paperbench/paperbench/solvers
 359 24 2025-12-01 2025-12-01 1 1 +489 / -0 giulio.starace@gmail.com giulio.starace@gmail.com
local.rs
in project/evmbench/ploit/crates/exec/src
 347 4 2026-02-18 2026-02-18 1 1 +479 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
gateway.py
in project/evmbench/evmbench/nano
 330 3 2026-02-18 2026-02-18 1 1 +553 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
transactions.rs
in project/evmbench/ploit/crates/core/src
 311 6 2026-02-18 2026-02-18 1 1 +466 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
smoke_test_agent_clis.py
in project/evmbench/scripts
 299 7 2026-02-18 2026-02-18 1 1 +370 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
alcatraz.py
in project/evmbench/evmbench
 292 2 2026-02-18 2026-02-18 1 1 +403 / -0 justin.wang@c-openai.com justin.wang@c-openai.com
solver.py
in project/swelancer/swelancer/solvers/swelancer_agent
 291 4 2025-07-17 2025-11-28 3 1 +402 / -26 giulio.starace@gmail.com giulio.starace@gmail.com
solver.py
in project/common/nanoeval/nanoeval/solvers/computer_tasks
 287 11 2025-09-24 2025-09-24 1 1 +400 / -0 simon@openai.com simon@openai.com
tasks.py
in project/paperbench/paperbench/rubric
 285 29 2025-07-15 2025-12-01 4 1 +118 / -58 giulio.starace@gmail.com giulio.starace@gmail.com
evaluation.py
in project/common/nanoeval/nanoeval
 285 1 2025-09-24 2025-09-24 1 1 +412 / -0 simon@openai.com simon@openai.com
solver.py
in project/paperbench/paperbench/solvers/basicagent
 280 4 2025-12-01 2025-12-02 2 1 +341 / -1 giulio.starace@gmail.com giulio.starace@gmail.com
Correlations

File Size vs. Number of Changes: 265 points

.pre-commit-config.yaml x: 43 lines of code y: 6 # changes project/evmbench/audits/2026-01-tempo-feeamm/config.yaml x: 12 lines of code y: 2 # changes project/evmbench/audits/2026-01-tempo-stablecoin-dex/config.yaml x: 23 lines of code y: 2 # changes project/evmbench/audits/2023-07-pooltogether/config.yaml x: 25 lines of code y: 1 # changes project/evmbench/audits/2023-10-nextgen/config.yaml x: 27 lines of code y: 1 # changes project/evmbench/audits/2024-01-canto/config.yaml x: 8 lines of code y: 1 # changes project/evmbench/audits/2024-01-curves/config.yaml x: 35 lines of code y: 1 # changes project/evmbench/audits/2024-01-init-capital-invitational/config.yaml x: 11 lines of code y: 1 # changes project/evmbench/audits/2024-01-renft/config.yaml x: 65 lines of code y: 1 # changes project/evmbench/audits/2024-02-althea-liquid-infrastructure/config.yaml x: 5 lines of code y: 1 # changes project/evmbench/audits/2024-03-abracadabra-money/config.yaml x: 15 lines of code y: 1 # changes project/evmbench/audits/2024-03-taiko/config.yaml x: 44 lines of code y: 1 # changes project/evmbench/audits/2024-04-noya/config.yaml x: 74 lines of code y: 1 # changes project/evmbench/audits/2024-06-size/config.yaml x: 46 lines of code y: 1 # changes project/evmbench/audits/2024-06-thorchain/avalanche/hardhat.config.js x: 130 lines of code y: 1 # changes project/evmbench/audits/2024-06-thorchain/ethereum/hardhat.config.js x: 39 lines of code y: 1 # changes project/evmbench/audits/2024-07-traitforge/config.yaml x: 20 lines of code y: 1 # changes project/evmbench/audits/2024-08-phi/config.yaml x: 58 lines of code y: 1 # changes project/evmbench/audits/2024-12-secondswap/config.yaml x: 13 lines of code y: 1 # changes project/evmbench/audits/2025-04-virtuals/config.yaml x: 34 lines of code y: 1 # changes project/evmbench/audits/2025-06-panoptic/config.yaml x: 29 lines of code y: 1 # changes project/evmbench/docker_build.py x: 118 lines of code y: 1 # changes project/evmbench/evmbench/agents/__init__.py x: 1 lines of code y: 1 # changes project/evmbench/evmbench/agents/agent.py x: 127 lines of code y: 1 # changes project/evmbench/evmbench/agents/codex/config.yaml x: 95 lines of code y: 1 # changes project/evmbench/evmbench/agents/opencode/config.yaml x: 51 lines of code y: 1 # changes project/evmbench/evmbench/agents/run.py x: 75 lines of code y: 1 # changes project/evmbench/evmbench/alcatraz.py x: 292 lines of code y: 1 # changes project/evmbench/evmbench/audit.py x: 173 lines of code y: 1 # changes project/evmbench/evmbench/bootstrap.py x: 136 lines of code y: 1 # changes project/evmbench/evmbench/nano/eval.py x: 204 lines of code y: 1 # changes project/evmbench/evmbench/nano/gateway.py x: 330 lines of code y: 1 # changes project/evmbench/evmbench/nano/grade/detect.py x: 102 lines of code y: 1 # changes project/evmbench/evmbench/nano/grade/exploit.py x: 195 lines of code y: 1 # changes project/evmbench/evmbench/nano/grade/patch.py x: 477 lines of code y: 1 # changes project/evmbench/evmbench/nano/logging.py x: 41 lines of code y: 1 # changes project/evmbench/evmbench/nano/solver.py x: 443 lines of code y: 1 # changes project/evmbench/evmbench/nano/task.py x: 510 lines of code y: 1 # changes project/evmbench/evmbench/ploit/config.py x: 179 lines of code y: 1 # changes project/evmbench/evmbench/ploit/veto.py x: 245 lines of code y: 1 # changes project/evmbench/ploit/crates/config/src/config.rs x: 220 lines of code y: 1 # changes project/evmbench/ploit/crates/config/src/lib.rs x: 22 lines of code y: 1 # changes project/evmbench/ploit/crates/config/src/manager.rs x: 278 lines of code y: 1 # changes project/evmbench/ploit/crates/core/src/setup.rs x: 450 lines of code y: 1 # changes project/evmbench/ploit/crates/core/src/transactions.rs x: 311 lines of code y: 1 # changes project/evmbench/ploit/crates/exec/src/local.rs x: 347 lines of code y: 1 # changes project/evmbench/ploit/crates/grader/src/memory.rs x: 192 lines of code y: 1 # changes project/evmbench/ploit/crates/grader/src/result.rs x: 238 lines of code y: 1 # changes project/evmbench/ploit/crates/grader/src/script.rs x: 182 lines of code y: 1 # changes project/evmbench/ploit/crates/utils/src/process.rs x: 104 lines of code y: 1 # changes project/evmbench/scripts/backfill_agent_tokens.py x: 159 lines of code y: 1 # changes project/evmbench/scripts/smoke_test_agent_clis.py x: 299 lines of code y: 1 # changes project/evmbench/scripts/stats.py x: 1115 lines of code y: 1 # changes project/evmbench/veto/crates/core/src/jsonrpc.rs x: 99 lines of code y: 1 # changes project/evmbench/veto/crates/core/src/server.rs x: 256 lines of code y: 1 # changes project/paperbench/paperbench/nano/eval.py x: 284 lines of code y: 12 # changes project/common/alcatraz/alcatraz/clusters/local.py x: 1590 lines of code y: 2 # changes project/swelancer/swelancer/eval.py x: 548 lines of code y: 5 # changes project/common/nanoeval_alcatraz/nanoeval_alcatraz/alcatraz_computer_interface.py x: 155 lines of code y: 4 # changes project/common/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py x: 67 lines of code y: 3 # changes project/paperbench/paperbench/solvers/basicagent/solver.py x: 280 lines of code y: 2 # changes project/paperbench/paperbench/nano/task.py x: 401 lines of code y: 6 # changes project/paperbench/paperbench/reproduce.py x: 259 lines of code y: 1 # changes project/paperbench/paperbench/solvers/apply_patch.py x: 359 lines of code y: 1 # changes project/paperbench/paperbench/solvers/base.py x: 178 lines of code y: 3 # changes project/paperbench/paperbench/solvers/basicagent/api.py x: 79 lines of code y: 1 # changes project/paperbench/paperbench/solvers/basicagent/completer.py x: 68 lines of code y: 1 # changes project/paperbench/paperbench/solvers/basicagent/tools/basic.py x: 248 lines of code y: 1 # changes project/paperbench/paperbench/solvers/basicagent/utils.py x: 268 lines of code y: 1 # changes project/paperbench/paperbench/solvers/direct_submission/solver.py x: 95 lines of code y: 3 # changes project/paperbench/paperbench/solvers/dummy/main.py x: 12 lines of code y: 3 # changes project/paperbench/paperbench/solvers/dummy/solver.py x: 64 lines of code y: 4 # changes project/paperbench/paperbench/solvers/upload.py x: 124 lines of code y: 1 # changes project/paperbench/paperbench/solvers/utils.py x: 149 lines of code y: 1 # changes project/paperbench/paperbench/utils.py x: 171 lines of code y: 8 # changes project/paperbench/paperbench/computer_utils.py x: 113 lines of code y: 1 # changes project/paperbench/paperbench/grade.py x: 184 lines of code y: 5 # changes project/paperbench/paperbench/nano/structs.py x: 180 lines of code y: 5 # changes project/paperbench/paperbench/nano/utils.py x: 68 lines of code y: 8 # changes project/paperbench/experiments/judge_eval/judge_eval_perf_cost.py x: 177 lines of code y: 1 # changes project/paperbench/paperbench/monitor/monitor.py x: 132 lines of code y: 4 # changes project/paperbench/paperbench/scripts/run_judge.py x: 97 lines of code y: 5 # changes project/paperbench/paperbench/scripts/run_judge_eval.py x: 247 lines of code y: 5 # changes project/paperbench/paperbench/scripts/run_monitor.py x: 187 lines of code y: 5 # changes project/paperbench/paperbench/infra/alcatraz.py x: 250 lines of code y: 8 # changes project/paperbench/paperbench/judge/judge_eval/registry.py x: 73 lines of code y: 3 # changes project/paperbench/paperbench/paper_registry.py x: 76 lines of code y: 3 # changes project/paperbench/paperbench/rubric/tasks.py x: 285 lines of code y: 4 # changes project/paperbench/paperbench/judge/simple.py x: 563 lines of code y: 6 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/oai_completions_turn_completer.py x: 118 lines of code y: 2 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/completer.py x: 131 lines of code y: 2 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/converters.py x: 383 lines of code y: 2 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/type_helpers.py x: 199 lines of code y: 1 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/utils.py x: 125 lines of code y: 3 # changes project/swelancer/swelancer/solvers/swelancer_agent/solver.py x: 291 lines of code y: 3 # changes project/paperbench/paperbench/judge/judge_eval/download_data.py x: 70 lines of code y: 2 # changes project/common/nanoeval/nanoeval/_executor_worker.py x: 376 lines of code y: 1 # changes project/common/nanoeval/nanoeval/_multiprocessing_utils.py x: 54 lines of code y: 1 # changes project/common/nanoeval/nanoeval/_persistent_db.py x: 107 lines of code y: 1 # changes project/common/nanoeval/nanoeval/asyncio_utils.py x: 121 lines of code y: 1 # changes project/common/nanoeval/nanoeval/evaluation.py x: 285 lines of code y: 1 # changes project/common/nanoeval/nanoeval/recorder_protocol.py x: 62 lines of code y: 1 # changes project/common/nanoeval/nanoeval/solvers/computer_tasks/limits.py x: 56 lines of code y: 1 # changes project/common/nanoeval/nanoeval/solvers/computer_tasks/pausable_timer.py x: 89 lines of code y: 1 # changes project/common/nanoeval/nanoeval/solvers/computer_tasks/simple_solver.py x: 82 lines of code y: 1 # changes project/common/nanoeval/nanoeval/solvers/mcq.py x: 171 lines of code y: 1 # changes project/swelancer/swelancer/run_swelancer.py x: 30 lines of code y: 3 # changes project/paperbench/paperbench/judge/base.py x: 143 lines of code y: 2 # changes project/paperbench/paperbench/judge/utils.py x: 117 lines of code y: 3 # changes project/paperbench/paperbench/nano/entrypoint.py x: 19 lines of code y: 4 # changes project/paperbench/paperbench/nano/logging.py x: 41 lines of code y: 3 # changes project/swelancer/swelancer/utils/custom_logging.py x: 48 lines of code y: 2 # changes project/swelancer/issues/29854_602/user_tool.py x: 92 lines of code y: 1 # changes project/swelancer/issues/32236_652/user_tool.py x: 116 lines of code y: 1 # changes project/swelancer/issues/44804_694/44804_694_VideoPlayerPreview.tsx x: 84 lines of code y: 1 # changes project/swelancer/runtime_scripts/replay.py x: 726 lines of code y: 1 # changes project/swelancer/scripts/build_images.py x: 165 lines of code y: 1 # changes project/paperbench/paperbench/gui/app.py x: 420 lines of code y: 1 # changes project/paperbench/paperbench/judge/judge_eval/evaluate.py x: 102 lines of code y: 2 # changes project/paperbench/paperbench/metrics.py x: 231 lines of code y: 4 # changes
12.0
# changes
  min: 1.0
  average: 1.48
  25th percentile: 1.0
  median: 1.0
  75th percentile: 1.0
  max: 12.0
0 1590.0
lines of code
min: 1.0 | average: 104.75 | 25th percentile: 18.0 | median: 49.0 | 75th percentile: 129.5 | max: 1590.0

Number of Contributors vs. Number of Changes: 265 points

.pre-commit-config.yaml x: 3 # contributors y: 6 # changes project/evmbench/audits/2026-01-tempo-feeamm/config.yaml x: 1 # contributors y: 2 # changes project/evmbench/audits/2023-07-pooltogether/config.yaml x: 1 # contributors y: 1 # changes project/paperbench/paperbench/nano/eval.py x: 3 # contributors y: 12 # changes project/common/alcatraz/alcatraz/clusters/local.py x: 2 # contributors y: 2 # changes project/swelancer/swelancer/eval.py x: 2 # contributors y: 5 # changes project/common/nanoeval_alcatraz/nanoeval_alcatraz/alcatraz_computer_interface.py x: 2 # contributors y: 4 # changes project/common/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py x: 2 # contributors y: 3 # changes project/paperbench/paperbench/nano/task.py x: 1 # contributors y: 6 # changes project/paperbench/paperbench/solvers/base.py x: 1 # contributors y: 3 # changes project/paperbench/paperbench/solvers/dummy/solver.py x: 1 # contributors y: 4 # changes project/paperbench/paperbench/utils.py x: 2 # contributors y: 8 # changes project/paperbench/paperbench/grade.py x: 1 # contributors y: 5 # changes project/paperbench/paperbench/infra/alcatraz.py x: 3 # contributors y: 8 # changes project/common/preparedness_turn_completer/preparedness_turn_completer/utils.py x: 3 # contributors y: 3 # changes
12.0
# changes
  min: 1.0
  average: 1.48
  25th percentile: 1.0
  median: 1.0
  75th percentile: 1.0
  max: 12.0
0 3.0
# contributors
min: 1.0 | average: 1.08 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 3.0

Number of Contributors vs. File Size: 265 points

.pre-commit-config.yaml x: 3 # contributors y: 43 lines of code project/evmbench/audits/2026-01-tempo-feeamm/config.yaml x: 1 # contributors y: 12 lines of code project/evmbench/audits/2026-01-tempo-mpp-streams/config.yaml x: 1 # contributors y: 14 lines of code project/evmbench/audits/2026-01-tempo-stablecoin-dex/config.yaml x: 1 # contributors y: 23 lines of code project/evmbench/audits/2023-10-nextgen/config.yaml x: 1 # contributors y: 27 lines of code project/evmbench/audits/2024-01-curves/config.yaml x: 1 # contributors y: 35 lines of code project/evmbench/audits/2024-01-renft/config.yaml x: 1 # contributors y: 65 lines of code project/evmbench/audits/2024-02-althea-liquid-infrastructure/config.yaml x: 1 # contributors y: 5 lines of code project/evmbench/audits/2024-03-taiko/config.yaml x: 1 # contributors y: 44 lines of code project/evmbench/audits/2024-04-noya/config.yaml x: 1 # contributors y: 74 lines of code project/evmbench/audits/2024-06-thorchain/avalanche/hardhat.config.js x: 1 # contributors y: 130 lines of code project/evmbench/audits/2024-06-thorchain/ethereum/hardhat.config.js x: 1 # contributors y: 39 lines of code project/evmbench/audits/2024-08-phi/config.yaml x: 1 # contributors y: 58 lines of code project/evmbench/docker_build.py x: 1 # contributors y: 118 lines of code project/evmbench/evmbench/agents/codex/config.yaml x: 1 # contributors y: 95 lines of code project/evmbench/evmbench/agents/opencode/config.yaml x: 1 # contributors y: 51 lines of code project/evmbench/evmbench/alcatraz.py x: 1 # contributors y: 292 lines of code project/evmbench/evmbench/audit.py x: 1 # contributors y: 173 lines of code project/evmbench/evmbench/bootstrap.py x: 1 # contributors y: 136 lines of code project/evmbench/evmbench/nano/eval.py x: 1 # contributors y: 204 lines of code project/evmbench/evmbench/nano/gateway.py x: 1 # contributors y: 330 lines of code project/evmbench/evmbench/nano/grade/detect.py x: 1 # contributors y: 102 lines of code project/evmbench/evmbench/nano/grade/exploit.py x: 1 # contributors y: 195 lines of code project/evmbench/evmbench/nano/grade/patch.py x: 1 # contributors y: 477 lines of code project/evmbench/evmbench/nano/solver.py x: 1 # contributors y: 443 lines of code project/evmbench/evmbench/nano/task.py x: 1 # contributors y: 510 lines of code project/evmbench/evmbench/ploit/config.py x: 1 # contributors y: 179 lines of code project/evmbench/evmbench/ploit/veto.py x: 1 # contributors y: 245 lines of code project/evmbench/ploit/crates/config/src/config.rs x: 1 # contributors y: 220 lines of code project/evmbench/ploit/crates/config/src/manager.rs x: 1 # contributors y: 278 lines of code project/evmbench/ploit/crates/core/src/setup.rs x: 1 # contributors y: 450 lines of code project/evmbench/ploit/crates/core/src/transactions.rs x: 1 # contributors y: 311 lines of code project/evmbench/ploit/crates/exec/src/local.rs x: 1 # contributors y: 347 lines of code project/evmbench/ploit/crates/grader/src/memory.rs x: 1 # contributors y: 192 lines of code project/evmbench/ploit/crates/grader/src/result.rs x: 1 # contributors y: 238 lines of code project/evmbench/ploit/crates/grader/src/script.rs x: 1 # contributors y: 182 lines of code project/evmbench/scripts/backfill_agent_tokens.py x: 1 # contributors y: 159 lines of code project/evmbench/scripts/smoke_test_agent_clis.py x: 1 # contributors y: 299 lines of code project/evmbench/scripts/stats.py x: 1 # contributors y: 1115 lines of code project/evmbench/veto/crates/core/src/server.rs x: 1 # contributors y: 256 lines of code project/paperbench/paperbench/nano/eval.py x: 3 # contributors y: 284 lines of code project/common/alcatraz/alcatraz/clusters/local.py x: 2 # contributors y: 1590 lines of code project/swelancer/swelancer/eval.py x: 2 # contributors y: 548 lines of code project/common/nanoeval_alcatraz/nanoeval_alcatraz/alcatraz_computer_interface.py x: 2 # contributors y: 155 lines of code project/common/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py x: 2 # contributors y: 67 lines of code project/paperbench/paperbench/nano/task.py x: 1 # contributors y: 401 lines of code project/paperbench/paperbench/reproduce.py x: 1 # contributors y: 259 lines of code project/paperbench/paperbench/solvers/apply_patch.py x: 1 # contributors y: 359 lines of code project/paperbench/paperbench/solvers/basicagent/api.py x: 1 # contributors y: 79 lines of code project/paperbench/paperbench/solvers/basicagent/utils.py x: 1 # contributors y: 268 lines of code project/paperbench/paperbench/solvers/upload.py x: 1 # contributors y: 124 lines of code project/paperbench/paperbench/solvers/utils.py x: 1 # contributors y: 149 lines of code project/paperbench/paperbench/utils.py x: 2 # contributors y: 171 lines of code project/paperbench/paperbench/scripts/run_monitor.py x: 2 # contributors y: 187 lines of code project/paperbench/paperbench/infra/alcatraz.py x: 3 # contributors y: 250 lines of code project/paperbench/paperbench/rubric/tasks.py x: 1 # contributors y: 285 lines of code project/paperbench/paperbench/judge/simple.py x: 1 # contributors y: 563 lines of code project/common/preparedness_turn_completer/preparedness_turn_completer/oai_completions_turn_completer.py x: 2 # contributors y: 118 lines of code project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/completer.py x: 2 # contributors y: 131 lines of code project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/converters.py x: 2 # contributors y: 383 lines of code project/common/preparedness_turn_completer/preparedness_turn_completer/utils.py x: 3 # contributors y: 125 lines of code project/common/nanoeval/nanoeval/_executor_worker.py x: 1 # contributors y: 376 lines of code project/common/nanoeval/nanoeval/_persistent_db.py x: 1 # contributors y: 107 lines of code project/common/nanoeval/nanoeval/solvers/computer_tasks/pausable_timer.py x: 1 # contributors y: 89 lines of code project/common/nanoeval/nanoeval/solvers/computer_tasks/simple_solver.py x: 1 # contributors y: 82 lines of code project/paperbench/paperbench/judge/base.py x: 1 # contributors y: 143 lines of code project/paperbench/paperbench/nano/entrypoint.py x: 2 # contributors y: 19 lines of code project/swelancer/runtime_scripts/replay.py x: 1 # contributors y: 726 lines of code project/swelancer/scripts/build_images.py x: 1 # contributors y: 165 lines of code project/paperbench/paperbench/gui/app.py x: 1 # contributors y: 420 lines of code project/paperbench/paperbench/metrics.py x: 2 # contributors y: 231 lines of code
1590.0
lines of code
  min: 1.0
  average: 104.75
  25th percentile: 18.0
  median: 49.0
  75th percentile: 129.5
  max: 1590.0
0 3.0
# contributors
min: 1.0 | average: 1.08 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 3.0

Number of Changes vs. Line Churn: 3616 points

.pre-commit-config.yaml x: 6 # changes y: 72 line churn project/evmbench/README.md x: 2 # changes y: 141 line churn .github/workflows/ci.yml x: 1 # changes y: 4 line churn .github/workflows/compatibility_api_lint.yml x: 4 # changes y: 68 line churn .github/workflows/paperbench_lint.yml x: 5 # changes y: 73 line churn .github/workflows/paperbench_tests.yml x: 8 # changes y: 123 line churn .github/workflows/preparedness_turncompleter_lint.yml x: 3 # changes y: 63 line churn project/evmbench/audits/2026-01-tempo-feeamm/test/ExploitH01.t.sol x: 2 # changes y: 980 line churn project/evmbench/audits/2023-07-pooltogether/patch/Vault.sol x: 1 # changes y: 1238 line churn project/evmbench/audits/2023-12-ethereumcreditguild/patch/ERC20RebaseDistributor.sol x: 1 # changes y: 746 line churn project/evmbench/audits/2024-01-renft/test/SetupExploit.sol x: 1 # changes y: 1708 line churn project/evmbench/ploit/Cargo.lock x: 1 # changes y: 4872 line churn project/evmbench/uv.lock x: 1 # changes y: 3532 line churn project/paperbench/paperbench/nano/eval.py x: 12 # changes y: 1769 line churn project/common/alcatraz/alcatraz/clusters/local.py x: 2 # changes y: 2763 line churn project/swelancer/swelancer/eval.py x: 5 # changes y: 685 line churn project/common/alcatraz/uv.lock x: 1 # changes y: 2469 line churn project/paperbench/uv.lock x: 7 # changes y: 10242 line churn project/swelancer/uv.lock x: 3 # changes y: 3391 line churn project/paperbench/README.md x: 10 # changes y: 339 line churn project/paperbench/paperbench/nano/task.py x: 6 # changes y: 1048 line churn project/paperbench/paperbench/solvers/dummy/solver.py x: 4 # changes y: 546 line churn project/paperbench/paperbench/scripts/alcatraz_services.py x: 10 # changes y: 742 line churn project/paperbench/tests/integration/test_run_agent.py x: 5 # changes y: 1559 line churn project/paperbench/paperbench/infra/alcatraz.py x: 8 # changes y: 530 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/CHANGELOG.md x: 2 # changes y: 1936 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/docs/providers.qmd x: 2 # changes y: 1154 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/src/inspect_ai/_cli/eval.py x: 2 # changes y: 2214 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/src/inspect_ai/_view/www/dist/assets/index.css x: 2 # changes y: 39534 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/src/inspect_ai/_view/www/dist/assets/index.js x: 2 # changes y: 132834 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/src/inspect_ai/_view/www/log-schema.json x: 2 # changes y: 10194 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/src/inspect_ai/_view/www/yarn.lock x: 2 # changes y: 3804 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/tools/vscode/tools/ts-to-mjs/yarn.lock x: 2 # changes y: 7522 line churn project/paperbench/paperbench/agents/aisi-basic-agent/inspect_ai/tools/vscode/yarn.lock x: 2 # changes y: 7020 line churn project/paperbench/paperbench/agents/run.py x: 6 # changes y: 857 line churn project/alcatraz/alcatraz/clusters/local.py x: 5 # changes y: 2529 line churn project/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/converters.py x: 3 # changes y: 968 line churn project/preparedness_turn_completer/uv.lock x: 4 # changes y: 5850 line churn project/paperbench/paperbench/judge/judge.py x: 3 # changes y: 1194 line churn project/paperbench/data/papers/bam/rubric.json x: 1 # changes y: 3063 line churn project/paperbench/data/papers/lbcs/rubric.json x: 1 # changes y: 4413 line churn project/paperbench/data/papers/pinn/rubric.json x: 1 # changes y: 7653 line churn
132834.0
line churn
  min: 1.0
  average: 211.0
  25th percentile: 8.0
  median: 64.0
  75th percentile: 158.0
  max: 132834.0
0 12.0
# changes
min: 1.0 | average: 1.49 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 2.0 | max: 12.0

Number of Contributors vs. Line Churn: 249 points

.pre-commit-config.yaml x: 3 # contributors y: 72 line churn project/evmbench/audits/2026-01-tempo-feeamm/config.yaml x: 1 # contributors y: 15 line churn project/evmbench/audits/2026-01-tempo-mpp-streams/config.yaml x: 1 # contributors y: 32 line churn project/evmbench/audits/2026-01-tempo-stablecoin-dex/config.yaml x: 1 # contributors y: 59 line churn project/evmbench/audits/2024-01-canto/config.yaml x: 1 # contributors y: 10 line churn project/evmbench/audits/2024-01-curves/config.yaml x: 1 # contributors y: 39 line churn project/evmbench/audits/2024-01-renft/config.yaml x: 1 # contributors y: 72 line churn project/evmbench/audits/2024-03-taiko/config.yaml x: 1 # contributors y: 49 line churn project/evmbench/audits/2024-04-noya/config.yaml x: 1 # contributors y: 95 line churn project/evmbench/audits/2024-06-thorchain/avalanche/hardhat.config.js x: 1 # contributors y: 134 line churn project/evmbench/docker_build.py x: 1 # contributors y: 155 line churn project/evmbench/evmbench/agents/agent.py x: 1 # contributors y: 151 line churn project/evmbench/evmbench/agents/codex/config.yaml x: 1 # contributors y: 114 line churn project/evmbench/evmbench/alcatraz.py x: 1 # contributors y: 403 line churn project/evmbench/evmbench/audit.py x: 1 # contributors y: 240 line churn project/evmbench/evmbench/bootstrap.py x: 1 # contributors y: 251 line churn project/evmbench/evmbench/experiments/generate_incorrect_findings.py x: 1 # contributors y: 262 line churn project/evmbench/evmbench/nano/gateway.py x: 1 # contributors y: 553 line churn project/evmbench/evmbench/nano/grade/base.py x: 1 # contributors y: 102 line churn project/evmbench/evmbench/nano/grade/exploit.py x: 1 # contributors y: 236 line churn project/evmbench/evmbench/nano/grade/patch.py x: 1 # contributors y: 600 line churn project/evmbench/evmbench/nano/solver.py x: 1 # contributors y: 523 line churn project/evmbench/evmbench/nano/task.py x: 1 # contributors y: 610 line churn project/evmbench/evmbench/ploit/config.py x: 1 # contributors y: 222 line churn project/evmbench/evmbench/ploit/veto.py x: 1 # contributors y: 316 line churn project/evmbench/evmbench/utils.py x: 1 # contributors y: 170 line churn project/evmbench/ploit/crates/config/src/config.rs x: 1 # contributors y: 351 line churn project/evmbench/ploit/crates/config/src/manager.rs x: 1 # contributors y: 342 line churn project/evmbench/ploit/crates/core/src/setup.rs x: 1 # contributors y: 721 line churn project/evmbench/ploit/crates/core/src/transactions.rs x: 1 # contributors y: 466 line churn project/evmbench/ploit/crates/exec/src/local.rs x: 1 # contributors y: 479 line churn project/evmbench/ploit/crates/utils/src/kill.rs x: 1 # contributors y: 77 line churn project/evmbench/scripts/backfill_agent_tokens.py x: 1 # contributors y: 184 line churn project/evmbench/scripts/count_disclosures.py x: 1 # contributors y: 273 line churn project/evmbench/scripts/smoke_test_agent_clis.py x: 1 # contributors y: 370 line churn project/evmbench/scripts/stats.py x: 1 # contributors y: 1273 line churn project/evmbench/veto/crates/core/src/jsonrpc.rs x: 1 # contributors y: 121 line churn project/evmbench/veto/crates/core/src/server.rs x: 1 # contributors y: 296 line churn project/paperbench/paperbench/nano/eval.py x: 3 # contributors y: 1769 line churn project/common/alcatraz/alcatraz/clusters/local.py x: 2 # contributors y: 2763 line churn project/swelancer/swelancer/eval.py x: 2 # contributors y: 685 line churn project/common/nanoeval_alcatraz/nanoeval_alcatraz/alcatraz_computer_interface.py x: 2 # contributors y: 297 line churn project/common/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py x: 2 # contributors y: 92 line churn project/paperbench/paperbench/nano/task.py x: 1 # contributors y: 1048 line churn project/paperbench/paperbench/solvers/base.py x: 1 # contributors y: 468 line churn project/paperbench/paperbench/solvers/basicagent/tools/basic.py x: 1 # contributors y: 309 line churn project/paperbench/paperbench/solvers/basicagent/utils.py x: 1 # contributors y: 382 line churn project/paperbench/paperbench/solvers/utils.py x: 1 # contributors y: 198 line churn project/paperbench/paperbench/utils.py x: 2 # contributors y: 325 line churn project/paperbench/paperbench/nano/utils.py x: 2 # contributors y: 508 line churn project/paperbench/paperbench/scripts/run_judge.py x: 1 # contributors y: 284 line churn project/paperbench/paperbench/scripts/run_judge_eval.py x: 1 # contributors y: 362 line churn project/paperbench/paperbench/scripts/run_monitor.py x: 2 # contributors y: 182 line churn project/paperbench/paperbench/infra/alcatraz.py x: 3 # contributors y: 530 line churn project/paperbench/paperbench/judge/simple.py x: 1 # contributors y: 1326 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/oai_completions_turn_completer.py x: 2 # contributors y: 158 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/converters.py x: 2 # contributors y: 461 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/utils.py x: 3 # contributors y: 158 line churn project/swelancer/swelancer/solvers/swelancer_agent/solver.py x: 1 # contributors y: 428 line churn project/common/nanoeval/nanoeval/_executor_worker.py x: 1 # contributors y: 585 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/solver.py x: 1 # contributors y: 400 line churn project/paperbench/paperbench/nano/entrypoint.py x: 2 # contributors y: 37 line churn project/swelancer/runtime_scripts/replay.py x: 1 # contributors y: 1086 line churn project/swelancer/swelancer/prompts.py x: 1 # contributors y: 189 line churn
2763.0
line churn
  min: 1.0
  average: 169.53
  25th percentile: 28.0
  median: 77.0
  75th percentile: 200.0
  max: 2763.0
0 3.0
# contributors
min: 1.0 | average: 1.08 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 3.0

File Size vs. Line Churn: 249 points

.pre-commit-config.yaml x: 43 lines of code y: 72 line churn project/evmbench/audits/2026-01-tempo-feeamm/config.yaml x: 12 lines of code y: 15 line churn project/evmbench/audits/2026-01-tempo-mpp-streams/config.yaml x: 14 lines of code y: 32 line churn project/evmbench/audits/2026-01-tempo-stablecoin-dex/config.yaml x: 23 lines of code y: 59 line churn project/evmbench/audits/2023-07-pooltogether/config.yaml x: 25 lines of code y: 27 line churn project/evmbench/audits/2023-10-nextgen/config.yaml x: 27 lines of code y: 30 line churn project/evmbench/audits/2024-01-canto/config.yaml x: 8 lines of code y: 10 line churn project/evmbench/audits/2024-01-curves/config.yaml x: 35 lines of code y: 39 line churn project/evmbench/audits/2024-01-init-capital-invitational/config.yaml x: 11 lines of code y: 14 line churn project/evmbench/audits/2024-01-renft/config.yaml x: 65 lines of code y: 72 line churn project/evmbench/audits/2024-02-althea-liquid-infrastructure/config.yaml x: 5 lines of code y: 6 line churn project/evmbench/audits/2024-03-abracadabra-money/config.yaml x: 15 lines of code y: 19 line churn project/evmbench/audits/2024-03-taiko/config.yaml x: 44 lines of code y: 49 line churn project/evmbench/audits/2024-04-noya/config.yaml x: 74 lines of code y: 95 line churn project/evmbench/audits/2024-06-size/config.yaml x: 46 lines of code y: 50 line churn project/evmbench/audits/2024-06-thorchain/avalanche/hardhat.config.js x: 130 lines of code y: 134 line churn project/evmbench/audits/2024-06-thorchain/ethereum/hardhat.config.js x: 39 lines of code y: 42 line churn project/evmbench/audits/2024-07-munchables/config.yaml x: 17 lines of code y: 22 line churn project/evmbench/audits/2024-07-traitforge/config.yaml x: 20 lines of code y: 23 line churn project/evmbench/audits/2024-08-phi/config.yaml x: 58 lines of code y: 65 line churn project/evmbench/audits/2024-08-wildcat/config.yaml x: 18 lines of code y: 19 line churn project/evmbench/audits/2025-04-virtuals/config.yaml x: 34 lines of code y: 39 line churn project/evmbench/audits/2025-06-panoptic/config.yaml x: 29 lines of code y: 32 line churn project/evmbench/docker_build.py x: 118 lines of code y: 155 line churn project/evmbench/evmbench/agents/agent.py x: 127 lines of code y: 151 line churn project/evmbench/evmbench/agents/claude/config.yaml x: 28 lines of code y: 33 line churn project/evmbench/evmbench/agents/codex/config.yaml x: 95 lines of code y: 114 line churn project/evmbench/evmbench/agents/opencode/config.yaml x: 51 lines of code y: 60 line churn project/evmbench/evmbench/agents/run.py x: 75 lines of code y: 95 line churn project/evmbench/evmbench/alcatraz.py x: 292 lines of code y: 403 line churn project/evmbench/evmbench/audit.py x: 173 lines of code y: 240 line churn project/evmbench/evmbench/bootstrap.py x: 136 lines of code y: 251 line churn project/evmbench/evmbench/experiments/__init__.py x: 1 lines of code y: 1 line churn project/evmbench/evmbench/experiments/generate_incorrect_findings.py x: 135 lines of code y: 262 line churn project/evmbench/evmbench/nano/eval.py x: 204 lines of code y: 252 line churn project/evmbench/evmbench/nano/gateway.py x: 330 lines of code y: 553 line churn project/evmbench/evmbench/nano/grade/base.py x: 73 lines of code y: 102 line churn project/evmbench/evmbench/nano/grade/detect.py x: 102 lines of code y: 148 line churn project/evmbench/evmbench/nano/grade/exploit.py x: 195 lines of code y: 236 line churn project/evmbench/evmbench/nano/grade/patch.py x: 477 lines of code y: 600 line churn project/evmbench/evmbench/nano/solver.py x: 443 lines of code y: 523 line churn project/evmbench/evmbench/nano/task.py x: 510 lines of code y: 610 line churn project/evmbench/evmbench/ploit/config.py x: 179 lines of code y: 222 line churn project/evmbench/evmbench/ploit/veto.py x: 245 lines of code y: 316 line churn project/evmbench/evmbench/utils.py x: 102 lines of code y: 170 line churn project/evmbench/ploit/.config/zepter.yaml x: 25 lines of code y: 42 line churn project/evmbench/ploit/crates/config/src/config.rs x: 220 lines of code y: 351 line churn project/evmbench/ploit/crates/config/src/constants.rs x: 12 lines of code y: 64 line churn project/evmbench/ploit/crates/config/src/lib.rs x: 22 lines of code y: 26 line churn project/evmbench/ploit/crates/config/src/manager.rs x: 278 lines of code y: 342 line churn project/evmbench/ploit/crates/core/src/addresses.rs x: 48 lines of code y: 115 line churn project/evmbench/ploit/crates/core/src/setup.rs x: 450 lines of code y: 721 line churn project/evmbench/ploit/crates/core/src/transactions.rs x: 311 lines of code y: 466 line churn project/evmbench/ploit/crates/exec/src/executor.rs x: 6 lines of code y: 15 line churn project/evmbench/ploit/crates/exec/src/local.rs x: 347 lines of code y: 479 line churn project/evmbench/ploit/crates/grader/src/memory.rs x: 192 lines of code y: 249 line churn project/evmbench/ploit/crates/grader/src/result.rs x: 238 lines of code y: 319 line churn project/evmbench/ploit/crates/grader/src/script.rs x: 182 lines of code y: 255 line churn project/evmbench/ploit/crates/utils/src/kill.rs x: 35 lines of code y: 77 line churn project/evmbench/ploit/crates/utils/src/process.rs x: 104 lines of code y: 172 line churn project/evmbench/ploit/crates/utils/src/sigsegv_handler.rs x: 106 lines of code y: 151 line churn project/evmbench/ploit/crates/utils/src/styles.rs x: 34 lines of code y: 44 line churn project/evmbench/scripts/backfill_agent_tokens.py x: 159 lines of code y: 184 line churn project/evmbench/scripts/count_disclosures.py x: 238 lines of code y: 273 line churn project/evmbench/scripts/smoke_test_agent_clis.py x: 299 lines of code y: 370 line churn project/evmbench/scripts/stats.py x: 1115 lines of code y: 1273 line churn project/evmbench/veto/crates/blocked/src/methods.rs x: 13 lines of code y: 37 line churn project/evmbench/veto/crates/config/src/config.rs x: 31 lines of code y: 41 line churn project/evmbench/veto/crates/config/src/constants.rs x: 66 lines of code y: 93 line churn project/evmbench/veto/crates/config/src/overrides.rs x: 35 lines of code y: 47 line churn project/evmbench/veto/crates/core/src/jsonrpc.rs x: 99 lines of code y: 121 line churn project/evmbench/veto/crates/core/src/server.rs x: 256 lines of code y: 296 line churn project/paperbench/paperbench/nano/eval.py x: 284 lines of code y: 1769 line churn project/common/alcatraz/alcatraz/clusters/local.py x: 1590 lines of code y: 2763 line churn project/swelancer/swelancer/eval.py x: 548 lines of code y: 685 line churn project/common/nanoeval_alcatraz/nanoeval_alcatraz/alcatraz_computer_interface.py x: 155 lines of code y: 297 line churn project/common/nanoeval_alcatraz/nanoeval_alcatraz/task_to_alcatraz_config.py x: 67 lines of code y: 92 line churn project/paperbench/paperbench/solvers/basicagent/solver.py x: 280 lines of code y: 342 line churn project/paperbench/paperbench/nano/task.py x: 401 lines of code y: 1048 line churn project/paperbench/paperbench/reproduce.py x: 259 lines of code y: 343 line churn project/paperbench/paperbench/solvers/apply_patch.py x: 359 lines of code y: 489 line churn project/paperbench/paperbench/solvers/base.py x: 178 lines of code y: 468 line churn project/paperbench/paperbench/solvers/basicagent/api.py x: 79 lines of code y: 108 line churn project/paperbench/paperbench/solvers/basicagent/completer.py x: 68 lines of code y: 110 line churn project/paperbench/paperbench/solvers/basicagent/logging.py x: 47 lines of code y: 66 line churn project/paperbench/paperbench/solvers/basicagent/prompts/templates.py x: 20 lines of code y: 64 line churn project/paperbench/paperbench/solvers/basicagent/tools/base.py x: 21 lines of code y: 53 line churn project/paperbench/paperbench/solvers/basicagent/tools/basic.py x: 248 lines of code y: 309 line churn project/paperbench/paperbench/solvers/basicagent/utils.py x: 268 lines of code y: 382 line churn project/paperbench/paperbench/solvers/direct_submission/solver.py x: 95 lines of code y: 466 line churn project/paperbench/paperbench/solvers/dummy/main.py x: 12 lines of code y: 77 line churn project/paperbench/paperbench/solvers/dummy/solver.py x: 64 lines of code y: 546 line churn project/paperbench/paperbench/solvers/upload.py x: 124 lines of code y: 142 line churn project/paperbench/paperbench/solvers/utils.py x: 149 lines of code y: 198 line churn project/paperbench/paperbench/utils.py x: 171 lines of code y: 325 line churn project/paperbench/paperbench/computer_utils.py x: 113 lines of code y: 158 line churn project/paperbench/paperbench/grade.py x: 184 lines of code y: 304 line churn project/paperbench/paperbench/nano/structs.py x: 180 lines of code y: 355 line churn project/paperbench/paperbench/nano/utils.py x: 68 lines of code y: 508 line churn project/paperbench/experiments/judge_eval/judge_eval_perf_cost.py x: 177 lines of code y: 19 line churn project/paperbench/experiments/judge_eval/judge_eval_perf_tables.py x: 124 lines of code y: 19 line churn project/paperbench/paperbench/monitor/monitor.py x: 132 lines of code y: 72 line churn project/paperbench/paperbench/scripts/run_judge.py x: 97 lines of code y: 284 line churn project/paperbench/paperbench/scripts/run_judge_eval.py x: 247 lines of code y: 362 line churn project/paperbench/paperbench/scripts/run_monitor.py x: 187 lines of code y: 182 line churn project/paperbench/paperbench/infra/alcatraz.py x: 250 lines of code y: 530 line churn project/paperbench/paperbench/judge/judge_eval/registry.py x: 73 lines of code y: 28 line churn project/paperbench/paperbench/paper_registry.py x: 76 lines of code y: 19 line churn project/paperbench/paperbench/rubric/tasks.py x: 285 lines of code y: 176 line churn project/paperbench/paperbench/judge/simple.py x: 563 lines of code y: 1326 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/completer.py x: 131 lines of code y: 182 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/oai_responses_turn_completer/converters.py x: 383 lines of code y: 461 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/type_helpers.py x: 199 lines of code y: 252 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/utils.py x: 125 lines of code y: 158 line churn project/paperbench/paperbench/constants.py x: 9 lines of code y: 10 line churn project/swelancer/swelancer/solvers/dummy/solver.py x: 124 lines of code y: 201 line churn project/swelancer/swelancer/solvers/swelancer_agent/solver.py x: 291 lines of code y: 428 line churn project/paperbench/paperbench/judge/judge_eval/download_data.py x: 70 lines of code y: 5 line churn project/common/alcatraz/alcatraz/clusters/_container_proc.py x: 38 lines of code y: 65 line churn project/common/alcatraz/alcatraz/clusters/_serialization.py x: 23 lines of code y: 52 line churn project/common/alcatraz/alcatraz/clusters/interface.py x: 16 lines of code y: 40 line churn project/common/compatibility_api/compatibility_api/task.py x: 33 lines of code y: 56 line churn project/common/nanoeval/nanoeval/_db.py x: 72 lines of code y: 162 line churn project/common/nanoeval/nanoeval/_executor_worker.py x: 376 lines of code y: 585 line churn project/common/nanoeval/nanoeval/_multiprocessing_utils.py x: 54 lines of code y: 124 line churn project/common/nanoeval/nanoeval/_persistent_db.py x: 107 lines of code y: 157 line churn project/common/nanoeval/nanoeval/asyncio_utils.py x: 121 lines of code y: 218 line churn project/common/nanoeval/nanoeval/eval.py x: 183 lines of code y: 310 line churn project/common/nanoeval/nanoeval/evaluation.py x: 285 lines of code y: 412 line churn project/common/nanoeval/nanoeval/library_config.py x: 173 lines of code y: 255 line churn project/common/nanoeval/nanoeval/metrics/agents.py x: 66 lines of code y: 98 line churn project/common/nanoeval/nanoeval/metrics/standard.py x: 103 lines of code y: 176 line churn project/common/nanoeval/nanoeval/monitor.py x: 179 lines of code y: 265 line churn project/common/nanoeval/nanoeval/recorder_protocol.py x: 62 lines of code y: 103 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/_versioning.py x: 28 lines of code y: 50 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/code_execution_interface.py x: 190 lines of code y: 383 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/limits.py x: 56 lines of code y: 88 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/pausable_timer.py x: 89 lines of code y: 118 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/simple_solver.py x: 82 lines of code y: 115 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/solver.py x: 287 lines of code y: 400 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/steps.py x: 50 lines of code y: 84 line churn project/common/nanoeval/nanoeval/solvers/computer_tasks/task.py x: 96 lines of code y: 157 line churn project/common/nanoeval/nanoeval/solvers/mcq.py x: 171 lines of code y: 233 line churn project/common/nanoeval/nanoeval/solvers/mcq_api.py x: 96 lines of code y: 130 line churn project/common/nanoeval/nanoeval/solvers/short_answer.py x: 76 lines of code y: 105 line churn project/common/preparedness_turn_completer/preparedness_turn_completer/turn_completer.py x: 72 lines of code y: 85 line churn project/swelancer/swelancer/run_swelancer.py x: 30 lines of code y: 62 line churn project/paperbench/paperbench/judge/base.py x: 143 lines of code y: 226 line churn project/paperbench/paperbench/judge/utils.py x: 117 lines of code y: 140 line churn project/paperbench/paperbench/nano/entrypoint.py x: 19 lines of code y: 37 line churn project/paperbench/paperbench/nano/logging.py x: 41 lines of code y: 80 line churn project/swelancer/swelancer/utils/custom_logging.py x: 48 lines of code y: 92 line churn project/swelancer/issues/16912_4/user_tool.py x: 59 lines of code y: 79 line churn project/swelancer/issues/28030_832/user_tool.py x: 55 lines of code y: 86 line churn project/swelancer/issues/29854_602/user_tool.py x: 92 lines of code y: 136 line churn project/swelancer/issues/32236_652/user_tool.py x: 116 lines of code y: 162 line churn project/swelancer/issues/44804_694/44804_694_VideoPlayerPreview.tsx x: 84 lines of code y: 108 line churn project/swelancer/issues/conftest.py x: 44 lines of code y: 72 line churn project/swelancer/runtime_scripts/replay.py x: 726 lines of code y: 1086 line churn project/swelancer/runtime_utils/check_flows.py x: 93 lines of code y: 169 line churn project/swelancer/runtime_utils/email_handler.py x: 97 lines of code y: 182 line churn project/swelancer/runtime_utils/intense_debug_logging.py x: 129 lines of code y: 253 line churn project/swelancer/runtime_utils/login.py x: 90 lines of code y: 199 line churn project/swelancer/runtime_utils/online_guard.py x: 83 lines of code y: 132 line churn project/swelancer/scripts/build_images.py x: 165 lines of code y: 250 line churn project/swelancer/swelancer/prompts.py x: 57 lines of code y: 189 line churn project/swelancer/swelancer/utils/general.py x: 35 lines of code y: 68 line churn project/paperbench/experiments/judge_max_depth/plot.py x: 56 lines of code y: 2 line churn project/paperbench/experiments/pbcd_correlation/plot.py x: 64 lines of code y: 2 line churn project/paperbench/paperbench/gui/app.py x: 420 lines of code y: 33 line churn project/paperbench/paperbench/judge/create_judge.py x: 43 lines of code y: 61 line churn project/paperbench/paperbench/judge/graded_task_node.py x: 109 lines of code y: 173 line churn project/paperbench/paperbench/judge/judge_eval/evaluate.py x: 102 lines of code y: 159 line churn project/paperbench/paperbench/metrics.py x: 231 lines of code y: 157 line churn project/paperbench/paperbench/rubric/utils.py x: 25 lines of code y: 17 line churn
2763.0
line churn
  min: 1.0
  average: 169.53
  25th percentile: 28.0
  median: 77.0
  75th percentile: 200.0
  max: 2763.0
0 1590.0
lines of code
min: 1.0 | average: 111.42 | 25th percentile: 21.0 | median: 58.0 | 75th percentile: 135.5 | max: 1590.0