An autoresearch loop that refactored `getRevenue`

A keep-or-discard harness drove an LLM agent through a recursive-to-iterative refactor of a 1,400-line revenue method — gated on a 2,400-cell byte-identical golden and a strict complexity metric.

PR dealops#6093 Author @pk675 Base main Head claude/refactor-getrevenue-iterative-k60uf7 Files 12 +/- +5534 / -527 Status open

What it adds

A standalone autoresearch harness under apps/server/autoresearch/getRevenue/ — steering brief, scalar fitness eval, LLM judge, keep/discard loop, and a 2,400-cell byte-identical golden snapshot.

What it produced

A landed refactor of PricingEngineService.getRevenue: recursion eliminated, strict cyclomatic complexity dropped from the high-30s/50s to 8, duplicated leaf math collapsed, side-effects lifted to the boundary.

What it preserves

Public signature unchanged — ~149 call sites untouched. Every kept round had to keep the golden byte-identical across all 2,400 cells and pass tsc, so behavior is preserved by construction.

Harness (fixed)

Target (edited by loop)

Pure leaf module

Orchestrator

Recursion (forbidden)

1. The shape of the problem

PricingEngineService.getRevenue sat at ~1,700 lines in pricingEngineService.ts, hand-tuned over years. It was the central revenue method, called from ~149 sites, and both the all and category selectors summed their products by calling this.getRevenue({ type: 'product', … }) inside a reduce — so it called itself once per product.

Recursive shape

all → loops products → getRevenue(product)
category → loops products → getRevenue(product)
product = base case (the actual math)

Why a hand refactor is risky

Three divisor sources, five ramp-flatten strategies, the ACV adjustment (only on the yearly all path), the ramped annually-billed /12 special case, the year-1-only zeroing, monthly-minimum with vs without a month index. One displaced number breaks billing.

Why a loop instead

Each round makes ONE small edit, gates on typecheck + a 2,400-cell byte-identical golden, then keeps the edit only if a scalar cost strictly drops. Progress accumulates. The agent physically cannot edit its own gate.

2. The harness, in pieces

Six files under apps/server/autoresearch/getRevenue/ form the fixed harness — the analogue of Karpathy's prepare.py. The loop is never allowed to edit them.

File	Role	Size
`program.md`	steering brief — objective, hard constraints, parity checklist	+137
`eval.ts`	scalar fitness `cost` — gates + AST metrics + judge	+545
`judge.ts`	LLM code-quality judge (0–4, per-dimension, actionable feedback)	+377
`loop.sh`	keep/discard driver, threads feedback into next candidate	+261
`__test__/getRevenueGolden.spec.ts`	2,400-cell byte-identical snapshot of original behavior	+751
`__test__/getRevenueGolden.json`	frozen recorded values	+2402

The cost function, phased

The loop minimizes a single scalar cost (lower = better) and reverts any round that doesn't strictly lower it. Three phases, each with a different cost shape, so progress can be banked incrementally rather than waiting for one heroic edit:

Phase	cost (lower = better)	Targets met when
1 — shrink + lift	`max CC`	max CC ≤ `CC_TARGET` (recursion allowed)
2 — de-recurse	`recursion × 1000 + CC`	0 recursion AND CC ≤ target
3 — polish	favours judge score	judge ≥ `TARGET_SCORE`, with min scores on the functional dimensions

Why CC, not recursion-edge count, in phase 1? Recursion is atomic — you can't bank "1 of N edges removed". Cyclomatic complexity is continuous: each extracted helper drops the score, so every kept round encodes a real win. Phase 2 then folds recursion × 1000 into the cost so partial de-recursion (3 → 2 → 1 → 0 functions on the cycle) is also banked.

3. How a round actually runs

Step through one full iteration of the loop. The orange node is what's happening now; green means done, purple means upcoming.

Step 1 of 6

Step 1 — PromptThe loop reads program.md (the steering brief — objective, hard constraints, parity checklist) and the previous round's eval-result.json feedback. The brief asks for ONE small change — extract a helper, collapse a duplication, or lift a side-effect — not the whole refactor.

4. The cheating-prevention layer

The harness gives a code-gen agent root access to the repo. Three concrete defenses keep the loop honest:

File-set enforcement

After every round, enforce_file_set diffs git status against the user's pre-existing dirty files and the two allowed paths (pricingEngineService.ts, pricingEngineRevenue.ts). Anything else — including the golden JSON or any harness file — is checked out from HEAD. The agent can't weaken its own gate.

Strict CC, closures included

With STRICT_CC=1 (default) the metric folds nested closures (.map/.reduce callbacks, IIFEs) into the enclosing function. getRevenue is ~16 per-function but ~32 strict. Complexity can't hide in a callback.

Call-graph recursion

Recursion is detected on the in-scope call graph, so getRevenue → helper → getRevenue still counts. Routing recursion through an extracted helper doesn't lower the score; a renamed monolith doesn't either.

Merge-base baseline

"max CC" is scoped to getRevenue + every function the refactor adds, measured against git merge-base HEAD origin/main — not HEAD. So a newly extracted helper keeps counting even after it's committed.

5. The refactor it produced

Running the loop produced the actual getRevenue rewrite, landed as commits on top of the harness. Three phases, each with measurable wins:

~32–52

Original strict max CC

After phase 3, strict CC

Self / mutual recursion

2,400

Golden cells, byte-identical

3 / 4

Final LLM judge score

149

Call sites untouched

Before / after — the call shape

Before — recursive

getRevenue({type:'all'})
  → products.reduce((sum, p) =>
      sum + getRevenue({type:'product', p}))
                ↑ recurses

getRevenue({type:'category'})
  → products.reduce((sum, p) =>
      sum + getRevenue({type:'product', p}))
                ↑ recurses

getRevenue({type:'product'})  // base case
  → ~1400 lines of inlined leaf math
    + impure reads buried inside

After — folds

getRevenue(selector, tp, …)
  → dispatch on selector.type

aggregateProductRevenue(products, applyAcv)
  → products.reduce((sum, p) =>
      sum + productSelectorRevenue(p, …))
                ↑ calls leaf directly

productSelectorRevenue(p, …)  // pure-ish leaf
  → resolveProductCase → computeAbsoluteRevenue
                       │ productMonthIdxRevenue
                       │ computeRecurringMonthlyUnitlessRevenue

Before / after — files

pricingEngineService.ts

+914 / −526. Now a thin dispatcher over small single-purpose methods. Holds the side-effecting reads (getTerm, evaluationEngine) at the boundary, threads them into pure leaves.

pricingEngineRevenue.ts (new)

+257. Free functions, readonly params, no this, no logging. Holds the percent-volume math, the bps-per-txn clamp context, and the time-period predicates.

The new pieces

Orchestrator (on the class)

aggregateProductRevenue — folds product lists, applies ACV only on yearly all
productSelectorRevenue — extracted single-product base case
resolveProductCase — guards + boundary avgTransactionSize read
resolveProductRevenueContext — per-product invariants resolved once
computeAbsoluteRevenue — total_contract/year_idx as explicit folds
computeRecurringMonthlyUnitlessRevenue — shared flat+percent assembly

Pure leaves (new module)

isYearlyRevenueTimePeriod, isZeroForYear1Only — predicates, lifted verbatim
applyAvgTxnSizeMultiply — the multiply leaf (with the bpsPercentPricing → 0 precedence preserved)
buildBpsPerTxnContext + bpsPerTxnArg — Plaid bps clamp, resolved once
computeMonthIdxPercentVolume, computeRecurringMonthlyPercentVolume — kept separate on purpose (different ramp-flatten strategies)

6. The golden net

What lets the loop edit aggressively without losing correctness. Generated once against the original recursive code and frozen — each fixture's recorded value was hand-verified to confirm the branch actually executes.

Fixture × selector × period × options

Deliberately exercises the branches a refactor is most likely to break: percent / percentWithLimits clamp, bpsPercentPricing per-txn clamp, avgTransactionSize multiply, ramped volume (all five flatten strategies), ramped price + annually-billed /12 special case, tiered percent, the three divisor sources, monthly minimum, ACV adjustment, multi-year fractional proration.

2,400 cells, zero error cells

Deterministic — passes against itself without REGEN_GOLDEN. One mismatched cent fails the gate. Every kept round had to be byte-identical across all 2,400 cells and pass tsc. Behavior is preserved by construction, not by inspection.

Show sample golden cells

"flat_monthly_12mo | all | abs:total_contract | plain": 7200,
"acvAdjustment_12mo | all | abs:total_contract | plain": 11500,
"acvAdjustment_12mo | all | abs:year_idx:1 | plain": -500,
"monthlyMinimum_flat_12mo | all | abs:month_idx:0 | plain": 333.33333333333326,
"monthlyMinimum_plaidRamp_12mo | all | abs:month_idx:0 | plain": 300,
"freqAnnually_24mo | all | abs:month_idx:0 | plain": 3600,
"freqFixedMonths_ramped_12mo | all | abs:month_idx:0 | plain": 300,
"customDates_18mo | all | abs:total_contract | prorate": 6000,
"oneTime_fee_12mo | all | abs:month_idx:0 | plain": 416.6666666666667,

7. What it doesn't change

Public signature. getRevenue(selector, timePeriod, prorateOptions?, skipMonthlyMinimumDiscount?) — ~149 call sites untouched.
Summation order. Products in filtered order, months in helper order, years 0..N. Float + is non-associative; reordering would break byte-identity.
The three divisor sources. Annually uses getTimePeriodDivisor (no flag); monthly uses {checkPercentVolume:true}; month_idx uses getRevenueTimePeriodDivisor. They are NOT unified.
The ACV adjustment. Applied only on the all yearly path; never on category or product.
The ramped annually-billed /12 special case. Still bypasses divisor and mask.
The pre-existing recursive helpers (tier-walk getListPrice/getCost). Out of scope; the loop's recursion check explicitly allows them.
Other tests. The full Jest suite + lint run on CI; the loop itself gates each round on typecheck + the golden only (for speed).

8. Open questions and follow-ups

Quality headroom flagged by the judge. The phase-3 judge scored immutability, performance (per-product-month memoization), and less_duplication at 2/4 — real polish opportunities, not required for correctness. They can be picked up by another phase-3 run with more iterations.

The mocha proration gate is opt-in. RUN_MOCHA=1 enables it; default is the golden-only Jest gate for loop speed. Both run on CI for the final merge.

Pnpm bypass. The eval calls tsc/jest/mocha/tsx binaries directly rather than via pnpm run. Pnpm's pre-run dependency check can abort on an unrelated CDN-pinned dependency (xlsx) when the network is restricted; the direct binary calls sidestep that gate.

9. How to run it

# From apps/server. Golden is already committed — no regen needed.
PHASE=1 CC_TARGET=10 GEN_TIMEOUT=240 ITERATIONS=24 \
  JUDGE_PROVIDER=anthropic ANTHROPIC_API_KEY=... JUDGE_MODEL=<model> \
  bash autoresearch/getRevenue/loop.sh

# …then PHASE=2, then PHASE=3.
# Or just score the current end state:
JUDGE_PROVIDER=anthropic ANTHROPIC_API_KEY=... JUDGE_MODEL=<model> \
  pnpm run eval:getRevenue

An autoresearch loop that refactored getRevenue

1. The shape of the problem

2. The harness, in pieces

The cost function, phased

3. How a round actually runs

4. The cheating-prevention layer

5. The refactor it produced

Before / after — the call shape

Before / after — files

The new pieces

6. The golden net

7. What it doesn't change

8. Open questions and follow-ups

9. How to run it

An autoresearch loop that refactored `getRevenue`