Give the validate/correct agent real tools — and stop guessing Transfer pricing

Stage 4's LLM now has catalog + pricing tools to repair concrete SKU errors, missing Transfer ATS defaults to $250 for bps mode, and six brittle deterministic rules move to the LLM.

Author @yijunz166 PR dealops#6187 Branch yijun/ai-quote-correction-guardrails Files 13 Diff +556 / −456 Area Dealops 2 · AI quoting

What it adds

The Stage 4 validate/correct LLM now runs as a tool-calling chat with get_product_catalog, lookup_products_by_name, and lookup_product_pricing — the same tools the generator uses — so it can repair missing SKUs and floors instead of only suggesting them in prose.

What it changes

Missing Transfer average transaction size no longer blocks the quote. It defaults to $250, which selects the bps variant. Ambiguous ATS evidence (e.g. "under $300") still blocks. Zero list prices no longer act as a hard ceiling for bps percent SKUs.

What it removes

Six deterministic guardrails driven by fuzzy text scanning of opportunity context — ABI/Core product inference, ATS mode matching, minimum-commitment and support-waiver detection — are deleted. The LLM owns those decisions now.

Stage 4 LLM (validate/correct)

Deterministic rules

Transfer / bps pricing

Catalog / pricing tools

Prompts

1. Why this exists

The AI quoting pipeline has a Stage 4 validate/correct step: an LLM reads the generated quote and the deterministic rule findings, then emits a corrected recommendation. Two failure patterns kept hitting production:

The corrector couldn't actually correct. It had no tools. If a hard rule fired for a missing SKU (e.g. ABI needs Balance) but Balance's productSpecId, list price, and L1 floor weren't already in the fetched rule context, the LLM would either invent numbers or leave the quote broken.
Deterministic rules over-fired on fuzzy text. Regexes scanning opportunity notes decided that mentions of "ABI" or "Plaid core products" meant those SKUs were required, and that missing Transfer ATS was always a blocker. These rules produced false positives that the corrector then couldn't fix. Transfer sales calls frequently omit ATS, so quotes stalled.

2. Before / after: Stage 4 responsibilities

Before Deterministic-heavy

Regex scans of opportunity text drove multiple hard rules:

CORE_PRODUCT_MISSING
ABI_PRODUCT_MISSING
TRANSFER_AVG_TRANSACTION_SIZE_MISSING
TRANSFER_PRICING_MODE_MISMATCH
MISSING_MINIMUM_COMMITMENT
MINIMUM_WAIVER_NOT_RAMPED
SUPPORT_WAIVER_NOT_RAMPED

Stage 4 LLM had no tools — just a single-shot generateContent.

After LLM-owned with tools

Only structural hard rules remain deterministic:

PRICE_BELOW_L1 / PRICE_ABOVE_LIST
MISSING_AUTO_SELECTED_ADDON
AUTO_DESELECT_CONFLICT
TRANSFER_MODE_CONFLICT
TRANSFER_COMPANION_MISSING
PRICE_TIER_SHAPE_INVALID

Stage 4 LLM runs as a chat with catalog + pricing tools, capped at 20 iterations.

3. Removed rule codes

The following AIRuleValidationCode values are deleted from validateAiRecommendationAgainstRules.ts and from the fallback message table in generateValidatedAiRecommendation.ts:

Removed code	Was fired by	Now handled by
`CORE_PRODUCT_MISSING`	Text scan for "plaid core products"	Stage 4 LLM (with catalog tools)
`ABI_PRODUCT_MISSING`	`contextMentionsAbiProducts` regex	Stage 4 LLM (prompt names Auth/Balance/Identity)
`TRANSFER_AVG_TRANSACTION_SIZE_MISSING`	ATS extractor returning null	$250 default for missing; LLM still blocks ambiguous
`TRANSFER_PRICING_MODE_MISMATCH`	Comparing extracted ATS to bps/Fixed variant	Stage 4 LLM prompt rules
`MISSING_MINIMUM_COMMITMENT`	`contextRequestsMinimumCommitment` regex	Stage 4 LLM
`MINIMUM_WAIVER_NOT_RAMPED`	Waiver regex + ramp inspection	Stage 4 LLM
`SUPPORT_WAIVER_NOT_RAMPED`	Waiver regex + support product ramp check	Stage 4 LLM

4. How the tool-calling loop works

In validateAiRecommendationWithLLM.ts, the single generateContent call becomes a chat with a tool-execution loop. The generator's helpers are exported and reused:

New exports from generateAiQuote.ts

buildTools(pricebook)
executeQuoteToolCall({ name, args, catalog, listPriceMap, l1PriceMap, pricebook })
stableToolCallKey(name, args)
tryParseJson(value)
normalizeMonthlyAmount(value)

The loop shape:

const chat = model.startChat();
let response = await callGeminiWithRetry('ai_quote_validate_correct_initial',
  () => chat.sendMessage(prompt), ...);

while (iterations < QUOTE_VALIDATE_CORRECT_MAX_TOOL_ITERATIONS /* 20 */) {
  const functionCalls = parts.filter(part => part.functionCall);
  if (functionCalls.length === 0) break;

  const functionResponses = functionCalls.map(part => {
    const toolCallKey = stableToolCallKey(fc.name, fc.args);
    const count = (toolCallCounts.get(toolCallKey) ?? 0) + 1;

    if (count > QUOTE_VALIDATE_CORRECT_REPEATED_TOOL_CALL_LIMIT /* 3 */) {
      // return cached result with an instruction to stop calling and answer
      return { functionResponse: { name: fc.name, response: { result: cached } } };
    }
    const result = executeQuoteToolCall({ name: fc.name, args: fc.args,
      catalog, listPriceMap, l1PriceMap, pricebook });
    toolResultCache.set(toolCallKey, result);
    return { functionResponse: { name: fc.name, response: { result } } };
  });

  response = await callGeminiWithRetry('ai_quote_validate_correct_tool_response',
    () => chat.sendMessage(functionResponses), ...);
  iterations++;
}

Safety rails. Hard cap of 20 iterations; if the agent is still requesting tools at the cap it throws. Identical tool calls (same name + stably-serialized args) get the cached result back with an explicit instruction to stop calling and answer — this prevents the "stuck in a lookup loop" failure mode.

5. The $250 default and bps zero-list-price rule

Missing ATS → $250

New in aiQuoteRuleHelpers.ts:

export const DEFAULT_AVERAGE_TRANSACTION_SIZE = 250;

export function resolveAverageTransactionSizeForPricing(text) {
  return extractAverageTransactionSizeEvidenceFromText(text)
    ?? { amount: 250, comparator: 'exact' };
}

generateV2Quote.ts switches from extractAverageTransactionSizeFromText to this resolver, so the V2 mapper always writes an avgTransactionSize term — $250 when nothing was found.

Zero list price ≠ ceiling for bps

capAtListPrice gains an ignoreZeroListPriceCeiling flag. It's set true when the catalog entry is a bps-percent product (via isBpsPercentCatalogProduct).

if (ignoreZeroListPriceCeiling && listPrice === 0) {
  return proposedPrice;
}

Same flag threads through buildQuotePrice, buildQuotePricePercent, and the deterministic validatePricePoint in the rules validator.

6. Prompt changes

quote_generation.v1.md

Replaces "do not guess; return only if a non-Transfer set exists" with the $250 default rule, and clarifies bps quotePrice semantics:

Missing ATS ⇒ use $250 default ⇒ select bps; must be called out in priceReasoning.
For bps variants, quotePrice is the percent/bps rate itself (0.1 means 0.1%) — do not multiply by ATS. The mapper attaches the ATS term separately.
For Fixed variants, quotePrice is the per-event unit price.

quote_validate_correct.v1.md

Teaches the corrector how to use its new tools and adds explicit ABI/Transfer correction procedures:

Prefer one batched product-name lookup and one batched pricing lookup; reuse returned values.
For ABI, add the exact Auth/Balance/Identity SKUs — call lookup_products_by_name with ["Auth","Balance","Identity"] if IDs aren't in context. Do not satisfy ABI with the Auth & Identity bundle.
Transfer bps rate handling matches the generation prompt.
$250 fallback must be labeled as a standard pricing default in priceReasoning, not presented as customer evidence.

7. Test surface change

Test	Before	After
ABI products missing	Deterministic rules assert Balance + Identity are flagged from context text.	Renamed: "does not infer missing ABI products from opportunity text in final guardrails" — passes with just Auth.
finalizeAiQuote Stage 6	Used the ABI text-inference path to detect a dropped Balance.	Uses `autoSelectProductIds: ['spec-balance']` on Auth, expects `MISSING_AUTO_SELECTED_ADDON` instead.
Transfer ATS missing / mismatch	Multiple tests asserting `TRANSFER_AVG_TRANSACTION_SIZE_MISSING` and `TRANSFER_PRICING_MODE_MISMATCH`.	Replaced with tests asserting the rules validator now passes when a single Transfer mode is selected, and does not infer mode from ATS text.
V2 quote mapper (new)	—	Two new tests: zero list price does not cap bps percent rate; missing ATS produces `avgTransactionSize = 250 USD` term.

8. What it doesn't change

Stage 4 still runs after generation; it still returns validation + correctedRecommendation in one call.
The generator's tool loop and prompt structure are unchanged aside from the Transfer-specific paragraphs.
Deterministic guardrails for structural correctness — L1 floors, list-price ceilings (for non-bps or nonzero list), auto-select add-ons, auto-deselect conflicts, tier shape, Transfer companion events, Transfer mode conflict within a rail/risk pair — all remain.
No schema, DB, or tRPC surface changes. This is entirely inside apps/server/src/dealops2/aiQuoting/pipeline/.
The V2 avgTransactionSize term shape is unchanged; the only difference is that it's now always populated for Transfer bps products.

9. Risks and open questions

LLM correctness on removed rules. Seven hard rules that used to fail deterministically now depend on the Stage 4 LLM correctly following the prompt (ABI product set, minimum-commitment handling, waiver ramps, Transfer mode selection). Regression risk is highest on opportunities where deterministic rules previously caught the LLM's mistakes.

Tool loop bounds are conservative. 20 iterations × 3 identical-call cap, with cached-result short-circuiting, matches the generator's pattern. Both agents share the same executeQuoteToolCall, so pricing math stays consistent across stages.

The $250 default is a policy choice. It changes previously-blocked Transfer quotes into completed bps quotes. The generation and validate/correct prompts both require explicit reasoning strings to flag when the default was used — reviewers should confirm downstream consumers (deal-desk review UI, quote export) surface those reasoning strings prominently.