Product2 lookup for the Product Catalog Agent
A local-first CrmProduct search with a queued Salesforce/Ampersand fallback, wired into the config-agent chat through a new SSE event and client-side job polling.
A reusable product2Lookup service, a BullMQ job that runs Salesforce queries off the request path, and two new agent tools (findProduct2CandidatesForAgent, upsertConfirmedProduct2ForAgent).
The Product Catalog Agent now resolves crmProductId via local CrmProduct first, then a queued Salesforce lookup. crmProductId is now nullable on agent writes. The catalog context replaces a 5k-row scan with JSON-field DB filters.
A new product2_lookup_job SSE event ends the turn with a jobId. The client polls jobs.getStatus and resumes the same thread with candidates or a failure message — the chat stream stays closed.
1. Why this exists
The Product Catalog Agent stages catalog products with a crmProductId that should map to the Salesforce Product2.Id for writeback. Before this PR, there was no way for the agent to actually look up a Product2: it would either skip CRM mapping entirely or scan up to 5,000 CrmProduct rows unfiltered, with no path to the live Salesforce object.
Salesforce/Ampersand queries can take seconds. Doing them inline in /config-agent/chat/stream would either time out the SSE stream or stall the agent's turn.
The local CrmProduct search loaded everything for the org and filtered in memory. Fine at 50 rows; not fine at thousands.
There was no way for a sub-agent to say "I started a job, come back later" and have the client resume the same conversation with results.
2. End-to-end flow
The agent always tries local first. Local hits resolve immediately. Local misses (with Salesforce configured) queue a job, end the SSE turn, and let the client take over polling. Step through the lifecycle:
findProduct2CandidatesForAgent. It checks CrmProduct by exact crmId, then by crmProductData.productCode, then by name — all using indexed Prisma JSON-path filters, not an in-memory scan.
3. The new server-side pieces
services/crm/product2Lookup.ts
Pure functions used by both the agent context and the BullMQ job:
product2LookupCriteria()— trims and normalizes inputsgetSalesforceProduct2LookupAvailabilityFromSpec()— checks CRM spec + env varsfetchSalesforceProduct2Candidates()— callscrmObjectService.queryProductsper criterion and merges match reasons
It does not write to CrmProduct — that's reserved for the post-confirmation upsert.
jobs/definitions/fetchSalesforceProduct2Candidates.ts
defineJob({
name: 'fetchSalesforceProduct2Candidates',
schema, // org + lookup fields
defaultJobOptions: {
attempts: 2,
backoff: {
type: 'exponential',
delay: 5000,
},
},
handler: async (job) =>
fetchSalesforceProduct2Candidates(job.data),
});
Transient Ampersand errors now throw, so BullMQ retries instead of resolving with a fake "failed" payload. Registered in jobs/registry.ts.
configAgent/catalogProductContext.ts
Adds three exported helpers:
findLocalProduct2CandidatesForAgent— exactcrmIdvia theorganizationId_crmIdunique, then JSON-pathfindManybyproductCodeandnamegetSalesforceProduct2LookupAvailabilityForAgentupsertConfirmedProduct2ForAgent— writes only the user-approved candidate
tools/catalogProductTableTools.ts
Two new LangChain tools registered with the agent:
findProduct2CandidatesForAgent— local first; on miss,dispatch('fetchSalesforceProduct2Candidates', …)and returns{ salesforceLookupQueued: true, jobId, jobName, lookup }upsertConfirmedProduct2ForAgent— explicitly post-approval only
Agent write schema now allows crmProductId: z.string().nullable().optional().
4. The streaming protocol change
The orchestrator inspects each askProductCatalogAgent result for a salesforceLookupQueued marker. If it finds one, it does not continue iterating tools — it emits a structured event and ends the turn.
A sub-agent that needed slow work had no way to hand off. The orchestrator would either block the stream or lose context after timing out.
// Inline Salesforce call here =
// stream hangs for ~5–30s,
// or the model's turn drifts.
// types.ts — new SSE variant
| {
type: 'product2_lookup_job';
jobName: 'fetchSalesforceProduct2Candidates';
jobId: string;
conversationId: string;
lookup: {
product2Id?: string | null;
skuId?: string | null;
productCode?: string | null;
name?: string | null;
};
}
The orchestrator emits a short user-facing token ("I started a Salesforce Product2 lookup…"), yields the structured event, then yields { type: 'done' }. The SSE stream closes.
Why subAgentSummarizer.ts was touched
The summarizer strips most fields from sub-agent JSON before passing it back up. Without explicitly whitelisting them, the orchestrator would never see the queue marker.
// subAgentSummarizer.ts
const KEPT_FIELDS = [
// …
'salesforceLookupQueued',
'jobName',
'jobId',
'lookup',
] as const;
5. The client polling-resume loop
useConfigAgent.ts grows a polling helper and a resume queue. The big idea: do not keep the chat stream open while waiting on a slow job. Instead, poll trpc.jobs.getStatus, then send a new message back into the same thread.
pollProduct2LookupJob(jobId, signal)
// every 2_000 ms
// up to 120_000 ms
// active statuses:
// waiting | active |
// delayed | waiting-children
Builds a markdown message containing the original lookup and the candidates payload, then calls enqueueProduct2Resume(threadId, message). The agent confirms the mapping with the user.
Reports the failure reason to the agent and explicitly invites it to continue staging with crmProductId=null. The user is never silently dropped.
Queue, version, and busy-thread guards
Three new refs back the resume machinery, all keyed by thread id:
| Ref | Purpose |
|---|---|
pendingProduct2ResumeRefs | FIFO of resume messages waiting to be sent on a thread that is currently streaming. |
product2ResumeVersionRefs | Monotonic counter per thread. Incremented on stop/reset. Stale poll completions check their captured version and silently drop. |
streamingThreadsRef | Mirror of streamingThreads state so synchronous code (resume flush, migrate, finally block) can read the latest value without waiting for a render. |
The finally block of sendMessage now only clears streaming state if the abort controller it owns is still the current one for the thread — preventing a slow finalize from clobbering a fresh send. After clearing, it calls flushProduct2ResumeQueue(threadId) to drain any queued resume messages.
Thread-id migration edge case
An UNASSIGNED thread becomes a real conversation id mid-stream. migrateThreadKey now also moves any queued resume messages and the resume version counter across, so a job that completes after the rename still posts to the right conversation.
6. The local-search performance fix
// effectively:
const all = await prisma.crmProduct
.findMany({
where: { organizationId: orgId },
take: 5000,
});
// then in-memory string match
// on productCode / name
Worked on small orgs. Pulled megabytes on large ones.
prisma.crmProduct.findMany({
where: {
organizationId: orgId,
crmProductData: {
path: ['productCode'],
equals: 'IDV',
},
},
orderBy: { updatedAt: 'desc' },
take: limit + 1,
});
Push-down filter on the JSON column, bounded result set, sorted for recency.
7. Agent prompt and tool-rule changes
- Step 3: call
findProduct2CandidatesForAgentbefore staging when writeback mapping is needed. - Step 4: never call
upsertConfirmedProduct2ForAgentbefore user approval. - Step 5: if no match or the user approves an unmapped SKU, stage with
crmProductId=null. - When the tool returns
salesforceLookupQueued=true, hoistjobId/jobName/lookupas top-level JSON fields and stop. Do not poll from the request path.
The orchestrator system prompt gains a matching SALESFORCE PRODUCT2 MAPPING section: include the chosen mapping in the user-facing action_plan, require one approval, and let crmProductId=null through with a warning rather than blocking.
8. What it doesn't change
- The
/config-agent/chat/streamendpoint contract — only a new SSE event type is added. - The
CrmProductschema. No migration in this PR. - Other staging tools (
stageCreateCatalogProduct,bulkStageCatalogProducts, etc.) keep their existing signatures — only thecrmProductIdfield onproductWriteSchemabecomes nullable. - Dealops 1 product/pricingcurve paths are untouched.
- The
crmObjectServicequery layer — this PR only consumesqueryProducts. - Pricebook resolution and attribute-schema discovery flows.
9. Risks & open questions
salesforceLookupQueued and then JSON-parsing either the whole reply or a regex-extracted object. If the model wraps the JSON oddly, the queue marker can be missed and the client will never get the SSE event. The whitelisting in subAgentSummarizer mitigates this but doesn't fully eliminate it.
crmProductId nullable on agent input). Reverting the PR removes the tools and the event; existing catalog flows continue to work because the agent's instructions only call the new tools when Salesforce writeback mapping is needed.