Langchain Test Org → catalog_product migration
A bespoke flat-price converter, an org-wide V3 versioning seed helper, and two shared schema fixes that keep renewal-seed config and custom end-date overrides from silently disappearing across the V2↔V3 round trip.
A per-org converter, migration runner, and V3 versioning-store seeder for Langchain Test Org — registered in the publish-path registry so the real publish seam round-trips this org 1-1.
Three top-level productSpec fields were being silently dropped on V2↔V3 round trip: customEndDateOverrideInWeeks and the three renewal-seed fields. Now structural, carried via systemAttributes.
Pulls the throwaway "seed attributes" / "seed flow" scripts into one shared _utils/versioningSeed.ts + a shared env/org safety guard. Xendit's initV3.ts is rewritten on top of it.
97/97 products (including the priceless internal commitment) and 96/96 datapoints round-trip identically through the registry seam. --write / publish / flow-type flip remain separate manual steps.
1. Why this exists
Dealops is migrating each customer org off the legacy V2 pricingSpec + dataSheet pair onto the unified catalog_product data model (Dealops 3). Every org needs its own converter because the way prices and attributes are encoded in V2 differs per org — and the publish seam must reproduce that org's V2 spec 1-1 after a round trip, or it can't be cut over safely.
The roundtrip-catalog-migration skill orchestrates this. This PR is the Langchain Test Org output of that skill, plus three corrections that fell out of running it: two silently-dropped V2 fields, and a refactor that lifts the previously-throwaway "seed the V3 versioning store" step into shared, reusable code.
productSpec, identified by a tag combination (deploymentType × supportLevel × segment × tier) and priced by exactly one LIST_PRICE currency datapoint. The default Dealops sku == datapoint planner would (a) rename productIds from the datapoint and (b) drop the internal priceless commitment product entirely. Both are unacceptable for a 1-1 round trip, hence a bespoke converter.
2. What changes — file map
3. The Langchain converter, in one picture
productId— opaque UUID-ishtags: deploymentType × supportLevel × segment × tiercategoryId(e.g.langsmithDeployment)- 0 or 1
LIST_PRICEcurrency datapoint - Optional
includeInRenewalSeed,customEndDateOverrideInWeeks
skuId= the V2productId, verbatimbaseProductId= theproductLine(= V2categoryId)baseProductName= that category's display namecustomAttributes: the 4 tag dimensionssystemAttributes: renewal-seed + end-date fieldsprice.flat_price= the one LIST_PRICE (or{}forcommitment)
Base product = the category
The baseProductId is the product's productLine attribute, which is literally V2's categoryId (e.g. langsmithDeployment, agentBuilder, langsmithOE, platformSubscription, professionalServices). baseProductName is that category's display name (from productCategories[].name). The category-less internal commitment product falls back to its own name.
This is catalog-UI grouping only — the reverse converter never reads it, so it cannot break the 1-1 round trip.
The internal commitment product
Langchain's V2 spec contains one priceless internal product used by the engine for minimum-commit calculations. The default sku==datapoint planner drops it (no datapoint → no SKU). The bespoke converter keeps it: price = {}, isActive: false, isNotSelectable: true, no pricebook, no category. It round-trips into V2 unchanged. adminOnly stays false deliberately — flipping it would change the V2 productSpec and break 1-1.
4. The two silent-drop bugs this PR fixes
Both bugs affected every org, not just Langchain. They surfaced because Langchain happens to use these fields in production data, and a 1-1 round trip exposed the loss.
Lived in OUT_OF_SCOPE_FIELD_KEYS alongside startDate/endDate. productSpecToRow dropped it. A reconstructed spec lost the value silently — a product configured with a 4-week override would come back with none.
OUT_OF_SCOPE_FIELD_KEYS = [
'startDate',
'endDate',
'customEndDateOverrideInWeeks', // dropped
];
Moved into STRUCTURAL_FIELD_KEYS. The V3 schema already has a top-level customEndDateOverrideInWeeks: z.number().optional(), so it now rides in systemAttributes and round-trips top-level.
STRUCTURAL_FIELD_KEYS = [
...,
'customEndDateOverrideInWeeks',
'includeInRenewalSeed',
'renewalSeedProductSpecIds',
'renewalSeedProductSpecId',
];
Renewal-seed fields — never had a V3 home
The renewal-substitution config (23 Langchain SKUs use it) consists of three top-level V2 productSpec fields the engine reads to determine what a product renews into. The V2↔V3 converter pair had no mapping for any of them. This PR:
- Adds them to
v3ProductSpecSchemainpricingSpec/types.ts. - Copies them V2→V3 in
pricingSpec/converter.ts(only when set, to avoid stampingundefined). - Copies them V3→V2 in
pricingSpec/v2Converter.tssymmetrically. - Lists them in
STRUCTURAL_FIELD_KEYSsoproductSpecToRowstashes them insystemAttributes.
// pricingSpec/converter.ts — convertProductSpec
...(v2.includeInRenewalSeed !== undefined && {
includeInRenewalSeed: v2.includeInRenewalSeed,
}),
...(v2.renewalSeedProductSpecIds?.length && {
renewalSeedProductSpecIds: v2.renewalSeedProductSpecIds,
}),
...(v2.renewalSeedProductSpecId && {
renewalSeedProductSpecId: v2.renewalSeedProductSpecId,
}),
The matching catalogProductMapping.test.ts assertion is updated so the "out-of-scope drop" test now verifies preservation of customEndDateOverrideInWeeks alongside the still-dropped startDate/endDate.
5. The registry seam — how publish picks this up
The shared _utils/registry.ts is the dispatch table the real publish path (trpc/router/v3Admin/_helpers.ts and publishDrafts.ts) consults. Adding an org is intentionally minimal — one entry, two function references:
'langchain-test-org': {
pricingSpecToV2: langchainCatalogToV2PricingSpec,
catalogToV2Datapoints: langchainCatalogToV2Datapoints,
},
The Langchain reverse functions are thin wrappers over the shared converters — the org has no variant families to collapse, and its single flat_price → LIST_PRICE mapping is exactly what convertToV2DataSheet(deriveV3Datasheet(...)) already produces. No bespoke per-org reverse logic, unlike Xendit.
6. The versioning-store seed, lifted into shared code
A migrated org isn't usable on DEALOPS_V3 from catalog_product rows alone. The V3 admin shell, publish, and the Config Agent all read from the versioning store (OrgSpecVersion rows + one OrgGlobalVersion), and all five SpecTypes must exist or those surfaces throw.
Previously this was done by hand with throwaway scripts (seedAttributesFromCatalog.ts, seedAttributesFromPayload.ts, …). This PR moves the work into _utils/versioningSeed.ts and rewrites Xendit's initV3.ts on top of it, so every future org seeds identically.
| SpecType | What gets seeded | Source |
|---|---|---|
pricingSpec |
The org envelope (currencies, pricebooks, flags) | convertPricingSpec(V2). Per-product specs come from catalog_product at publish time. |
datasheet |
Base datasheet body (priced datapoints rebuilt from catalog price at publish) |
convertDataSheet for Langchain; { datapoints: [] } for Xendit (shared converter throws on Xendit fee types). |
attributes |
Custom-attribute schema | Derived from the planned catalog rows — union of customAttributes keys across the migration's own SKUs. |
flowSpec |
Minimal valid body (empty products / calculator / finish options) | buildMinimalFlowSpec() — relaxed; authored properly later. |
workflowSpec |
Minimal valid body (no approval edges) | buildMinimalWorkflowSpec() — relaxed; authored properly later. |
Attribute derivation enforces the reader's rules
deriveCustomAttributeDefs() walks the planned rows and infers each key's valueType (boolean / number / enum ≤ 50 distinct strings / string / multi_enum for array values). It drops anything that would make loadAttributeSchema throw at read time:
- Reserved keys (
productId,pricebookId) — dropped. - Keys colliding with a system attribute — dropped (else "Key conflict" at read).
- Unrepresentable values (nested objects) — dropped.
- Every emitted def is validated against
attributeDefSchema.
Atomic five-spec write
// seedV3VersioningStore — applies in fixed order:
['pricingSpec', 'datasheet', 'attributes', 'flowSpec', 'workflowSpec']
// 1. validate every body against its strict V3 schema
// 2. refuse if org already has a global version (unless --force-reseed)
// 3. saveSpecVersion × 5
// 4. createGlobalVersion pointing at all five
7. Safety guards — env + org pinning
Both migrate.ts and initV3.ts run in dry-run by default. Any DB write goes through two independent guards, lifted into _utils/safety.ts:
--env production requires both a production DB host and process.env.RENDER set. Pointing it at a local/neon DB, or running off Render, fails fast. Without the flag, the script refuses any non-localhost/non-neon host.
assertOrgIs(orgId, expectedName, expectedSlug) fetches by id and hard-asserts both fields match exactly. A mistyped UUID can never let a migration act on another org. The per-org safety.ts just supplies the constants.
8. Verification — what the PR proves
97/97 products (incl. internal commitment) and 96/96 datapoints reproduced 1-1 through the registry seam.
pnpm run typecheck green across the monorepo.
Full Mocha suite green — 5131 passing. The catalogProductMapping test is updated to assert preservation of customEndDateOverrideInWeeks.
The verify.ts harness diffs through the real publish seam (the registry dispatch), bucketing diffs into acceptable (engine-neutral attributes mirror, isRequired: false → undefined) vs must-resolve. A non-clean run exits non-zero.
9. What this PR does not change
- No DB writes.
migrate.tsis dry-run by default;--writeupserts draft rows but isn't run here. - No publish.
publishDrafts(promotes drafts to live + writes V2) is a separate explicit step run from the Render shell. - No
pricingFlowTypeflip. Switching the org toDEALOPS_V3is deliberately decoupled from seeding. - No V3→V2 sync from
initV3.ts. That happens at publish via the per-org registry seam. - No change to other orgs' converters. The registry edit is purely additive; default-preserving for everyone else.
- Internal
commitmentstays the same shape.adminOnlyis leftfalseso the V2 round trip is byte-identical.
10. Order of operations (when this gets rolled out)
| Step | Command | Effect |
|---|---|---|
| 1 | migrate.ts --org-id=<uuid> --env production --write --user-id=<uuid> | Upsert draft catalog rows. Idempotent on (orgId, skuId, state='draft'). |
| 2 | APPLY=1 initV3.ts --org-id=<uuid> --env production | Seed the five OrgSpecVersion rows + one OrgGlobalVersion. Refuses if already seeded (use --force-reseed). |
| 3 | publish (publishDrafts) | Promote drafts → live; the registry seam writes V2 back from the catalog. |
| 4 | Flip Organization.pricingFlowType = DEALOPS_V3 | Cutover. Reads start using V3. |
11. Risks & open notes
customEndDateOverrideInWeeks are now copied verbatim in both directions for all orgs. They're additive and gated on presence (undefined stays absent), but any org that had a stale renewal-seed value in V2 will start round-tripping it through V3 instead of silently dropping it.
catalogProductMapping.test.ts previously asserted customEndDateOverrideInWeeks was dropped; it's now updated to assert preservation. That test is the contract — if a future refactor reverts the move, it fails loudly.
initV3.ts is rewritten on top of the shared helper. Net diff is +86 / −88, and the seeded shape changes from "pricingSpec + empty datasheet only" to "all five specs (with derived attributes + minimal flow/workflow)". If Xendit's versioning store has already been seeded in some environment with the old two-spec shape, a reseed would advance the global pointer — the guard requires --force-reseed to do this on purpose.