SDK / collector integration guide
How Simiaroom engineers instrument, validate, and ship events to CX Orbit without Orbit tribal knowledge.
This is an operator guide. It does not replace the wire contract. For envelope fields, HTTP status, retry, and dedup rules, use:
| Spec | Path |
|---|---|
| Envelope + collector HTTP contract | SDK-AND-DATA-INGESTION.md §7, §11 |
| Contract tests | collector-ingest-contract-tests.md |
| Producer of record | SOURCE-OWNERSHIP.md |
| Event contracts + fixtures | fixtures/events/README.md |
| Stack / hosting | adr/0001-stack-hosting-auth.md |
| Collector is sole write ingress | adr/0002-collector-sdk-outbox.md |
| Ops: rejects, dupes, lag, outage | ops/COLLECTOR-RUNBOOK.md |
Non-negotiable: SDKs POST to the collector only. Never DATABASE_URL, never @cx-orbit/db, never direct writes to Orbit storage.
1. What you are integrating
Simiaroom backend → outbox (T2.14) → relay POST /v1/events:batch → collector
↑
only write ingress (ADR 0002)
| Piece | Location | Role |
|---|---|---|
| Collector | apps/collector |
Auth, validate, quarantine, raw + canonical store |
| Backend SDK | packages/node (@cx-orbit/node) |
Typed P0 helpers + batch transport |
| P0 outbox | packages/outbox (@cx-orbit/outbox) |
Durable queue + relay for agreed P0 (T2.14) |
| Producer stub | apps/producer-stub |
Stand-in for Simiaroom services until product PRs land |
| Tracking plan | docs/fixtures/events/contracts.yaml + packages/shared/schemas/ |
Known types and JSON Schema |
| DQ dashboard | apps/web /dq |
Rejects, dupes, lag, missingness, queue |
P0 authoritative outcomes use typed SDK methods (cx.bookingConfirmed(...)). Generic track() is for development and approved non-P0 types only.
Analytics failures must not fail booking, payment, or session workflows. Commit the business fact first; emit afterward via the transactional outbox (ops/OUTBOX-RELAY.md, T2.14). Do not put agreed P0 outcomes on the in-memory SDK queue.
2. Local and staging endpoints
| Env | Collector | Web / DQ | Write key |
|---|---|---|---|
| local | http://127.0.0.1:8080 |
http://localhost:3000/dq |
.env COLLECTOR_SERVER_WRITE_KEY |
| staging | $COLLECTOR_PUBLIC_URL |
$WEB_PUBLIC_URL/dq |
Railway staging key (never local, never prod) |
CX_ORBIT_ENV on the SDK must match the collector (development | staging | production). Mismatch → item ENVIRONMENT_MISMATCH, quarantined.
# local boot (repo root)
cp .env.example .env # if needed; do not commit .env
pnpm --filter @cx-orbit/collector dev
Health (no auth):
curl -sS http://127.0.0.1:8080/health
# { "status": "ok", "service": "cx-orbit-collector", "environment": "development", ... }
Staging sketch: deploy/railway-staging.md.
3. How to instrument a new event
Do this before writing producer code. State-changing P0 events need a typed, reviewed contract (SDK-AND-DATA-INGESTION.md §10).
3.1 Decide ownership
- Confirm the outcome is authoritative (backend proves it) vs behavioral (browser, T2.13 — out of Wave B).
- Assign exactly one producer of record in
SOURCE-OWNERSHIP.md. - Pick
source(stable producer ID, e.g.simiaroom_booking) andsource_event_idconstruction:- Prefer the source system's outcome/row ID.
- If the type fires at most once per entity:
{canonical_type}:{entity.id}. - Never reuse a bare entity PK across confirm vs cancel (or any two types).
- Canonical idempotency key is
project_id + source + source_event_id. Transport retries keep the sameevent_id.
3.2 Contract + schema (tracking plan)
Copy an existing P0 type (template: booking_confirmed) and add:
| Artifact | Path |
|---|---|
| Contract metadata | docs/fixtures/events/contracts.yaml — new key under events: |
| JSON Schema | packages/shared/schemas/events/<canonical_type>.v1.schema.json |
| Valid fixture | docs/fixtures/events/<canonical_type>/valid.json |
| Invalid fixtures | .../invalid-missing-user-id.json, .../invalid-missing-required-property.json |
| Ownership row | docs/SOURCE-OWNERSHIP.md |
| Slice catalog (if in the P0 emit set) | docs/fixtures/staging/p0-backend-slice.json |
| Source ledger (reconcile) | docs/fixtures/staging/source-records.json |
Schema $id must be https://cx-orbit.dev/schemas/events/<canonical_type>.v1.json. File name: <canonical_type>.v<event_version>.schema.json.
Envelope required fields (all events): event_id, canonical_type, event_version, project_id, environment, source, source_event_id, occurred_at, identity, entity, actor, properties. P0 backend outcomes also require identity.user_id.
Privacy: no therapy notes, recordings, or raw assessment answers. Fixtures use synthetic IDs only (usr_123, bkg_789, …).
3.3 SDK typed helper
In @cx-orbit/node:
- Add a
P0_EVENTSentry inpackages/node/src/events.ts(canonicalType,source,entityType,eventVersion,journeyRef). - Add a properties TypeScript interface (camelCase; wire is snake_case).
- Add a method on
CxOrbitClientinpackages/node/src/client.ts(same pattern asbookingConfirmed). - Export types from
packages/node/src/index.tsif they are part of the public API.
Do not import @cx-orbit/db or read DATABASE_URL in the SDK.
3.4 Producer (Simiaroom or stub)
Until the Simiaroom product repo is wired, apps/producer-stub is the producer-of-record stand-in.
- Point
p0-backend-slice.jsonat the new type, producersource, andsdk_method. - Add a matching row in
source-records.json(source_event_idmust match the fixture). - Emit only from the owning producer — ownership tests reject the wrong service.
Live Simiaroom services: after business commit, the P0 envelope must already be in cx_outbox_events (same DB transaction). Relay with @cx-orbit/outbox / relayOnce — do not flush() P0 through the in-memory SDK queue. Preserve event_id across retries. Guide: ops/OUTBOX-RELAY.md.
3.5 Validate the contract locally (no collector required)
pnpm --filter @cx-orbit/shared test
# or
node packages/shared/scripts/validate-event-fixtures.mjs
Valid fixtures must pass JSON Schema; invalid fixtures must fail.
pnpm --filter @cx-orbit/node test
pnpm --filter @cx-orbit/producer-stub test
4. Dry-run: dummy event type (do not add to the tracking plan)
Use this to prove the collector rejects unknown types instead of silently dropping them. Do not merge a dummy type into contracts.yaml.
environment must match the running collector (development locally unless you changed CX_ORBIT_ENV).
set -a && source .env && set +a
ENV="${CX_ORBIT_ENV:-development}"
COLLECTOR="${COLLECTOR_URL:-http://127.0.0.1:8080}"
# 1. Collector up
curl -sS "$COLLECTOR/health" | jq '{status, environment, service}'
# 2. Unknown type — expect HTTP 422, status=rejected, UNKNOWN_EVENT_TYPE
curl -sS -D - -o /tmp/t411-dummy.json -X POST "$COLLECTOR/v1/events:batch" \
-H "Authorization: Bearer $COLLECTOR_SERVER_WRITE_KEY" \
-H "Content-Type: application/json" \
-d "{
\"events\": [{
\"event_id\": \"0198a1c2-dddd-7e8f-9a0b-1c2d3e4f5a6b\",
\"canonical_type\": \"dummy_event_type\",
\"event_name\": \"dummy_event_type\",
\"event_version\": 1,
\"project_id\": \"simiaroom\",
\"environment\": \"$ENV\",
\"source\": \"simiaroom_booking\",
\"source_event_id\": \"dummy_src_001\",
\"occurred_at\": \"$(date -u +%Y-%m-%dT%H:%M:%S.000Z)\",
\"identity\": { \"anonymous_id\": null, \"user_id\": \"usr_123\", \"session_id\": null },
\"entity\": { \"type\": \"booking\", \"id\": \"bkg_dummy\" },
\"actor\": { \"type\": \"system\", \"id\": \"t4.11-dry-run\" },
\"properties\": {}
}]
}"
jq '{http_hint: .status, request_id, accepted, rejected, quarantined, results}' /tmp/t411-dummy.json
Expected
| Check | Value |
|---|---|
| HTTP | 422 |
status |
rejected |
accepted |
0 |
rejected |
1 |
quarantined |
1 |
results[0].code |
UNKNOWN_EVENT_TYPE |
results[0].retryable |
false |
results[0].outcome |
quarantined |
Do not retry this item. To actually ship a new type, follow §3 (contract → schema → typed helper → producer), then re-POST a valid envelope.
Unknown-type counters also move on GET /metrics (rejected, quarantined, quarantine.pending).
File-checklist dry-run (existing type)
Confirm the §3.2 paths exist for the template type — no new files required:
test -f packages/shared/schemas/events/booking_confirmed.v1.schema.json
test -f docs/fixtures/events/booking_confirmed/valid.json
rg -n "booking_confirmed:" docs/fixtures/events/contracts.yaml
rg -n "bookingConfirmed" packages/node/src/events.ts packages/node/src/client.ts
rg -n "booking_confirmed" docs/fixtures/staging/p0-backend-slice.json docs/SOURCE-OWNERSHIP.md
5. How to validate in staging (or local stand-in)
Railway staging uses the same commands with COLLECTOR_URL=$COLLECTOR_PUBLIC_URL and CX_ORBIT_ENV=staging. Local is the default stand-in when the Railway collector is not up.
5.1 Happy path — 12-event P0 slice
# terminal 1
pnpm --filter @cx-orbit/collector dev
# terminal 2
set -a && source .env && set +a
pnpm --filter @cx-orbit/producer-stub emit
# one type:
# pnpm --filter @cx-orbit/producer-stub emit -- --type booking_confirmed
Expect JSON: emitted: 12, flush.accepted: 12 (or accepted + duplicate covering all emitted), quarantined: 0. Exit 0.
Inspect canonical key refs (write-key gated; no property dump):
curl -sS http://127.0.0.1:8080/v1/debug/store \
-H "Authorization: Bearer $COLLECTOR_SERVER_WRITE_KEY" | jq '{store, raw_count, canonical_count, types: [.events[].canonical_type]}'
Expect raw_count and canonical_count ≥ 12 after a fresh collector process (in-memory store resets on restart).
5.2 Typed SDK smoke (optional)
set -a && source .env && set +a
COLLECTOR_URL=http://127.0.0.1:8080 ./apps/collector/node_modules/.bin/tsx packages/node/scripts/smoke-flush.ts
5.3 Collector outage — outbox then relay (T2.14)
# collector stopped
set -a && source .env && set +a
pnpm --filter @cx-orbit/producer-stub emit -- --outbox ./cx-outbox.json
# expect pending: 12, exit 0
# terminal 1 — start collector
pnpm --filter @cx-orbit/collector dev
# terminal 2
pnpm --filter @cx-orbit/outbox relay -- --store ./cx-outbox.json
# expect tick.delivered: 12 (or delivered + duplicate covering the set)
Full drill + replay/dedup: ops/OUTBOX-RELAY.md.
5.4 Reconciliation vs source ledger
set -a && source .env && set +a
# Script name is `run`; `pnpm … run` only lists scripts (exit 0). Use `run run`.
pnpm --filter @cx-orbit/reconcile-job run run
Expect "within_dq_target": true, "completeness_pct": 100, "mismatch_count": 0, exit 0.
Pilot bars (SDK-AND-DATA-INGESTION.md §13): completeness ≥ 99%, duplicate rate after canonicalization ≤ 0.1%.
Write the report where /dq can read it:
RECONCILE_REPORT_PATH=docs/fixtures/dq/last-reconcile-report.json pnpm --filter @cx-orbit/reconcile-job run run
5.5 DQ dashboard
pnpm --filter @cx-orbit/web dev
# live: http://localhost:3000/dq
# API: curl -sS 'http://localhost:3000/api/dq' | jq '{pilotGate: .view.pilotGate, widgets: [.view.widgets[] | {id, actual, status}]}'
Live page polls collector GET /metrics every 30s. Completeness / canonical duplicate rate come from the last reconcile JSON (RECONCILE_REPORT_PATH).
Fixture inject (no live collector; widgets still move):
| URL | Widget that moves |
|---|---|
/dq?fixture=healthy |
All within target |
/dq?fixture=reject |
Rejects |
/dq?fixture=dup |
Duplicates |
/dq?fixture=lag |
Lag |
/dq?fixture=missing |
Missingness |
5.6 Staging checklist (before calling an event “validated”)
- Contract + schema + fixtures green (
@cx-orbit/sharedtests) - Typed helper exists; producer of record documented
-
CX_ORBIT_ENVand write key match the target collector - Emit accepted (
outcomeacceptedoraccepted_with_warnings); notUNKNOWN_EVENT_TYPE - Event visible in
GET /v1/debug/store?canonical_type=<type> - Reconcile job matches
source_event_idinsource-records.json(or Simiaroom source table) -
/dqrejects widget is not accumulating new quarantine for this type - Duplicate resubmit of the same envelope returns
DUPLICATE_EVENT_ID(success — do not treat as loss)
6. HTTP cheat sheet (do not retry blindly)
| Client sees | Retry this request? |
|---|---|
| Network error, 408, 429, 500, 502, 503, 504 | Yes — same event_id; honor Retry-After |
| 200 / 207 | No — inspect results[]; only items with retryable: true |
| 400, 401, 403, 404, 413, 422 | No — fix payload or config |
Item duplicate |
No — success |
Item UNKNOWN_EVENT_TYPE / SCHEMA_VALIDATION_FAILED / IDEMPOTENCY_CONFLICT |
No — fix contract or producer; replay via ops after fix |
Full catalog: SDK-AND-DATA-INGESTION.md §11.5–11.6.
7. Residual limits (honest)
- Collector HTTP persist to Postgres is not wired;
GET /v1/debug/storeis the in-memory inspect API and resets on process restart. - There is no public quarantine-payload inspector (counters only on
/metricsand/dq). Itemresults[]on the ingest response is the per-event diagnosis. - Durable Simiaroom outbox is shipped (
@cx-orbit/outbox, T2.14). Agreed P0 outcomes must use it; in-memory SDK queues can still drop non-outbox traffic. - Browser SDK (T2.13) is P1; do not instrument RET-01 from
simiaroom_booking. - Live Railway hostnames are assigned per environment; this guide uses env vars, not hardcoded URLs.