#SDK & Data Ingestion — CX Orbit MVP

This document defines how Simiaroom systems send data to CX Orbit. It supplements the product and journey source documents; it does not replace their event definitions.

#1. Decision summary

CX Orbit will expose a versioned event protocol and collector API. Thin SDKs make that protocol convenient to use.

The SDK is not the data platform and must not contain lifecycle, churn, risk, or clinical interpretation logic.

Recommended integration order:

1. Canonical schemas and collection API
2. Backend SDK for authoritative business events
3. CDC/batch reconciliation for existing operational records
4. Browser SDK for behavioral funnel detail
5. Additional mobile or language SDKs only when a real integration requires them

#2. Provider patterns adopted

The design follows recurring patterns from established event and telemetry providers:

For a mental-health product, explicit instrumentation is preferred over broad DOM autocapture.

#3. Collection paths

#3.1 Browser SDK

The browser SDK records behavior and client-side context:

• Page and entry views
• Assessment start, progress, completion intent, and abandonment
• Therapist list/profile/availability views
• Booking flow interactions
• Payment UI interactions
• Form exposure and submission intent
• Client-side technical failures that affect the journey

Browser events are not authoritative for payment, booking, session, refund, pause, completion, or case outcomes.

#3.2 Backend SDK

The backend SDK records authoritative domain outcomes:

• Assessment completed
• Referral created
• Booking confirmed/cancelled
• Payment succeeded/failed/refunded
• Session completed/cancelled/no-show
• Therapist change completed
• Pause declared
• Treatment completed
• Internal case resolved

Every authoritative event must have exactly one producer of record.

#3.3 CDC, API sync, and imports

SDK instrumentation does not replace historical and reconciliation data. Use controlled CDC, APIs, webhooks, or batch imports for:

• Historical clients, therapists, bookings, sessions, payments, and refunds
• Package balances and current operational status
• Porsline/MyOutcomes responses
• Support tickets and operational follow-ups
• Corrections or records created outside instrumented application paths

#4. Logical collection architecture

flowchart LR
  Web[Browser SDK] --> Collector[CX Orbit Collector API]
  Backend[Backend SDK] --> Collector
  Mobile[Future Mobile SDK] --> Collector
  Hooks[Webhooks] --> Collector
  CDC[CDC / Scheduled Sync] --> Map[Source Mapping]
  Imports[File Imports] --> Map

  Collector --> Auth[Auth + Rate Limits]
  Auth --> Validate[Schema + Privacy Validation]
  Validate -->|valid| Queue[Durable Buffer / Queue]
  Validate -->|invalid| Quarantine[Restricted Quarantine]
  Queue --> Raw[(Immutable Raw Events)]
  Map --> Raw
  Raw --> Canonical[Canonical Mapping + Identity]
  Canonical --> Operational[(PostgreSQL Operational Store)]
  Canonical --> Analytics[(Analytical Store)]
  Operational --> API[CX Orbit API]
  Analytics --> API

SDKs must never connect directly to PostgreSQL, ClickHouse, or another warehouse.

#5. Package boundaries

Recommended initial packages:

@cx-orbit/core
  Event envelope, transport interfaces, common context, errors

@cx-orbit/events
  Generated event types and domain methods

@cx-orbit/browser
  Anonymous identity, sessions, attribution, consent, queue, browser transport

@cx-orbit/node
  Server authentication, queue, batch transport, flush/shutdown hooks

cx-orbit-collector
  HTTP ingestion, authentication, validation, privacy controls, routing

cx-orbit-schema
  Tracking plan and versioned JSON Schemas

Do not build SDKs for languages Simiaroom does not currently use. Unsupported services can call the HTTP API directly.

#6. Public SDK API

Keep a small provider-compatible core:

cx.track("therapist_profile_viewed", properties);
cx.identify("user_123", traits);
cx.page("therapist_profile", properties);
cx.flush();
cx.reset();
cx.setConsent(consent);

Production domain events should use generated typed methods:

cx.therapistProfileViewed({
  therapistId: "thr_123",
  referralId: "ref_456",
  optionRank: 2,
});

cx.bookingConfirmed({
  bookingId: "bkg_123",
  therapistId: "thr_123",
  scheduledStartUtc: "2026-08-10T14:00:00Z",
  bookingContext: "first_session",
});

The generic track API is allowed for development and approved behavioral events. State-changing P0 events require a typed, reviewed contract.

#7. Canonical event envelope v1 (locked)

This section is the collector/SDK wire contract. It matches the lock in MVP-EVENT-SCOPE.md. Field names below are authoritative for storage and validation.

#7.1 Required sender fields

Collector-owned fields such as received_at, authentication result, validation status, ingestion ID, and collector version must not be trusted from the sender.

Public SDK methods may keep camelCase arguments (bookingId); the envelope on the wire is snake_case as shown here. Typed methods still emit this envelope — they do not write storage.

#7.2 Identity refs

"identity": {
  "anonymous_id": "string|null",
  "user_id": "string|null",
  "session_id": "string|null"
}

Rules (same as §9, applied to the envelope):

• identity.session_id is the analytics/web session, not the therapy session
• Authoritative P0 backend outcomes require identity.user_id
• Browser events may omit user_id until identify / identity_link_created
• Flat aliases anonymous_id / user_id / actor_type from older sketches are accepted only if they match the nested objects

#7.3 Entity refs

"entity": { "type": "string", "id": "string" }

entity is the primary business object for the event. Related IDs (therapist_id, booking_id, payment_id, session_id, …) go in properties and must not contradict entity.

#7.4 Canonical idempotency key

Locked key: project_id + source + source_event_id.

This is source + source_event_id scoped by project_id. Transport-level project_id + event_id remains for SDK retries (§8.2) and does not replace this key.

source_event_id construction:

1. Prefer the source system's outcome/row ID (verification ID, confirmation event ID, completion record ID)
2. If none exists and the type fires at most once per entity: {canonical_type}:{entity.id}
3. Never use a bare entity PK as source_event_id when that entity can emit more than one canonical type (confirm vs cancel)

#7.5 Envelope example — `booking_confirmed` (BKG-S03)

{
  "event_id": "0198a1c2-3b4d-7e8f-9a0b-1c2d3e4f5a6b",
  "canonical_type": "booking_confirmed",
  "event_name": "booking_confirmed",
  "event_version": 1,
  "occurred_at": "2026-08-03T10:20:30.123Z",
  "sent_at": "2026-08-03T10:20:31.004Z",
  "project_id": "simiaroom",
  "environment": "production",
  "source": "simiaroom_backend",
  "source_event_id": "bkg_evt_confirm_789",
  "source_version": "3.18.0",
  "sdk": {
    "name": "@cx-orbit/node",
    "version": "0.2.0"
  },
  "actor": {
    "type": "system",
    "id": "booking-service"
  },
  "identity": {
    "anonymous_id": null,
    "user_id": "usr_123",
    "session_id": null
  },
  "entity": {
    "type": "booking",
    "id": "bkg_789"
  },
  "journey_ref": "J-04",
  "context": {
    "request_id": "req_123",
    "timezone": "Europe/Berlin",
    "locale": "fa"
  },
  "properties": {
    "therapist_id": "thr_456",
    "payment_id": "pay_456",
    "slot_id": "slot_1001",
    "session_number": 1,
    "user_timezone": "Europe/Berlin",
    "booking_context": "first_session",
    "scheduled_start_utc": "2026-08-10T14:00:00Z"
  }
}

context.timezone / properties.user_timezone may be a diaspora zone. Storage timestamps stay UTC; Orbit UI still displays Asia/Tehran (§8.6).

#7.6 Further P0 examples

#`payment_succeeded` (PAY-06)

{
  "event_id": "0198a1c2-aaaa-7e8f-9a0b-1c2d3e4f5a6b",
  "canonical_type": "payment_succeeded",
  "event_name": "payment_succeeded",
  "event_version": 1,
  "occurred_at": "2026-08-03T10:20:28.000Z",
  "sent_at": "2026-08-03T10:20:29.100Z",
  "project_id": "simiaroom",
  "environment": "production",
  "source": "simiaroom_backend",
  "source_event_id": "pay_ver_456",
  "source_version": "3.18.0",
  "sdk": {
    "name": "@cx-orbit/node",
    "version": "0.2.0"
  },
  "actor": {
    "type": "system",
    "id": "payment-verification"
  },
  "identity": {
    "anonymous_id": null,
    "user_id": "usr_123",
    "session_id": null
  },
  "entity": {
    "type": "payment",
    "id": "pay_456"
  },
  "journey_ref": "J-05",
  "context": {
    "request_id": "req_122",
    "timezone": "Asia/Tehran",
    "locale": "fa"
  },
  "properties": {
    "order_id": "ord_789",
    "booking_id": "bkg_789",
    "package_id": "pkg_12",
    "amount": 2500000,
    "currency": "IRR",
    "gateway": "zarinpal",
    "failure_code": null
  }
}

#`session_completed` (SES-06)

{
  "event_id": "0198a1c2-bbbb-7e8f-9a0b-1c2d3e4f5a6b",
  "canonical_type": "session_completed",
  "event_name": "session_completed",
  "event_version": 1,
  "occurred_at": "2026-08-10T14:50:00.000Z",
  "sent_at": "2026-08-10T14:50:02.400Z",
  "project_id": "simiaroom",
  "environment": "production",
  "source": "simiaroom_backend",
  "source_event_id": "ses_complete_321",
  "source_version": "3.18.0",
  "sdk": {
    "name": "@cx-orbit/node",
    "version": "0.2.0"
  },
  "actor": {
    "type": "staff",
    "id": "stf_88"
  },
  "identity": {
    "anonymous_id": null,
    "user_id": "usr_123",
    "session_id": null
  },
  "entity": {
    "type": "session",
    "id": "ses_321"
  },
  "journey_ref": "J-06",
  "context": {
    "request_id": "req_900",
    "timezone": "Asia/Tehran",
    "locale": "fa"
  },
  "properties": {
    "booking_id": "bkg_789",
    "therapist_id": "thr_456",
    "session_number": 1,
    "attendance_status": "completed",
    "actual_duration": 50
  }
}

Schema-narrative checks for these examples: required envelope fields present; canonical_type === event_name; identity.user_id set; entity matches the primary ID in properties where overlapping; occurred_at/sent_at are UTC Z; source_event_id is an outcome ID, not a reused entity PK across types.

#8. Delivery semantics

#8.1 Non-blocking application behavior

Analytics failures must not fail bookings, payments, sessions, or other primary workflows.

Authoritative business transactions complete in the Simiaroom system of record first. Tracking is attempted afterward, preferably using an outbox pattern where feasible.

#8.2 At-least-once delivery and idempotency

The pipeline assumes at-least-once delivery. Exactly-once claims are out of scope.

Transport (SDK retry of the same envelope):

• Generate event_id before the first send
• Preserve the same ID through all retries
• Deduplicate raw ingest by project_id + event_id

Canonical (one source fact → one business event):

• Idempotency key = project_id + source + source_event_id (locked; see §7.4)
• CDC, import, webhook, and SDK paths for the same source fact must share source + source_event_id

Do not promise global event order. Use occurred_at, sent_at, and collector received_at to reason about late and out-of-order data.

#Duplicates

HTTP item codes for these rows are locked in §11.8.

Retries and full or partial batch resubmits must be safe under these rules.

#Late events

An event is late when received_at - occurred_at exceeds the threshold, or when a fact with a later occurred_at for the same client/entity was already applied.

Late events are still stored. Event time for history and metrics is occurred_at, not arrival time. Derived profiles and funnels recompute from ordered history. Events older than the threshold are flagged for data quality; they are not dropped.

#Out-of-order events

Arrival order is not event order.

• Interpretation: Current Event + Current State + Accumulated History ordered by occurred_at then event_id
• An earlier fact arriving after a later one is appended to history and triggers derived-state recompute (SYS-01 path)
• Do not rewrite a previously accepted canonical payload; fix with a compensating event
• Stage, session numbering, and risk must not depend on ingest order

#8.3 Batching

Initial defaults:

Requests should remain below an agreed payload limit, initially 256 KB.

#8.4 Retry policy

HTTP status classes, per-item retryable flags, and backoff defaults are locked in §11. Summary:

• Retry transport failures and transient statuses: 408, 429, 500, 502, 503, 504. Honor Retry-After.
• Do not retry permanent failures: 400, 401, 403, 404, 413, 422 — fix the request or configuration first.
• A processed batch (200 / 207 / item-level 422) is not retried as a whole. Inspect results[]; retry only items with retryable: true (schema and idempotency conflicts are never retryable as-is).
• Preserve event_id across retries. Bounded attempts or elapsed time. Queue overflow and dropped events must be observable.

#8.5 Browser navigation

Preferred order:

1. fetch with keepalive and small batches
2. Explicit flush on pagehide
3. sendBeacon only as a fallback because it does not provide reliable response handling

#8.6 Timezone policy (locked)

• Store occurred_at, sent_at, and collector received_at in UTC (Z)
• Convert sender-local / offset timestamps to UTC at the collector; keep the sender zone in context.timezone when known
• properties.user_timezone and slot local-time fields are scheduling metadata, not the storage clock
• Display in CX Orbit UI and default calendar-day metrics: Asia/Tehran (IANA)

#9. Identity contract

The SDK may maintain anonymous_id, user_id, and session_id, subject to consent.

Browser consent, first-party ID storage, attribution-window alignment, and send-minimization are locked in BROWSER-CONSENT-POLICY.md (T1.14). This section remains the identity-link contract (ID-01 / T2.3). @cx-orbit/browser (T2.13, P1) must not contradict that policy. Backend @cx-orbit/node is not consent-gated.

Anonymous visit: anonymous_id=anon_A, user_id=null
Valid login:     identify(user_123)
Known activity:  anonymous_id=anon_A, user_id=user_123
Logout:          clear user_id and rotate session identity as required

Rules:

• Emit an explicit identity-link event; do not rewrite immutable raw events
• Same-device authenticated linkage may be deterministic
• Verified OTP/email/phone linkage may be deterministic within the approved window
• IP-only evidence never merges identities
• Shared devices must not merge separate accounts
• Cross-domain identifiers must be short-lived, integrity-protected, and approved by security

#10. Schema governance

Every event contract must define:

name: booking_confirmed
canonical_type: booking_confirmed
version: 1
owner: booking-team
description: A booking became operationally confirmed.
authoritative_source: backend
privacy_class: operational
required:
  - booking_id
  - user_id
  - therapist_id
  - scheduled_start_utc
properties:
  booking_context:
    type: string
    enum: [first_session, follow_up, return]

name and canonical_type are the same identifier. Envelope validation also requires event_id, source, source_event_id, occurred_at, identity, entity, and properties as specified in §7.

Contract lifecycle:

Draft → Approved → Implemented → Staging validated
→ Production monitored → Deprecated

Breaking meaning, type, or required-field changes create a new event version. Multiple supported versions may coexist during rollout.

#11. Collector ingest contract (locked T1.13)

This section is the HTTP wire contract for cx-orbit-collector. T2.2 implements it; T2.2a SDKs consume it; T2.2c relies on the dedup rules. It must not contradict the T0.4 envelope lock (§7–§8.2) or ADR 0002.

Normative test plan: collector-ingest-contract-tests.md (executable in T2.2).

Invariant: no silent drop. Every event in a processed batch appears in results[] with an explicit outcome. Invalid items are reported and, when enough identity exists, quarantined. Valid items are accepted or deduplicated — never discarded without a result row.

#11.1 Endpoint and request

Each HTTP request receives a new collector request_id (prefix ing_, then ULID or UUID). Retries of the same envelopes get a new request_id; dedup is by event_id / canonical key, not request_id. Sender context.request_id is correlational only.

CDC, import, and webhook paths use the same validation, quarantine, and dedup rules via the collector (ADR 0002 internal ingest path). They do not write storage directly.

#11.2 HTTP status vs batch `status`

Two layers:

1. Request-level — the collector did not process items (auth, parse, size, availability). Body is an error object (status: "error"). No results[].
2. Batch-processed — every item was classified. Body is a batch result with status: "ok" | "partial" | "rejected" and a results[] entry per input index.

207 is still 2xx (fetch ok === true). SDKs must read status and results[], not assume HTTP 200-only.

If the collector durably accepted events and then fails to respond, the client retries. Transport and canonical dedup make that retry safe (§11.8).

#11.3 Success / partial / reject response shapes

Shared batch-processed fields:

accepted + duplicate + rejected equals events.length. quarantined ≤ rejected.

#Success — HTTP 200

{
  "status": "ok",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y2",
  "received_at": "2026-08-12T16:54:00.123Z",
  "accepted": 2,
  "duplicate": 1,
  "rejected": 0,
  "quarantined": 0,
  "results": [
    {
      "index": 0,
      "event_id": "0198a1c2-3b4d-7e8f-9a0b-1c2d3e4f5a6b",
      "outcome": "accepted",
      "code": "ACCEPTED",
      "retryable": false,
      "quarantine_id": null,
      "path": null,
      "message": null
    },
    {
      "index": 1,
      "event_id": "0198a1c2-3b4d-7e8f-9a0b-1c2d3e4f5a6b",
      "outcome": "duplicate",
      "code": "DUPLICATE_EVENT_ID",
      "retryable": false,
      "quarantine_id": null,
      "path": null,
      "message": "Already ingested; canonical fact unchanged"
    },
    {
      "index": 2,
      "event_id": "0198a1c2-cccc-7e8f-9a0b-1c2d3e4f5a6b",
      "outcome": "accepted_with_warnings",
      "code": "LATE_EVENT",
      "retryable": false,
      "quarantine_id": null,
      "path": null,
      "message": "received_at - occurred_at exceeds P0 late threshold; stored"
    }
  ]
}

Duplicates are success. The client must not retry them and must not treat them as loss.

#Partial — HTTP 207

{
  "status": "partial",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y3",
  "received_at": "2026-08-12T16:54:01.000Z",
  "accepted": 18,
  "duplicate": 0,
  "rejected": 2,
  "quarantined": 2,
  "results": [
    {
      "index": 0,
      "event_id": "evt_ok_1",
      "outcome": "accepted",
      "code": "ACCEPTED",
      "retryable": false,
      "quarantine_id": null,
      "path": null,
      "message": null
    },
    {
      "index": 8,
      "event_id": "evt_9",
      "outcome": "quarantined",
      "code": "SCHEMA_VALIDATION_FAILED",
      "retryable": false,
      "quarantine_id": "q_01j8q2k4n6p8r0s2t4v6w8x0z1",
      "path": "properties.therapist_id",
      "message": "required property missing"
    }
  ]
}

The example omits middle results rows; a real response includes all indexes 0 … n-1.

#Reject (all items) — HTTP 422, batch processed

{
  "status": "rejected",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y4",
  "received_at": "2026-08-12T16:54:02.000Z",
  "accepted": 0,
  "duplicate": 0,
  "rejected": 2,
  "quarantined": 2,
  "results": [
    {
      "index": 0,
      "event_id": "evt_a",
      "outcome": "quarantined",
      "code": "MISSING_REQUIRED_FIELD",
      "retryable": false,
      "quarantine_id": "q_01j8q2k4n6p8r0s2t4v6w8x0z2",
      "path": "source_event_id",
      "message": "source_event_id is required"
    },
    {
      "index": 1,
      "event_id": null,
      "outcome": "quarantined",
      "code": "MISSING_REQUIRED_FIELD",
      "retryable": false,
      "quarantine_id": "q_01j8q2k4n6p8r0s2t4v6w8x0z3",
      "path": "event_id",
      "message": "event_id is required"
    }
  ]
}

event_id in results[] may be null when the item omitted it. index is still required so nothing is silent.

#Reject (request-level) — HTTP 4xx/5xx, not processed

{
  "status": "error",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y5",
  "received_at": "2026-08-12T16:54:03.000Z",
  "code": "AUTH_INVALID",
  "retryable": false,
  "message": "Missing or invalid write key"
}

Rate-limit example (HTTP 429, header Retry-After: 12):

{
  "status": "error",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y6",
  "received_at": "2026-08-12T16:54:04.000Z",
  "code": "RATE_LIMITED",
  "retryable": true,
  "message": "Rate limit exceeded",
  "retry_after_seconds": 12
}

Unavailable example (HTTP 503, Retry-After optional):

{
  "status": "error",
  "request_id": "ing_01j8q2k4n6p8r0s2t4v6w8x0y7",
  "received_at": "2026-08-12T16:54:05.000Z",
  "code": "COLLECTOR_UNAVAILABLE",
  "retryable": true,
  "message": "Collector unavailable"
}

#11.4 Per-item `results[]`

Prefer quarantined over bare rejected whenever a sanitized payload can be stored.

#11.5 Transient vs permanent codes

retryable on the response is authoritative. This catalog is the default mapping T2.2 must implement.

#Permanent — do not retry as-is (`retryable: false`)

Schema-invalid, privacy-invalid, and conflict items are not automatically retried. Fix the producer or payload, then replay from quarantine (§11.7) if eligible.

#Transient — retry (`retryable: true`)

Item-level transient codes are not used in MVP: once the collector is processing a batch, each item is classified synchronously into a permanent outcome. If the process fails mid-batch, the client sees a request-level 5xx and retries the whole batch; dedup prevents double-counting.

#Success codes (`retryable: false` — already done)

#11.6 Client retry guidance

Applies to @cx-orbit/node, outbox relays, and raw HTTP callers. Browser SDK (P1) uses the same classes with a shorter budget (§8.5).

Retry the HTTP request when:

• TCP/TLS/DNS failure, or no response
• HTTP 408, 429, 500, 502, 503, 504
• status: "error" and retryable: true

Do not retry the HTTP request when:

• HTTP 400, 401, 403, 404, 413, 422
• HTTP 200 or 207 (batch was processed)

On 207 / item 422: drop accepted, accepted_with_warnings, and duplicate items from the outbox/queue. Do not resubmit them. Resubmit only items with retryable: true (none for schema/conflict in MVP). Permanent item failures go to the Simiaroom dead-letter / alert path (ADR 0002 §5.1), not an infinite collector retry.

Full or partial batch resubmit is safe under §11.8 even if the client resends already-accepted items (for example after a 500). Prefer filtering to reduce load; correctness does not depend on filtering.

Backoff (backend / outbox defaults):

Preserve event_id generated before the first send. Never rotate IDs on retry.

Overflow: if the in-memory queue is full, increment a dropped-event metric. P0 authoritative outcomes must use the transactional outbox so they are not in that in-memory path (ADR 0002 L1).

Analytics failures must not fail booking, payment, or session workflows (§8.1).

#11.7 Quarantine fields and replay

Invalid or conflicting items write event_quarantine (see also DATA-MODEL.md §10). Retention default: 90 days (T0.7). Restricted access; sanitized payload only.

#Required quarantine fields

retry_eligible on the row matches retryable on the HTTP result. Schema-invalid and IDEMPOTENCY_CONFLICT rows are retry_eligible: false.

#Replay path

Replay is an Orbit ops / data-eng action after the producer or schema is fixed. SDKs do not call replay. Automatic retry of quarantined schema errors is forbidden.

1. Inspect quarantine (redacted inspector / DQ dashboard — T2.11)
2. Fix producer, schema, or confirm the row should be discarded
3. Replay: submit the corrected envelope through the collector ingest path
   (same auth, validation, privacy, dedup as a live SDK POST)
4. On accept or compatible duplicate: set replay_status = replayed;
   store replayed_at and replay_request_id. Do not delete the quarantine row.
5. On another validation failure: leave pending; append a new quarantine row
   (do not overwrite the original)
6. Discard: replay_status = discarded with actor + reason (audit)

Replay rules:

• Replayed envelopes should keep the original event_id and source_event_id when those were valid, so transport/canonical dedup stays stable.
• If event_id was missing, generate one once at replay; record it on the quarantine row.
• IDEMPOTENCY_CONFLICT: do not replay as-is. Investigate. Never overwrite the existing canonical fact. Correction is a compensating event with a new source_event_id (T0.4).
• EVENT_ID_REUSED: do not replay the conflicting body under the same event_id.
• Replay is not a backdoor around validation.

#11.8 Dedup behavior on resubmit

Locked keys (T0.4):

CDC, import, webhook, and SDK copies of the same source fact must share source + source_event_id (each path may mint a new transport event_id).

Material fields for “compatible” vs “conflict” (canonical key already matches):

• canonical_type (after alias normalization)
• identity.user_id, identity.anonymous_id
• entity.type, entity.id
• occurred_at (normalized UTC)
• Required properties from the event contract (T1.12)

First accepted envelope wins. Extra optional properties on a later compatible submit are ignored. Do not rewrite the stored canonical payload.

Counts, funnels, and operational state must not double on any row in this table.

#11.9 Contract tests (implement in T2.2)

The catalog below is normative. Fixtures, assertions, and file layout: collector-ingest-contract-tests.md.

T2.2 must also cover authenticated ingest, rate-limit enforcement, and metrics counters (accepted, rejected, quarantined, latency) per that task’s DoD. CT-13 is the contract shape for rate limits; numeric thresholds are a T2.2 implementation choice.

#11.10 Skeleton vs T2.2

apps/collector returns §11.3 success and request-level error shapes (status, request_id, counts, results on 200; code / retryable on 401/400). It does not yet persist, validate per-type schemas, quarantine, or dedup. Persistence, CT-02–CT-20, and metrics land in T2.1 / T2.2 / T2.2c.

#12. Security and privacy defaults

Browser-side consent gate, anonymous ID storage, and the denylist of fields/classes the browser must not send: BROWSER-CONSENT-POLICY.md (T1.14). This section remains collector/SDK security defaults. T2.13 implements the browser half; P0 server-key ingest is not consent-gated.

• Separate restricted browser write keys from private server credentials
• Restrict browser keys by project, origin, environment, rate, and permitted event classes
• Never put server credentials in browser or mobile packages
• Do not collect therapy notes, recordings, or session content
• Do not send raw sensitive assessment or feedback answers in behavioral events
• Use field allowlists rather than property denylists for P0 events
• Provide setConsent, reset, and configurable memory/cookie/local-storage behavior
• Remove or anonymize IP data unless explicitly required and approved
• Mask or tokenize direct identifiers before general analytical storage
• Keep sensitive ORS/SRS and free-text feedback in restricted storage and expose aggregated/minimal views
• Audit sensitive-profile access

#13. Observability and developer experience

Operator docs (T4.11): how to instrument and validate — INTEGRATION-GUIDE.md. On-call rejects / dupes / lag / outage — ops/COLLECTOR-RUNBOOK.md. This section stays the SLO and tooling contract; those guides do not replace §7 / §11.

Required MVP tools:

• Staging project and keys
• Live event inspector with sensitive values redacted
• Search by event_id and request_id
• Schema fixture validator in CI
• Accepted/rejected/quarantined counters
• Ingestion latency and source freshness
• Queue depth, retry count, and dropped-event metrics
• SDK debug logging disabled by default in production
• Reconciliation reports against authoritative source tables

Suggested service objectives for P0 backend events — locked for pilot in DATA-QUALITY-BAR.md (T1.10). T2.11 /dq measures every row except collector availability.

Go-live FAIL/UNKNOWN/PASS: DATA-QUALITY-BAR §6. Do not loosen these numbers without a T1.10 reopen. Re-measure volume in T4.1 / T4.12 (volume_v2).

#14. Storage decision: PostgreSQL and ClickHouse

Use PostgreSQL for operational current state:

• Internal users and permissions
• Cases, actions, owners, statuses, and SLAs
• Rule/configuration versions
• Current canonical entity state required by transactional APIs

ClickHouse is a suitable analytical event store for high-volume event history, funnels, cohorts, retention, and interactive dashboards. It is not the system of record for booking, payment, or case workflows.

MVP decision rule:

• Start with partitioned PostgreSQL events and reporting tables when volume/concurrency are modest
• Add ClickHouse when event volume, retention, or interactive query latency provides a measured benefit
• If ClickHouse is selected early, prefer a managed deployment and avoid introducing self-managed Kafka without a demonstrated need
• Keep collector and schema contracts storage-neutral so the database choice can change without reinstrumenting Simiaroom

#15. MVP rollout

#Stage 1 — Protocol and vertical slice

• Approve event envelope v1 — spec locked (T0.4) in this document §7–§8.6 and MVP-EVENT-SCOPE.md
• Approve collector HTTP contract — spec locked (T1.13) in this document §11 and collector-ingest-contract-tests.md
• Implement collector authentication and validation
• Implement 8–12 authoritative P0 backend events
• Store raw + canonical events
• Produce one verified journey/retention report

#Stage 2 — Reconciliation and operations

• Add CDC/batch reconciliation for sessions, bookings, and payments
• Add invalid-event quarantine and data-quality reporting
• Add identity-link events
• Add client timeline and at-risk workflow inputs

#Stage 3 — Behavioral detail

• Introduce browser SDK
• Add consent and anonymous session handling — policy locked in BROWSER-CONSENT-POLICY.md (T1.14); code is T2.13
• Instrument selected P1 funnel events
• Add navigation-safe delivery and developer inspector

#Stage 4 — Scale only when measured

• Evaluate ClickHouse from real event volume and query latency
• Add durable streaming infrastructure only if buffering and throughput requirements justify it
• Add additional SDK languages only for confirmed integrations

#16. References

• Segment data governance
• RudderStack Event Stream
• Snowplow tracker delivery
• Amplitude Browser SDK
• OpenTelemetry Collector
• OpenTelemetry protocol delivery semantics
• ClickHouse real-time analytics architecture
• Collector ingest contract tests (T1.13 / T2.2)
• ADR 0002 — collector, SDK, outbox
• Browser consent & minimization (T1.14)
• Data quality bar (T1.10)