Next.js selected observability

Only two logging variables are needed in Vercel (Production):

• OBSERVABILITY_GATEWAY_URL=https://data-api.seekalgo.com/observability/v1/events
• OBSERVABILITY_PRODUCER_TOKEN: the Next app credential, matching OBSERVABILITY_NEXT_APP_TOKEN in Doppler.

Server and browser logging enable automatically when both are present. Redeploy after configuring them so the server-rendered browser activation metadata updates. Release IDs come from Vercel's commit SHA. Service identity is fixed to next-app. No extra enable switch, actor key, public environment variable or manual release setting is needed. Actor pseudonyms use a domain-separated HMAC with the producer credential; rotating that credential also rotates actor pseudonyms.

The collector has two producer credentials: one for this app and one shared by all downloader-owned services. Worker names still identify the failing service. Only the collector owns Axiom's separate ingest credential. Existing OBSERVABILITY_ENABLED, OBSERVABILITY_SERVICE, OBSERVABILITY_RELEASE, OBSERVABILITY_ACTOR_KEY, NEXT_PUBLIC_OBSERVABILITY_ENABLED and NEXT_PUBLIC_OBSERVABILITY_RELEASE settings may be removed; code no longer reads them. This change does not provision account credentials.

Delivery and noise policy

API routes register Next after before auth or work starts. Builder NDJSON routes hold completion until the terminal work, session save and stream close settle. HTTP 200 error streams and MCP isError results produce semantic failure events. Logging never uses the app DB or changes the existing product response or transaction outcome. Unexpected exceptions translated to 400/401 remain visible.

Builder messages, backtest accepted/save transitions and paper-launch outcomes are retained. Failed stages preserve a shared failure ID; successful repair records recovery. Routine successful operations use deterministic 1% event sampling plus unsampled aggregate counter deltas. There is no spend ceiling, usage-based cutoff or changing sample rate.

The first trusted engine hash and subsequent hash changes are retained per process; unchanged compatibility checks use routine sampling and counters. Short builder engine completions with a run ID remain retained for later save correlation.

One bounded producer POST is attempted after each request completes, with a 2-second timeout and no producer retry loop. A collector 202 is durable collector acceptance, not Axiom acceptance. The gateway owns central retries, deduplication and summary windows. Request buffers are bounded at 128 events/256KiB, with collector batches bounded at 50 records/64KiB. Hard request timeouts, process crashes, a browser close or failure before collector acceptance can lose events. Counter totals remain best effort.

When a request overfills its buffers, final request/lifecycle outcomes take priority over routine records and the retained terminal event reports dropped. Collector packing also reports records omitted by its per-batch resource limit. These limits do not grow into a global or daily allowance.

Browser relay

POST /api/observability/browser accepts at most 5 flattened events and 10KiB read from the actual request stream. It requires the exact same origin, rejects unknown fields, arbitrary strings, invalid timestamps/IDs/enums, and any supplied trust/service/actor/alert authority. The exact browser-relay POST bypasses general session middleware, so fatal/auth-dependency reports remain reachable during sign-in failures. Other API paths and methods keep their existing session protection. The relay performs its own Supabase cookie auth independently of the app Postgres pool. The relay derives actor metadata and always marks browser assertions client_unverified, with separate client and relay releases.

Anonymous intake permits only explicit app fatal and auth dependency failure categories. Per-process source controls allow bursts up to 120 intake calls per minute, keep at most 2,048 active source entries and expire entries after one minute. Temporary IP metadata is never exported. These are abuse/transport controls, not daily logging allowances; production edge protection and trusted forwarding-header configuration should be verified before activation. Pending duplicate browser failures are coalesced with occurrence counts, and recoveries retain a separate event. The gateway coalesces subsequent repeated failures across calls.

Browser parent execution observes one terminal engine outcome and separately reports result/status save failures. No new networking or logging is added inside the isolated sandbox, and its CSP remains unchanged. Direct browser Parquet requests keep existing CORS/cache behavior and do not send the new correlation headers; their dataset/time association remains approximate until the separate CORS/range rollout is verified. Owned server Data API and MCP bridge calls carry fresh downstream IDs, parent IDs and operation IDs.

Local verification and activation

Run npm run test:observability, the existing applicable test scripts, npx tsc --noEmit, and touched-file ESLint. Build with npm run build:workers followed by npx next build --webpack; the normal npm run build includes deployment migrations and must not be used for this local observability check. CI runs the observability behavior fixtures without a live DB or provider.

The implementation's macOS verification environment stalled in the unchanged Monaco esbuild bundler and the Next webpack compiler at zero CPU. Temporary build configuration was restored. The full worker and Next webpack production build subsequently passed in GitHub CI against implementation commit 66ac14c2794f297f48f141994ce7a91dc4cc8352, without the deployment-migration command. All 28 observability tests, typecheck, migrations and Vercel preview checks passed. A real browser workflow with logging enabled remains an activation check.

The webpack boundary fixture compiles the actual instrumentation entry using Next's bundled webpack and its runtime substitution. It verifies Edge/browser graphs exclude the Node producer. Node-only imports remain inside positive NEXT_RUNTIME === "nodejs" branches; an early return alone does not remove later imports from webpack's dependency graph.

Before enabling production, verify the gateway receipt/readback, actual release IDs, token scope, account entitlement, resource limits and the shared downloader runbook/monitors. Verify one real builder stream, backtest/save and closed-bar paper workflow. Measure event volume/drop counts, storage and query compute separately. Local fixture tests do not prove production Axiom acceptance or coverage completeness. Keep native bounded platform logs as fallback.

The event schema creates fields from allowlists before serialization. It exports no prompts, tool arguments/results, source, raw errors, stack traces, SQL, URLs/query strings, environment dumps, credentials, financial/account data, candle rows or script console output. PostHog remains separate.