go-otel/instrumentation.md

Instrumentation guide

This document describes the recommended instrumentation contract for this receiver. It is available at /instrumentation.md so frontend and backend agents can read it before integrating telemetry.

Shared identity

Use the same values across all signals from one deployed application:

• service.name: stable application name, for example checkout-web.
• service.release: immutable deploy identifier, for example 2026.09.18-a1b2c3.
• service.environment: production, staging, or development.

The release must exactly match the source-map upload release.

Browser frontend

Load the RUM script and initialize it once, before the application starts:

<script src="https://otel.example.com/static/otel-rum.js"></script>
<script>
  const rum = OtelRUM.init({
    serviceName: "checkout-web",
    release: "2026.09.18-a1b2c3",
    environment: "production",
    baseUrl: "https://otel.example.com"
  });
</script>

To avoid parser-blocking downloads, use a deferred script with declarative initialization instead of the two scripts above:

<script defer src="https://otel.example.com/static/otel-rum.js"
  data-service-name="checkout-web"
  data-release="2026.09.18-a1b2c3"
  data-environment="production"
  data-base-url="https://otel.example.com"></script>

Declarative initialization runs at DOMContentLoaded. Use the earlier, explicit initialization when capturing errors and requests during application startup matters more than avoiding parser blocking. Do not initialize twice.

The collector captures uncaught errors, unhandled promise rejections, failed HTTP requests, breadcrumbs, Web Vitals, and optional trace context. Correlate frontend requests with backend spans when possible:

fetch("/api/orders", { headers: rum.traceHeaders() });

For manually handled errors:

try {
  riskyOperation();
} catch (error) {
  rum.captureException(error, { "feature.name": "checkout" });
}

Do not put secrets, tokens, or sensitive customer data in breadcrumbs or custom attributes.

Cookieless diagnostics and consent

Neither RUM nor Todo reads or writes cookies, sessionStorage, or localStorage. Configuration and consent are held only in memory. Integrate with the application's existing consent manager rather than adding a tracking cookie or another banner:

const rum = OtelRUM.init({ serviceName: "checkout-web" });
rum.setConsent("granted");
rum.setConsent("denied");

For a previously accepted choice, supply consent: "granted" at initialization or data-consent="granted" on the script. Omitted consent means "denied"; explicit configuration overrides the attribute. rum.getConsent() returns the current choice. The application is responsible for persisting that choice and applying it on the next page load.

The SDK separates error diagnostics from optional performance collection:

Policy	Error payload	Performance collection
Default counts	Generic event, severity, recognised error type, HTTP status, release/environment	Off without consent
Configured minimal	Counts plus application-sanitized diagnostic fields	Still off without consent
consented	Detailed errors, stacks, context, breadcrumbs and page metadata	On unless disabled
Explicit, host-allowlisted development	Detailed diagnostics for an already-approved development site	On unless disabled

rum.getPrivacyMode() reports the configured active policy. An individual minimal event falls back to counts if its sanitizer does not approve it. Counts/minimal payloads exclude automatic URLs, user agents, viewport/connection details, session ids, DOM selectors, breadcrumbs, context and custom attributes. Per-request trace-header instrumentation still runs without persistent visitor identity, but counts/minimal log records do not include trace ids. Neither policy is a guarantee of anonymity or a consent exemption.

Minimal actual-error diagnostics

Only configure this policy after separately assessing the diagnostic purpose, ePrivacy position and GDPR legal basis; it is not automatically enabled when tracking consent is rejected. Provide an application-owned sanitizer that maps known errors to useful, reviewed diagnostics:

const safeDiagnostics = new Map([
  ["checkout.price-missing", {
    message: "Checkout price is missing",
    code: "CHECKOUT_PRICE_MISSING"
  }]
]);

const rum = OtelRUM.init({
  serviceName: "checkout-web",
  minimalDiagnostics: {
    route: "/checkout/:step",
    sanitize: event => safeDiagnostics.get(event.message) || null
  }
});

rum.captureMessage("checkout.price-missing");

The sanitizer receives bounded message, optional type and stacktrace. It may return {message, code?, stacktrace?}; only those fields and the configured route template are added. Return null for unknown/free-text events. Missing, throwing or invalid sanitizers fall back to counts without sending the original message or stack. Output lengths are bounded, but the SDK trusts sanitizer output: it does not automatically scrub returned text. Only return reviewed messages and necessary scrubbed stack frames; never forward event unchanged. Exclude credentials, query strings, fragments, user-specific paths, user content and sensitive data. The route must be an application-controlled template, not location.pathname.

On withdrawal, detailed queued records are dropped and an independently configured minimal policy continues. Without that policy, counts continue. Do not use this to relabel consent-required processing after refusal. Even counts require a deployment-specific assessment. See docs/EU-RUM-PRIVACY.md.

Development sites with existing access and acceptance

Development mode is explicit and requires an exact hostname allowlist:

const rum = OtelRUM.init({
  serviceName: "checkout-web",
  environment: "development",
  devMode: accessState.loggedIn && accessState.diagnosticsAccepted,
  devModeHosts: ["localhost", "checkout.dev.example.com"]
});

Here accessState is provided by the host application's existing login and acceptance flow. Do not initialize development mode before that flow completes. Login alone does not establish consent or another legal basis. The SDK cannot verify the login, notice, acceptance or purpose; the application must do so. The allowlist is an accidental-production safeguard, not an authentication boundary. Wildcards/subdomains do not match automatically; an environment label never enables this mode. Ports are not part of the hostname match.

For declarative setup after the access gate, use data-dev-mode="true" and data-dev-mode-hosts="localhost,checkout.dev.example.com". Explicit devMode: false overrides the attribute. Manual captureMessage(message, "INFO"), exceptions, context and breadcrumbs are available for development improvement logs. Do not include passwords, tokens or sensitive user content.

Development mode does not manufacture a consent grant: getConsent() still returns the actual configured choice. Call rum.setConsent("denied") on logout or withdrawal; it disables development mode even if consent was already denied, discards queued data and returns to minimal/counts. Re-enabling requires a new page initialization after the access gate. Todo remains consent-controlled separately, and is still disabled by default.

Performance permission and withdrawal

Web Vitals is not loaded until consent is granted or allowed development mode is active. captureWebVitals: false disables it in all modes. If a separate assessment establishes permission for cookieless performance collection, set captureWebVitalsWithoutConsent: true or data-vitals-without-consent="true". Explicit false overrides the attribute; this option is not a legal exemption. In that mode only values, deltas and numeric attribution are sent without consent. A random, in-memory page-view id deduplicates snapshots within one load; metric ids, selectors and other text attribution are omitted.

Detailed diagnostics use an in-memory-only session id, renewed on reload or re-grant. Every consent change discards queued records, breadcrumbs, context and pending vitals; data already sent cannot be recalled. Reapply identifying context only after permission. Withdrawal suppresses later vitals callbacks and reports unless cookieless performance was independently enabled. Observers already installed inside the vendored Web Vitals library cannot be unloaded by this SDK; reload the page after withdrawal if complete observer teardown is required. If consent is revoked before its download completes, no metric subscriptions are started.

Telemetry uses fetch with credentials: "omit", referrerPolicy: "no-referrer" and keepalive: true in every mode; no cookie-bearing sendBeacon fallback is used. If fetch is unavailable or fails, telemetry is dropped. Browser-managed network headers and client IP addresses remain visible to the receiving infrastructure, so this is client-side data minimization, not a guarantee of legal anonymization or a determination that consent is unnecessary. Review server logging, retention, enrichment and applicable requirements separately.

Building the browser scripts

The maintained TypeScript sources are rum/src/otel-rum.ts and rum/src/otel-todo.ts; public options and APIs are defined in rum/src/types.ts. Do not edit the generated static/otel-rum.js or static/otel-todo.js directly.

Run make build-rum to install locked dependencies, typecheck with strict TypeScript, and generate separate minified IIFE bundles at the existing /static/otel-rum.js and /static/otel-todo.js URLs. Todo stays separate so it is never downloaded unless enabled. Run make test-rum for the same build plus regression tests against the generated JavaScript. After installing dependencies, npm --prefix rum test also runs the build and tests.

make build builds the browser scripts and Go server. Docker builds and tests the browser scripts in an isolated Node stage automatically, so build.sh also uses fresh bundles. Generated JavaScript remains checked in for direct go run ./cmd/server development; rebuild it whenever TypeScript changes.

External source maps are generated locally but are not downloaded by browsers automatically. Docker omits them by default; PUBLIC_SOURCEMAPS=true retains them, as for the UI. TypeScript and esbuild are build-time dependencies only.

React error boundaries

Rendering errors are not reliably caught by global window.onerror handlers. Report them from an error boundary and render a recovery screen:

import React from "react";

export class AppErrorBoundary extends React.Component<
  { children: React.ReactNode },
  { error: Error | null }
> {
  state = { error: null as Error | null };

  static getDerivedStateFromError(error: Error) {
    return { error };
  }

  componentDidCatch(error: Error, info: React.ErrorInfo) {
    window.OtelRUM?.last?.captureException(error, {
      "exception.type": "ReactErrorBoundary",
      "react.component_stack": info.componentStack,
    });
  }

  render() {
    if (this.state.error) {
      return <button onClick={() => this.setState({ error: null })}>Retry</button>;
    }
    return this.props.children;
  }
}

Use the same pattern in your application, including a user-visible fallback for individual views and the application root.

Todo widget

otel-todo.js adds a small floating symbol and a keyboard shortcut (default Alt+Shift+T) that opens a corner dialog for filing a todo/issue from the page, with an optional component picker. User-submitted text, selected components and explicitly picked element references are sent as requested; URLs, user agents and session metadata are attached only with consent. A note can itself contain personal information, so it is not guaranteed anonymous.

Todo is disabled by default in otel-rum.js. Opt in with OtelRUM.init({ enableTodo: true }) or data-enable-todo="true" on the RUM script. It reuses the service name, base URL and ingest key, and loads from the same directory as the RUM script.

When enabled, both loading and initialization wait until after window.load, then run during requestIdleCallback with no forced timeout. If idle callbacks are unavailable, loading waits three seconds after load. On a continuously busy page, idle loading may be delayed indefinitely. Error capture and request instrumentation do not wait for Todo.

Explicit enableTodo: false overrides the opt-in attribute. The legacy excludeTodo: true and data-exclude-todo="true" remain supported and veto opt-in. Passing excludeTodo: false no longer enables Todo.

Alternatively, load it standalone (this bypasses RUM's idle scheduling):

<script src="https://otel.example.com/static/otel-todo.js"
        data-service-name="checkout-web"
        data-ingest-key="..."
        data-components="Header,Cart,Payment"></script>

Components are also discovered from [data-component="..."] elements. Other options (on the RUM or todo script, or OtelRUM.init({ todo: {...} })): data-shortcut, data-hide-button="true", data-base-url.

When loaded by RUM, Todo reads the current RUM consent at submission time, including after withdrawal. Standalone Todo defaults to denied consent; use OtelTodo.instance.setConsent("granted" | "denied") from the consent manager, or initialize with consent / data-consent. Submissions always omit browser credentials and referrers, and no browser storage is used.

• POST /v1/todos (ingest key, CORS) body {project, text, component, path, hash, url, ua, session}
• GET /api/todos?project=<service>&limit=100 (authenticated; omit project for all), newest first

Todos go to ClickHouse when configured, else to the JSONL file at OTEL_TODOS_PATH (default otel-todos.jsonl).

CSP violation reports

Browsers report Content-Security-Policy violations straight to the collector. Point the policy's report-uri (or a Reporting API report-to endpoint) at /v1/csp; both the legacy application/csp-report object and the Reporting API's application/reports+json batch are accepted:

policy: default-src 'self'; report-uri https://otel.example.com/v1/csp?key=...

Delivery is allowed cross-origin from *.k6n.net, *.tornberg.me and *.knatofs.se (the collector answers CORS preflights and echoes the origin for those domains only). Like the vitals beacon, the optional ingest key travels as ?key= on the reporting URL, since browsers cannot attach headers to violation reports. Storage follows the configured store: a dedicated typed ClickHouse table (csp_reports, 90-day TTL), else the JSONL file at OTEL_CSP_PATH (default otel-csp-reports.jsonl).

• POST /v1/csp (ingest key as ?key=, CORS) — browser-reported, send nothing by hand
• GET /api/csp/reports?site=&directive=&q=&hours=&limit= (authenticated), newest first
• GET /api/csp/summary?hours=24 (authenticated) — totals plus top directives, blocked hosts and sites

Backend instrumentation

Instrument backend services with an OpenTelemetry SDK or auto-instrumentation and export OTLP to this receiver:

• OTLP/gRPC: otel.example.com:4317
• OTLP/HTTP traces: https://otel.example.com/v1/traces
• OTLP/HTTP metrics: https://otel.example.com/v1/metrics
• OTLP/HTTP logs: https://otel.example.com/v1/logs

Set the resource attributes on every provider/exporter:

service.name=checkout-api
service.release=2026.09.18-a1b2c3
service.environment=production

Create spans around inbound requests, database calls, external HTTP calls, queue operations, and important business operations. Record HTTP status, route, method, and exception details. Mark failed spans with an error status. The receiver derives backend issues from error spans and ERROR/FATAL logs.

If HTTP ingest authentication is enabled, provide OTEL_INGEST_KEY as the X-Otel-Ingest-Key header or key query parameter. Keep that key server-side; browser clients should not receive it.

For non-interactive agents that need to query protected debug endpoints, use a separate OTEL_AGENT_TOKEN service token. Send it as X-Otel-Agent-Token: $OTEL_AGENT_TOKEN; do not put it in browser code or telemetry payloads.

Source maps

Build the frontend with the same immutable release used by RUM. The command and output directory depend on your build tool; the example below assumes a standard dist/ output:

export RELEASE="2026.09.18-a1b2c3"
VITE_APP_RELEASE="$RELEASE" npm run build

After deployment, upload the contents of each generated JavaScript map to the public multipart endpoint. No repository checkout or upload script is needed. source must match the generated bundle URL/path captured in the browser stack, while file=@... reads the local source-map file contents:

curl --fail-with-body --silent --show-error \
  -H "X-Sourcemap-Key: $OTEL_SOURCEMAP_KEY" \
  -X POST "https://otel.example.com/api/sourcemaps" \
  -F "release=$RELEASE" \
  -F "source=/assets/app-HASH.js" \
  -F "file=@dist/assets/app-HASH.js.map;type=application/json"

For example, if the deployed JavaScript is available at https://app.example.com/assets/main-abc123.js, upload its matching map with source=/assets/main-abc123.js. The source-map filename on the build machine does not need to match the URL, but the source field must.

Maps are parsed, written to OTEL_SOURCEMAP_DIR, and reloaded on receiver start. Issue responses include a resolved stack when service.release and the generated bundle path match. Uploaded maps are never returned, but when a map embeds sourcesContent the RUM "JS errors" view shows a few lines of original source around each mapped frame to signed-in users.

Requirements checklist

A stack only resolves when every one of these holds. When it does not, check them in this order:

1. The build emits maps, with sources embedded. Enable source maps in the bundler (Vite build.sourcemap: true, webpack devtool: "source-map"). Keep sourcesContent (the default) or the code view has nothing to show.
2. The browser reports a release. Pass the release to OtelRUM.init({ release }) (or data-release on the script tag), injected at build time. An event with no release, or dev, can never match an upload.
3. The uploaded release is byte-for-byte the same string. Use one value for both the bundle build and the upload (a git short SHA works).
4. source matches the path in the browser stack. Everything before /assets/ and any ?query is ignored, so source=/assets/main-abc123.js matches https://app.example.com/assets/main-abc123.js. Content-hashed names must match exactly, and chunks each need their own upload.
5. The map belongs to the deployed bundle. Upload the maps from the very build that produced the deployed files. A second local rebuild can differ (dependencies, Node version) and silently mismatch. In Docker, copy the maps out of the build stage (docker build --target <stage>, then docker cp) instead of rebuilding. build.sh does this.
6. Every chunk is uploaded. Loop over all *.js.map files and fail the deploy when none are found, since an empty loop otherwise succeeds silently.
7. The receiver keeps its maps. OTEL_SOURCEMAP_DIR must be on persistent storage (a PVC in Kubernetes, not emptyDir). Maps written to an ephemeral directory vanish on restart.
8. Maps are not published with the site. Delete *.map from the served image unless public maps are intended (see below).

Verifying a deploy

List what the receiver has loaded (requires a signed-in session or token):

curl -H "Authorization: Bearer $TOKEN" https://otel.example.com/api/sourcemaps
# [{"release":"2026.09.18-a1b2c3","source":"/assets/main-abc123.js"}, ...]

Then trigger a real uncaught error from application code, for example a button that runs setTimeout(() => { throw new Error("source-map check") }) (this repository's RUM "JS errors" view has a Generate test error button), and confirm the card shows Source-mapped with the original file and line.

Frame status in the stack explorer explains a miss: no-release (event has no release), no-map (nothing uploaded for that release and file), or unmapped (a map exists but has no mapping for that position).

The Docker build removes .map files from production images by default. For development or staging, public maps can be enabled for Chrome DevTools:

PUBLIC_SOURCEMAPS=true ./build.sh

Public maps expose application source code and should not be enabled in production unless that is intentional.

Verification checklist

1. Trigger a controlled frontend exception.
2. Trigger a controlled backend error span or ERROR log.
3. Confirm both appear under Issues as separate frontend/backend issues.
4. Confirm the issue sample contains the expected release.
5. Confirm a mapped frontend stack shows original file and line information (see Requirements checklist under Source maps if it does not).
6. Confirm trace IDs connect frontend requests to backend traces where headers were propagated.