go-otel/instrumentation.md

Instrumentation guide

This document describes the recommended instrumentation contract for this receiver. It is available at /instrumentation.md so frontend and backend agents can read it before integrating telemetry.

Shared identity

Use the same values across all signals from one deployed application:

• service.name: stable application name, for example checkout-web.
• service.release: immutable deploy identifier, for example 2026.09.18-a1b2c3.
• service.environment: production, staging, or development.

The release must exactly match the source-map upload release.

Browser frontend

Load the RUM script and initialize it once, before the application starts:

<script src="https://otel.example.com/static/otel-rum.js"></script>
<script>
  const rum = OtelRUM.init({
    serviceName: "checkout-web",
    release: "2026.09.18-a1b2c3",
    environment: "production",
    baseUrl: "https://otel.example.com"
  });
</script>

The collector captures uncaught errors, unhandled promise rejections, failed HTTP requests, breadcrumbs, Web Vitals, and optional trace context. Correlate frontend requests with backend spans when possible:

fetch("/api/orders", { headers: rum.traceHeaders() });

For manually handled errors:

try {
  riskyOperation();
} catch (error) {
  rum.captureException(error, { "feature.name": "checkout" });
}

Do not put secrets, tokens, or sensitive customer data in breadcrumbs or custom attributes.

React error boundaries

Rendering errors are not reliably caught by global window.onerror handlers. Report them from an error boundary and render a recovery screen:

import React from "react";

export class AppErrorBoundary extends React.Component<
  { children: React.ReactNode },
  { error: Error | null }
> {
  state = { error: null as Error | null };

  static getDerivedStateFromError(error: Error) {
    return { error };
  }

  componentDidCatch(error: Error, info: React.ErrorInfo) {
    window.OtelRUM?.last?.captureException(error, {
      "exception.type": "ReactErrorBoundary",
      "react.component_stack": info.componentStack,
    });
  }

  render() {
    if (this.state.error) {
      return <button onClick={() => this.setState({ error: null })}>Retry</button>;
    }
    return this.props.children;
  }
}

Use the same pattern in your application, including a user-visible fallback for individual views and the application root.

Todo widget

otel-todo.js adds a small floating symbol and a keyboard shortcut (default Alt+Shift+T) that opens a corner dialog for filing a todo/issue from the page, with an optional component picker. The current path and hash are posted with it.

otel-rum.js loads it automatically (from the same directory as the RUM script) and reuses its service name, base URL and ingest key. Opt out with OtelRUM.init({ excludeTodo: true }) or data-exclude-todo="true". Or load it standalone:

<script src="https://otel.example.com/static/otel-todo.js"
        data-service-name="checkout-web"
        data-ingest-key="..."
        data-components="Header,Cart,Payment"></script>

Components are also discovered from [data-component="..."] elements. Other options (on the RUM or todo script, or OtelRUM.init({ todo: {...} })): data-shortcut, data-hide-button="true", data-base-url.

• POST /v1/todos (ingest key, CORS) body {project, text, component, path, hash, url, ua, session}
• GET /api/todos?project=<service>&limit=100 (authenticated; omit project for all), newest first

Todos go to ClickHouse when configured, else to the JSONL file at OTEL_TODOS_PATH (default otel-todos.jsonl).

CSP violation reports

Browsers report Content-Security-Policy violations straight to the collector. Point the policy's report-uri (or a Reporting API report-to endpoint) at /v1/csp; both the legacy application/csp-report object and the Reporting API's application/reports+json batch are accepted:

policy: default-src 'self'; report-uri https://otel.example.com/v1/csp?key=...

Delivery is allowed cross-origin from *.k6n.net, *.tornberg.me and *.knatofs.se (the collector answers CORS preflights and echoes the origin for those domains only). Like the vitals beacon, the optional ingest key travels as ?key= on the reporting URL, since browsers cannot attach headers to violation reports. Storage follows the configured store: a dedicated typed ClickHouse table (csp_reports, 90-day TTL), else the JSONL file at OTEL_CSP_PATH (default otel-csp-reports.jsonl).

• POST /v1/csp (ingest key as ?key=, CORS) — browser-reported, send nothing by hand
• GET /api/csp/reports?site=&directive=&q=&hours=&limit= (authenticated), newest first
• GET /api/csp/summary?hours=24 (authenticated) — totals plus top directives, blocked hosts and sites

Backend instrumentation

Instrument backend services with an OpenTelemetry SDK or auto-instrumentation and export OTLP to this receiver:

• OTLP/gRPC: otel.example.com:4317
• OTLP/HTTP traces: https://otel.example.com/v1/traces
• OTLP/HTTP metrics: https://otel.example.com/v1/metrics
• OTLP/HTTP logs: https://otel.example.com/v1/logs

Set the resource attributes on every provider/exporter:

service.name=checkout-api
service.release=2026.09.18-a1b2c3
service.environment=production

Create spans around inbound requests, database calls, external HTTP calls, queue operations, and important business operations. Record HTTP status, route, method, and exception details. Mark failed spans with an error status. The receiver derives backend issues from error spans and ERROR/FATAL logs.

If HTTP ingest authentication is enabled, provide OTEL_INGEST_KEY as the X-Otel-Ingest-Key header or key query parameter. Keep that key server-side; browser clients should not receive it.

For non-interactive agents that need to query protected debug endpoints, use a separate OTEL_AGENT_TOKEN service token. Send it as X-Otel-Agent-Token: $OTEL_AGENT_TOKEN; do not put it in browser code or telemetry payloads.

Source maps

Build the frontend with the same immutable release used by RUM. The command and output directory depend on your build tool; the example below assumes a standard dist/ output:

export RELEASE="2026.09.18-a1b2c3"
VITE_APP_RELEASE="$RELEASE" npm run build

After deployment, upload the contents of each generated JavaScript map to the public multipart endpoint. No repository checkout or upload script is needed. source must match the generated bundle URL/path captured in the browser stack, while file=@... reads the local source-map file contents:

curl --fail-with-body --silent --show-error \
  -H "X-Sourcemap-Key: $OTEL_SOURCEMAP_KEY" \
  -X POST "https://otel.example.com/api/sourcemaps" \
  -F "release=$RELEASE" \
  -F "source=/assets/app-HASH.js" \
  -F "file=@dist/assets/app-HASH.js.map;type=application/json"

For example, if the deployed JavaScript is available at https://app.example.com/assets/main-abc123.js, upload its matching map with source=/assets/main-abc123.js. The source-map filename on the build machine does not need to match the URL, but the source field must.

Maps are parsed, written to OTEL_SOURCEMAP_DIR, and reloaded on receiver start. Issue responses include a resolved stack when service.release and the generated bundle path match. Uploaded maps are never returned, but when a map embeds sourcesContent the RUM "JS errors" view shows a few lines of original source around each mapped frame to signed-in users.

Requirements checklist

A stack only resolves when every one of these holds. When it does not, check them in this order:

1. The build emits maps, with sources embedded. Enable source maps in the bundler (Vite build.sourcemap: true, webpack devtool: "source-map"). Keep sourcesContent (the default) or the code view has nothing to show.
2. The browser reports a release. Pass the release to OtelRUM.init({ release }) (or data-release on the script tag), injected at build time. An event with no release, or dev, can never match an upload.
3. The uploaded release is byte-for-byte the same string. Use one value for both the bundle build and the upload (a git short SHA works).
4. source matches the path in the browser stack. Everything before /assets/ and any ?query is ignored, so source=/assets/main-abc123.js matches https://app.example.com/assets/main-abc123.js. Content-hashed names must match exactly, and chunks each need their own upload.
5. The map belongs to the deployed bundle. Upload the maps from the very build that produced the deployed files. A second local rebuild can differ (dependencies, Node version) and silently mismatch. In Docker, copy the maps out of the build stage (docker build --target <stage>, then docker cp) instead of rebuilding. build.sh does this.
6. Every chunk is uploaded. Loop over all *.js.map files and fail the deploy when none are found, since an empty loop otherwise succeeds silently.
7. The receiver keeps its maps. OTEL_SOURCEMAP_DIR must be on persistent storage (a PVC in Kubernetes, not emptyDir). Maps written to an ephemeral directory vanish on restart.
8. Maps are not published with the site. Delete *.map from the served image unless public maps are intended (see below).

Verifying a deploy

List what the receiver has loaded (requires a signed-in session or token):

curl -H "Authorization: Bearer $TOKEN" https://otel.example.com/api/sourcemaps
# [{"release":"2026.09.18-a1b2c3","source":"/assets/main-abc123.js"}, ...]

Then trigger a real uncaught error from application code, for example a button that runs setTimeout(() => { throw new Error("source-map check") }) (this repository's RUM "JS errors" view has a Generate test error button), and confirm the card shows Source-mapped with the original file and line.

Frame status in the stack explorer explains a miss: no-release (event has no release), no-map (nothing uploaded for that release and file), or unmapped (a map exists but has no mapping for that position).

The Docker build removes .map files from production images by default. For development or staging, public maps can be enabled for Chrome DevTools:

PUBLIC_SOURCEMAPS=true ./build.sh

Public maps expose application source code and should not be enabled in production unless that is intentional.

Verification checklist

1. Trigger a controlled frontend exception.
2. Trigger a controlled backend error span or ERROR log.
3. Confirm both appear under Issues as separate frontend/backend issues.
4. Confirm the issue sample contains the expected release.
5. Confirm a mapped frontend stack shows original file and line information (see Requirements checklist under Source maps if it does not).
6. Confirm trace IDs connect frontend requests to backend traces where headers were propagated.