Security Telemetry Data Contracts

A stable field name can hide a broken promise

An identity provider emits an actor field for months. A service update keeps the field but changes it from the initiating human to the application that submitted the request. The parser succeeds, dashboards remain green, and detections begin joining actions to the wrong person.

A data contract records the promise between a telemetry producer and its consumers. It states what an event means, when it is produced, which populations it represents, how identities and times are expressed, which fields are required or conditional, and how change and failure are communicated.

The contract exists because structure alone is insufficient. A valid string can contain the wrong identity. A present timestamp can use the wrong clock. A complete event can represent an attempt when the detection assumes a successful outcome. Meaning is part of the interface.

Describe the observation before describing the fields

Begin with the source and observation point. Which component creates the record? Which action or state causes it? Does the producer see a request, an authorization decision, a completed outcome, or a later summary? Which identities does it know directly?

Then describe fields in that context. An actor should specify principal type, namespace, tenant, and whether delegation or impersonation is preserved. Time should specify event, observation, or processing semantics and expected precision. Outcome should distinguish success, failure, unknown, and partial completion.

Examples belong in the contract because they reveal ambiguity faster than type declarations. Include an ordinary event, a privileged event, a delegated action, a failure, and a record with conditional fields absent. The examples explain how values relate; they are not substitutes for the written semantics.

Make population, delivery, and quality measurable

A feed can be fresh overall while one tenant or event class has disappeared. Define expected producers and populations at a level where loss can be seen. Record delivery mode, buffering, retry behavior, duplicate expectations, ordering, typical and maximum latency, and the handling of late records.

Quality measures should follow detection needs. Required-field completeness, value distributions, parser errors, producer coverage, duplicate rates, and event-to-ingest delay are more informative than total volume alone. A field that is 99 percent complete may still be absent for the one privileged operation an analytic needs.

The telemetry quality and blind spots review should connect each measure to affected claims. When a condition fails, consumers need to know which detections are degraded, not merely that a pipeline metric changed.

Treat transformations as contracts too

Collectors, parsers, normalizers, enrichers, and storage layers each create a new boundary. A parser promises how source values become fields. A normalizer promises how provider-specific operations map to common concepts. An enrichment service promises which reference time and ownership source it uses.

Version those mappings and preserve lineage. If a common action collapses grant, activation, and use into one privilege-change value, downstream consumers need either source fields or a reversible path to recover the distinction. If current asset ownership is added to historical events, label it as current enrichment rather than event-time fact.

Contract compatibility is semantic, not only syntactic. Adding a field may be structurally safe while changing which records populate an existing one. Test consuming analytics against representative old and new records before promotion.

Give change and failure an owner

Name the producer owner, pipeline owner, security consumers, and escalation path. Define how proposed changes are announced, which consumers must review them, how long parallel versions exist, and what evidence permits migration. Emergency changes should leave the same record even when review occurs afterward.

Retention, access, privacy, and cost also belong in the agreement. A detection that requires ninety days of event-time history cannot rely on a thirty-day searchable tier without stating the loss. Sensitive payload fields need purpose, access control, and deletion rules.

The contract succeeds when a consumer can answer three questions: what does this record establish, how do I know the promise is healthy now, and who acts when it is not? That shared memory prevents quiet source changes from becoming quiet detection failures.