Normalization Without Losing Source Fidelity

Common field names do not guarantee common meaning

One provider uses user for the authenticated actor. Another uses it for the account being modified. Mapping both values into user.name produces syntactic consistency and analytical confusion.

Normalization transforms source-specific records into a shared representation. Its purpose is to make common reasoning easier, not to declare different observations identical. Begin by describing the source event: producer, trigger, actor, target, operation, outcome, and time semantics.

Only then choose common concepts. If the shared schema cannot preserve a distinction, retain it in source fields or an extension. A normalized event should be a view of the evidence, not a replacement for what the producer actually said.

Map relationships, not isolated labels

An event can contain an initiating user, application, delegated subject, target account, resource owner, and reviewer. Mapping values one field at a time can put the right strings into the wrong roles.

Define typed objects and relationships: actor performed action on target using session under authority. Map operation and outcome as a pair; a requested delete, denied delete, and completed delete should not become the same activity.

Use representative source examples and counterexamples. If two provider operations map to one common value, document which downstream claims remain safe and which require the source distinction. The mapping is a security interpretation and deserves review like analytic logic.

Version and trace every transformation

Record source schema version, parser version, normalization mapping, enrichment versions, transformation time, and destination schema. Preserve source identifiers and raw or protected source values required for verification.

A field can change without its name changing. Enum values can gain new meanings. A parser can begin defaulting absent outcomes to success. Historical events reprocessed with a new mapping can differ from records interpreted when they first arrived.

The security telemetry data contracts should state compatibility and change rules. When a mapping changes, identify affected detections, replay representative cases, and decide whether historical data will be migrated or remain versioned.

Null, unknown, absent, and not applicable are different

A missing actor can mean the producer did not supply one, collection dropped it, parsing failed, the action was system-initiated, or the field does not apply. Replacing every case with unknown erases failure evidence.

Preserve presence and reason where possible. Avoid default values that satisfy detection conditions. An absent outcome should not become success. An unrecognized operation should not be forced into the closest known category without a visible mapping warning.

Monitor mapping coverage, unknown enum rates, parse failures, field completeness by source and operation, and changes in value distribution. Aggregate success can hide one critical operation falling into an unmapped bucket.

Validate semantic parity, not identical output counts

A source-specific analytic and a normalized analytic may produce different counts because deduplication, identity roles, outcomes, or time changed. Compare representative events and the security claim they support, not only totals.

Test intended behavior, benign alternatives, missing fields, new enum values, one-to-many transformations, and rollback to source. Ask an analyst to reconstruct why the normalized event supports the result and verify it against the producer record.

Normalization succeeds when it lowers the cost of shared reasoning while keeping disagreement inspectable. If a common schema makes every provider look alike but prevents the consumer from checking meaning, it has simplified the data by weakening the evidence.