← Detection Engineering: From Telemetry to Tuned Detections
Foundations of Detection Engineering
What Is Detection Engineering? The Detection Engineering Lifecycle
Telemetry and Data Sources
Understanding Telemetry Sources Data Quality and Detection Blind Spots
Writing Detection Logic and Mapping Adversary Behavior
From Technique to Detection Logic MITRE ATT&CK Mapping for Detection Engineers
Testing, Tuning, and Reducing False Positives
Testing Detections with Adversary Emulation Tuning Detections and Reducing False Positives
Operating Detections in Production
Building a Sustainable Detection Program
Telemetry and Data Sources

Data Quality and Detection Blind Spots

Learn how missing fields, broken collection, inconsistent formats, and blind spots undermine detections, and how to assess data quality before writing rules.

In this lesson, you will learn to:

  • By the end of this lesson, learners will be able to identify common data quality problems, recognize detection blind spots, and perform a basic data assessment before building a detection.

Data Quality and Detection Blind Spots

This lesson focuses on data quality as the foundation of detection reliability. Students learn to recognize missing fields, broken ingestion, inconsistent formats, and coverage gaps. They practice a simple data assessment approach that prevents building detections on faulty foundations.

Common Data Quality Problems

Data quality problems are the silent killer of detections. A detection may look correct in a test environment and still fail in production because the data it depends on is missing, broken, or inconsistent.

Missing fields

A detection rule might reference a field such as process.command_line or source.ip. If that field is not present in the data, the detection will either fail or, more dangerously, silently never match. A common example is a detection that requires a field that is only populated in newer versions of an endpoint agent. On older agents the field is empty, so the detection never fires and nobody notices.

Broken ingestion

Log sources can break without warning. A connector may stop sending data. A file may be rotated and never read again. Time zones can be configured incorrectly, so events arrive with timestamps hours off. If ingestion is broken, the detection is blind even though the rule is perfect.

Inconsistent formats

The same kind of event may be formatted differently across systems. One data source may write usernames as jdoe while another writes DOMAIN\jdoe. A detection that searches for one format misses the other. Normalization is the process of making fields consistent, and it is a critical data engineering task.

Schema changes

Log formats change over time. A vendor may rename a field or change its type. If a detection is not updated, it may break. This is one reason detections should be treated as living artifacts, not set-and-forget rules.

The takeaway

Before writing detection logic, always ask: does this data exist, is it complete, is it consistent, and is it reliable? A short investigation can prevent hours of wasted work on a detection that can never succeed.

Data Quality Problems That Break DetectionsA light-theme diagram showing four common data quality issues: missing fields, broken ingestion, inconsistent formats, and schema changes, each with a warning marker.Data Quality Problems Break DetectionsA perfect rule is useless if the data behind it is brokenMissing FieldsRequired field is absent or emptyDetection silently never matchesBroken IngestionConnector stops sending dataDetection is blind to new eventsInconsistent FormatsSame field written differentlyRule matches one format onlySchema ChangesVendor renames or retypes fieldDetection fails silentlyAlways assess data before buildingCheck existence, freshness, field population, and format consistencyA thirty-minute data check prevents hours of broken detection work
Data Quality Problems That Break Detections — A light-theme diagram showing four common data quality issues: missing fields, broken ingestion, inconsistent formats, and schema changes.

Detection Blind Spots

A blind spot is an area of the environment where you have little or no visibility. Attackers actively look for blind spots because they can operate there without detection. Understanding your blind spots is as important as understanding your data.

Common blind spots

Several areas are frequently unmonitored:

  • Legacy systems. Older servers, industrial devices, or custom applications may produce little or no telemetry.
  • Encrypted traffic. While encryption protects privacy, it also hides payloads. You may see that a connection occurred without seeing what was sent.
  • BYOD and personal devices. Devices outside central management often lack endpoint agents.
  • SaaS applications. Third-party services may provide limited audit logs, and some activity may not be logged at all.
  • Third-party and supply chain access. Vendors and partners may have access to systems with less visibility than internal users.
How to handle blind spots

You cannot always eliminate a blind spot, but you can reduce its risk. First, document known blind spots so the team is aware. Second, add compensating controls, such as network monitoring where endpoint monitoring is impossible. Third, prioritize detections that work within the data you actually have. Finally, communicate blind spots to leadership so security expectations are realistic.

The danger of assuming full visibility

A common failure is assuming that because you have a SIEM and an EDR, you see everything. You do not. Attackers count on this overconfidence. An honest assessment of visibility gaps is a sign of a mature detection program. It is better to know your limits than to be surprised by them during an incident.

Assessing Data Before You Build

Before writing a detection, take thirty minutes to assess the data. This simple habit prevents most data quality failures.

A practical data assessment checklist

Ask these four questions:

  1. Does the data source exist in my platform? Confirm the source is connected and indexed.
  2. Is data arriving recently? Check timestamps and volume over the past days. A flat or missing trend is a warning sign.
  3. Do the required fields exist and contain values? Query for the fields your detection will use. Count null or empty values.
  4. Are the values in the expected format? Look at sample events. Confirm usernames, IPs, hashes, and other fields are normalized.
A simple example

Suppose you want to detect a known malicious file hash. Your detection will search for file.hash matching a specific value. Before building the rule, search for file.hash in your endpoint data. Ask:

  • Is file.hash present in process creation events?
  • Does it contain a value for most events, or is it often empty?
  • Is the hash in SHA256 format consistently?

If the field is missing from a large portion of events, your detection will have a blind spot. You may need to enrich the data or adjust the detection to use a different field, such as file path or filename.

Document what you find

Record the result of your data assessment in the detection documentation. Note which sources and fields you validated and any gaps you found. This documentation helps future engineers and auditors understand why the detection works and where its limits are.

Treat data assessment as the first real step of detection development. It is not busywork. It is the foundation.

Data Assessment ChecklistA light-theme checklist diagram showing four questions to assess data before building a detection: existence, freshness, field population, and format consistency.Data Assessment ChecklistAsk these four questions before writing any detection logic1Does the data source exist in my platform?Confirm it is connected and indexed2Is data arriving recently?Check timestamps and volume trends over the past days3Do the required fields exist and contain values?Count null or empty values for fields your detection will use4Are the values in the expected format?Review sample events for consistent usernames, IPs, hashes, and timestampsDocument your findings in the detection notes for future engineers
Data Assessment Checklist — A light-theme checklist diagram showing the four questions to ask before building a detection on a data source.
← Previous: Understanding Telemetry Sources Next: From Technique to Detection Logic →