Anomaly Detection in OT: Building Baselines That Hold Up

The Myth of the ‘Normal’ Baseline

When industrial operators first approach us regarding OT security, they often arrive with a specific tool in mind: an anomaly detection platform. They want to plug in their network taps, let the algorithms run for a month, and then turn on the alarms. The expectation is that once the system “learns” the environment, it will automatically distinguish between routine PLC communication and a sophisticated adversarial campaign.

This approach fails because it treats Operational Technology (OT) as a static IT environment with noisy inputs. It does not account for the dynamic nature of industrial processes, the strict timing requirements of control loops, or the human element of plant operations. As we have observed across our engagements, monitoring is not just IT intrusion detection pointed at a plant network. It requires asset awareness, protocol context, behavioral baselines, operational knowledge, and alert handling that avoids false positives and unnecessary disruption.

In this post, we will explore why building baselines that hold up during commissioning, maintenance, and normal process variation is the critical first step in OT security. We will also look at recent intelligence from the water utility sector to understand the cost of getting this wrong.

Why Protocol Awareness Trumps Generic Heuristics

The fundamental flaw in many legacy monitoring solutions is their inability to interpret the semantics of industrial protocols. In an IT network, a connection attempt to port 445 might be suspicious depending on the source IP. In an OT network using Modbus TCP or DNP3, the context is entirely different.

Detection capabilities must account for industrial protocols, legacy systems, low-bandwidth links, and air-gapped or segmented architectures. For example, a Rockwell Automation PLC communicating via EtherNet/IP expects specific polling intervals. If an attacker attempts to write to a holding register outside of that sequence, the deviation is significant. However, if a Siemens S7-1500 performs a firmware update via TIA Portal, the traffic pattern will look like a sustained, high-volume data transfer that could easily trigger a generic “data exfiltration” alert in an IT-focused SIEM.

To build a baseline that holds up, your monitoring stack must be protocol-aware. It needs to understand OPC UA method calls, recognize the heartbeat signals of a Schneider Electric Modicon PLC, and identify when a Honeywell DCS is behaving outside its normal operational envelope. Without this deep inspection, you are merely collecting logs, not generating actionable intelligence.

The Human Context: Reducing False Positives in Live Plants

One of the most significant challenges in OT anomaly detection is the “boy who cried wolf” effect caused by excessive false positives. In our engagements, we have found that anomaly detection is especially valuable when the system can distinguish normal operational variation from suspicious activity. This distinction relies heavily on human context.

Consider a scenario where a maintenance team replaces a pressure transmitter in a chemical processing unit. The new device may have slightly different response times, calibration offsets, or communication polling rates. To an untrained algorithm, this looks like an unauthorized device insertion or a configuration drift. To the plant manager, it is routine maintenance.

This is why OT analysts should understand operations well enough to separate malicious activity from maintenance activity, commissioning activity, and normal process changes. Your anomaly detection strategy must include a “whitelist” of known operational events. When commissioning work begins, the monitoring system should switch to a “learning mode” or a “maintenance window” where deviations are logged but not alarmed until the work is complete.

Furthermore, consider the role of shift handovers. Process parameters often change between day and night shifts to optimize energy consumption or throughput. A robust baseline accounts for these periodic variations. If your baseline only captures a single 24-hour cycle, it will flag every subsequent deviation as an incident. Instead, baselines must be dynamic, adapting to seasonal changes, product mix adjustments, and equipment aging.

Recent Water Utility Attacks: A Cautionary Tale

The importance of accurate baselining is not just theoretical; it is a matter of national security and public safety. Recent events in the water utility sector have highlighted severe gaps in OT monitoring capabilities.

According to a report by Forescout Vedere Labs, Minnesota water utilities lack adequate network segmentation and monitoring for OT systems. The attacks revealed that legacy OT equipment in water treatment facilities remains unpatched and exposed to cyber threats. This is not an isolated incident. The OTCC recently urged CISA to issue a binding OT cybersecurity directive after cyberattacks disrupted water utilities across at least seven U.S. states.

These incidents underscore a critical failure mode: when baselines are absent or inaccurate, operators cannot distinguish between a sensor malfunction and a malicious command. In a water treatment plant, a false negative could lead to chemical dosing errors, while a false positive that triggers an emergency shutdown could disrupt the supply chain for an entire region.

The Nozomi Networks Labs research on securing substations and power grids further confirms this trend, noting that ICS anomaly detection can identify subtle, previously undetected threats in power grid infrastructure. Their approach integrates behavioral analysis to reduce false positives in OT security monitoring. This is the same principle that applies to water, oil and gas, and manufacturing: you must know what “good” looks like before you can spot “bad”.

Building Baselines That Hold Up: A Practical Framework

To implement effective anomaly detection, we recommend a phased approach that prioritizes operational continuity while enhancing security visibility. Here is a step-by-step procedure for building baselines that hold up under pressure:

1. Conduct a Comprehensive Asset Inventory. You cannot monitor what you do not know exists. Monitoring should maintain an evolving picture of assets, configurations, firmware, and communication patterns. New devices, unauthorized changes, or control logic modifications can be early indicators of risk. This inventory must include both IT-facing gateways and the actual field devices (valves, pumps, RTUs).
2. Define Operational Use Cases. Work with process engineers to identify critical workflows. For example, what is the normal startup sequence for a boiler? What are the expected DNP3 read cycles for a substation? These use cases become the foundation of your baseline profiles.
3. Implement Protocol-Aware Monitoring. Deploy sensors that can parse Modbus TCP, Ethernet/IP, Profinet, and OPC UA. Ensure these sensors can correlate events across different protocols. For instance, if an HMI workstation initiates a write command via OPC UA that results in a Modbus coil change downstream, the monitoring system should link these events.
4. Establish Dynamic Baselines. Avoid static thresholds. Use machine learning models that adapt to seasonal changes, production shifts, and equipment aging. Regularly validate these baselines against operational logs to ensure they reflect current reality.
5. Integrate Human-in-the-Loop Verification. Create a workflow where alerts are reviewed by OT analysts who understand the process. Provide them with context: “This alert coincided with a scheduled maintenance window for Pump 3.” This reduces fatigue and increases trust in the system.
6. Test During Planned Outages. Before relying on anomaly detection for critical security decisions, test your baselines during planned maintenance windows. Verify that the system correctly identifies deviations from the baseline during known operational changes.

The Role of Governance and Standards

Technology alone cannot solve the OT monitoring challenge. It must be supported by strong governance. In our engagements, we often find that organizations struggle with incomplete asset inventories, missing or non-OT-specific security policies, undefined security roles/responsibilities, lack of adopted security standards/frameworks, insufficient OT-specific awareness/training, and continuous monitoring not tied to business/security objectives.

To address this, align your monitoring strategy with recognized frameworks such as IEC 62443, NIST SP 800-82, and NERC CIP. These standards provide a structured approach to defining security zones, conduits, and monitoring requirements.

For example, NERC CIP requires specific logging and monitoring for Bulk Electric System (BES) cyber assets. By mapping your anomaly detection alerts to CIP requirements, you can streamline compliance reporting and demonstrate due diligence to regulators. Logging, evidence collection, and reporting can support frameworks such as NERC CIP, IEC 62443, NIS2, and industry-specific requirements.

Conclusion: From Detection to Understanding

Building baselines that hold up in an OT environment is not a one-time project; it is an ongoing process of refinement. It requires a deep understanding of industrial protocols, operational processes, and the human factors that drive plant operations.

As we have seen in the water utility sector and power grid infrastructure, the cost of inaccurate monitoring is too high. Operators need to distinguish between benign operational anomalies and malicious activities in real-time. This demands more than just tools; it requires a culture of continuous learning and collaboration between IT security teams and OT engineers.

By focusing on protocol awareness, dynamic baselining, and human context, you can build an OT monitoring program that reduces false positives and enhances your ability to detect true threats. Don’t let your anomaly detection system become another source of noise in an already noisy environment. Make it a source of clarity.

Ready to Secure Your OT Environment?

If you are struggling to build effective baselines or need help assessing the maturity of your OT monitoring program, Red Trident is here to help. Our team of OT security experts has deep experience in industrial protocols, compliance, and operational technology. We offer a free OT security assessment consultation to help you identify gaps and prioritize your security investments.

Contact us today to schedule your consultation and take the first step toward a more resilient OT environment.