A Question Every Leader Should Ask
In the past few months, I’ve been asked the same question twice — by two different CEOs of utility companies on two different continents:
“Why am I scoring this low while we’ve invested so much in purchasing the latest technologies?”
Both had leading firewalls, OT detection tools, and carefully written cybersecurity policies. Yet, their assessed maturity level was below 1 out of 5.
That question captures a global truth: Technology spend doesn't equal security maturity. The difference lies in one word — effectiveness.
From Compliance to Confidence
Many organizations mistake compliance for capability. They can produce audit trails and policies, but when an event occurs, detection lags, response slows, and coordination breaks down.
Frameworks like NIST CSF 2.0, ISA/IEC 62443, and MITRE ATT&CK for ICS are essential — but they serve only as blueprints. What matters is whether the controls built on them actually perform when needed.
You don’t measure maturity by what you own. You measure it by what you can prove works.
Policy on Paper vs. Policy in Practice
Too often, incident response plans exist only in binders. When a real incident happens, teams scramble to interpret roles, contacts, and approvals.
Tabletop exercises are the simplest, most revealing test of policy effectiveness. They surface hidden gaps in:
- IT/OT coordination under pressure
- Escalation paths and communication with leadership
- Safety and operations approvals under MOC
- Recovery decision authority
A policy that hasn't been practiced is just an assumption.
Technology Isn't the Problem — Testing Is
Controls degrade quietly over time. Rules change. Exceptions get added. Tools fall out of tune. Without validation, visibility becomes a comfort illusion.
Every organization should regularly test:
- 🔹 Firewalls: Do segmentation rules really block what they should?
- 🔹 OT IDS: Have all assets been discovered and classified? Are anomaly models tuned and current?
- 🔹 SOC Integration: Are alerts routed for action — or just stored?
- 🔹 Detection Effectiveness: Do alerts fire when realistic ATT&CK for ICS behaviors are simulated?
- 🔹 Response Timing: How long does it take for a detection to reach a decision-maker (MTTD/MTTR)?
Testing reveals what dashboards can't.
Control Effectiveness = Assurance
A mature OT security program continually validates, measures, and adjusts. That means:
- Asset assurance: Know every PLC, HMI, and engineering node — and verify it quarterly.
- Segmentation assurance: Simulate cross-zone traffic to confirm zone/conduit integrity.
- Detection assurance: Use controlled simulation to test visibility and alert quality.
- Recovery assurance: Restore PLC logic in a sandbox; don't just trust the backup log.
Cyber resilience grows through evidence, not effort.
Why Many Still Score Low
Because maturity frameworks measure assurance, not acquisition, they're designed to show how much you know, not how much you've bought.
The real question every leader should now ask is:
“Can we prove our controls are effective — or just that they exist?”
That's where leadership turns awareness into assurance.
The OT CISO View
This month's OT CISO Leadership Advisory focuses on the effectiveness of control and technology in the face of coordinated ICS threat campaigns observed globally. It offers a concise executive checklist and 24-hour action plan aligned with NIST CSF 2.0, IEC 62443, and MITRE ATT&CK for ICS.
Final Thought
Technology investment shows intent. Effectiveness testing shows control. Cyber maturity, in the end, is not what you purchase — it’s what you can prove works when you need it most.