[ PROOFECONOMY ]™GlossaryOriginSpecsSecurity Effectiveness
Security effectiveness is the degree to which security controls actually prevent, detect or limit attacks in a real environment. As commonly measured by security validation tools, it is usually reported by the vendor's own tooling; independently witnessed, it becomes Proof of Efficacy.
What it means
Security effectiveness asks a fair question: do the controls we paid for actually work against real attacks? It became an industry category with security validation and breach and attack simulation (BAS) tools, which run attack techniques against an organization’s controls and report what was prevented, detected or missed.
The term is widely used. The Proof Economy uses it too, and adds two questions most measurements of it skip.
The two questions it usually skips
- Who witnessed the result? In most security effectiveness programs, the attack tool, the scoring and the report come from the same vendor or the same internal team, and the evidence is the controls’ own logs and alerts. That is verification: the system speaking for itself. When it is sold as proof, it is a Self-Attestation Oxymoron.
- Did the attack fail, or did something just alert? Many effectiveness scores count detections. A control that alerts on an attack that still succeeds has not been effective. The outcome has to be observed where the attack’s effect would land.
Security effectiveness and Proof of Efficacy
| Security effectiveness (as usually measured) | Proof of Efficacy | |
|---|---|---|
| Question | Are the controls working? | Did the control produce the required outcome, under real conditions, at a standard good enough to matter? |
| Evidence | Control logs, alerts, tool reports | The observed outcome at the Effect Point |
| Witness | The validation tool or the team running it | An independent Proof Witness, out of band |
| Expectations | Often set or interpreted after the run | Committed before the run to an independent time source |
| Tied to | A product or a dashboard | The fingerprint of the exact control tested. See Open Control |
Proof of Efficacy is security effectiveness measured so that someone other than the vendor can rely on it.
Example
A validation tool runs 200 techniques and reports 85 percent security effectiveness, based on the controls’ alerts. Under Proof of Efficacy, the same techniques run as a blind mix with legitimate traffic, the expected outcomes are sealed beforehand, and an independent witness at each target records whether the attack’s effect actually landed. Some of the alerted attacks turn out to have succeeded anyway, and the real effectiveness number is lower and can be checked.
Related terms
- Open Control
- Proof of Control
- Proof of Efficacy
- Proof of X
- Proof Witness
- Self-Attestation Oxymoron (SAO)
- Verification of Control