Metrics
How fast alerts get looked at and decided, and how often Raksa agrees with analysts.
MTTD, mean time to detect
generated_at to reviewed_at: how long an alert sat in the queue before a human opened it.
MTTR, mean time to respond
reviewed_at to decided_at: how long from opening an alert to a recorded decision.
MTTD and MTTR trend, minutes (before Raksa, then with Raksa)
Demo data. Real baseline will be measured in the POC.Median time to decision per analyst, minutes
Demo data. Real baseline will be measured in the POC.Agreement vs override
Demo data. Real baseline will be measured in the POC.Analyst approved pre-decision87%
Analyst overrode13%
Based on 400 decisions. Every override reason feeds Rule Health.
Confusion matrix: AI pre-decision vs final decision
Demo data. Real baseline will be measured in the POC.| AI \ Final | False Positive | Exclusion / Whitelist | Open Ticket |
|---|---|---|---|
| False Positive | 217 | 9 | 4 |
| Exclusion / Whitelist | 11 | 90 | 7 |
| Open Ticket | 14 | 6 | 42 |
11
Missed escalations: analyst opened a ticket, Raksa had not suggested one
20
False escalations: Raksa suggested a ticket, analyst closed it otherwise