Data Science Behind Our CVE Probability of Exploit

Orpheus model ranks CVEs as exploited 94% of the time a month before exploitation is confirmed.

Orpheus uses AUROC, which compares the True Positive Rate against the True Negative Rate at each threshold.

The CWP model has an AUROC of 0.94 when considering snapshots of features a month before exploitation is confirmed. This means that the model ranks CVEs that are later confirmed as exploited above non-exploited CVEs 94% of the time. Six months before exploitation is confirmed, the AUROC is 0.91. These values were calculated on the test set, a subset of historical data the model had not yet seen.

Using the confusion matrix below, we can assess that by patching just the top 5% of extant CVEs, we would capture 85% of the CVEs that will be exploited in the next 30 days.