FasterRisk builds risk scores you can compute on an index card

Risk scores have been the most common interpretable model for high-stakes decisions in criminal justice, healthcare, and elsewhere for a long time. They are a short list of features, each worth a few integer points, summed into a score that maps to a probability. A person can compute one without a calculator, and counterfactuals (“what if this patient were older?”) are trivial to reason about.

The trouble is how they get made. Traditionally they come either from expert knowledge, validated only after deployment, or from a semi-manual process of picking features by hand and rounding logistic regression coefficients. Neither reliably produces a good score, and learning one directly from data is NP-hard. Jiachang Liu, Chudi Zhong, Boxuan Li, Margo Seltzer, and Cynthia Rudin of Duke University built FasterRisk to make that learning fast enough to be practical.

This post was written by Renz Tegrado with layout by Dahna Galvez.

Approach

FasterRisk works in three steps. First, a beam search finds a high-quality sparse continuous solution, the best small model at each iteration by logistic loss. Second, it generates a diverse pool of continuous sparse solutions that are almost as good as the first but use different support sets, so a user can pick one that matches their domain knowledge. Third, it converts each continuous solution to integers with a “star ray” search: multipliers extend the coefficients outward from the origin like rays, and a local search with sequential rounding finds integer solutions along them. A theoretical bound on the loss gap between the continuous and rounded solutions keeps the result close to optimal.

The models were scored by AUC and compared with baseline methods, and runtime was compared against RiskSLIM, the previous state of the art.

Findings

FasterRisk outperformed the baselines on every dataset tested, including RiskSLIM, on both training and test AUC. The authors credit the multipliers, which enlarge the hypothesis space. The advantage held even when RiskSLIM was given more time. ElasticNet produced low-quality sparse solutions and could not control exact model size. FasterRisk also finished far faster than RiskSLIM, which timed out at 900 seconds on many of the larger problems, and it scaled to high-dimensional feature sets. One example model, predicting malignancy of a breast lesion from five features, reached an AUC of 0.854 on training data and 0.853 on test data.

Limits

FasterRisk does not guarantee an optimal solution; that is the trade for its speed. Its models are not causal, they are sparse additive models with limited capacity, and they inherit whatever flaws the training data has. Interpretability makes bias easier to spot than in a black box, but does not remove it, so the authors encourage users to inspect several models from the pool and look for a fairer one. The code is open source.

Source: Liu, J., Zhong, C., Li, B., Seltzer, M., and Rudin, C. (2022). FasterRisk: Fast and Accurate Interpretable Risk Scores. Advances in Neural Information Processing Systems 35. arXiv:2210.05846. Code: github.com/interpretml/FasterRisk.

Originally shared on Facebook.