Predicting suicide risk in 137,112 people with severe mental illness in Finland: external validation of the Oxford Mental Illness and Suicide tool (OxMIS)

Oxford Mental Illness and Suicide tool (OxMIS) is a standardised, scalable, and transparent instrument for suicide risk assessment in people with severe mental illness (SMI) based on 17 sociodemographic, criminal history, familial, and clinical risk factors. However, alongside most prediction models in psychiatry, external validations are currently lacking. We utilised a Finnish population sample of all persons diagnosed by mental health services with SMI (schizophrenia-spectrum and bipolar disorders) between 1996 and 2017 (n = 137,112). To evaluate the performance of OxMIS, we initially calculated the predicted 12-month suicide risk for each individual by weighting risk factors by effect sizes reported in the original OxMIS prediction model and converted to a probability. This probability was then used to assess the discrimination and calibration of the OxMIS model in this external sample. Within a year of assessment, 1.1% of people with SMI (n = 1475) had died by suicide. The overall discrimination of the tool was good, with an area under the curve of 0.70 (95% confidence interval: 0.69-0.71). The model initially overestimated suicide risks in those with elevated predicted risks of >5% over 12 months (Harrell's E-max = 0.114), which applied to 1.3% (n = 1780) of the cohort. However, when we used a 5% maximum predicted suicide risk threshold as is recommended clinically, the calibration was excellent (ICI = 0.002; E-max = 0.005). Validating clinical prediction tools using routinely collected data can address research gaps in prediction psychiatry and is a necessary step to translating such models into clinical practice.

Automated summary

The Oxford Mental Illness and Suicide tool (OxMIS) is a standardized tool for assessing suicide risk in individuals with severe mental illness. However, external validations for this tool are currently lacking. In this study, the performance of OxMIS was evaluated using a Finnish population sample of individuals diagnosed with severe mental illness. The tool showed good overall discrimination and excellent calibration when using a recommended 5% maximum predicted suicide risk threshold.