Sensitivity analysis for model risk management
Sensitivity testing with dependence has the potential for a wide range of applications in reporting, such as for Solvency II, IFRS 17, and balance sheet valuation.
In March, a 25-year-old police employee filed a workers’ compensation claim for lower back pain that started shortly after tackling a suspect. A few days later, a 52-year-old police officer filed a claim for a hand injury he sustained on the job. Which claim will develop higher costs?
Back injuries are notoriously costly, even seemingly minor ones, compared with a hand injury. The smart money might be on the back injury claim. But add co-morbidities like diabetes and arthritis to the hand injury and the odds become more of a toss-up.
Identifying workers’ compensation claims with potentially high costs – before those costs have developed – has, for the most part, eluded insurance and self-insured companies, even though they pay more than $60 billion in workers’ compensation benefits every year. Over 80% of all workers’ compensation costs come from a small number of unpredictable, high-cost claims. Conversely, lower-cost med-only claims account for 75% of total claims but only 5% of the cost.
These hard facts are the claims reality that insurers and self-insurers like the Intergovernmental Risk Management Agency (IRMA) face daily. Focused on improving performance and reducing costs for its 70-plus member organizations, the Chicago-area member-owned public risk pool had been looking for a way to increase efficiency around workers’ compensation losses while improving accident prevention and claims responses for its members’ workforce.
A main challenge to peeling back the mystery in its “unknown high-cost claims” was finding a way to access the predictive data in adjusters’ notes, case manager notes, and other sources of text data. This unstructured, free-form data could provide an early warning of potentially costly claims. Getting at it, however, requires claims managers to read through reams of adjusters’ notes. This manual process typically delays identification of potentially problematic claims and the assignment of appropriate resources, and, given the glut of material to sift through, can lead to hit-or-miss results.
Developed for early claims intervention, Milliman’s web-based predictive modeling and decision-support system seemed to be an excellent fit for IRMA. Using machine learning to analyze a wide range of claims data, the software, known as Nodal, triaged incidents shortly after they were reported, scoring claims based on the likelihood that they would incur high costs.
The software’s text-finding algorithm systematically reads through adjusters’ notes and other unstructured data, scraping the formerly underutilized information for significant characteristics like co-morbidities and other health factors that were previously not easily accessible to IRMA claims managers. The software then flags claims that, while they often look like any other, have latent characteristics associated with potentially high costs; these are tagged as potential jumper claims, meaning they are likely to increase significantly in value in the future.
Armed with this information, IRMA claim managers were able to make timely, informed decisions in assigning adjusters with the appropriate experience to specific claims based on the claims’ cost potential. Jumper claims were assigned to a more experienced adjuster who could bring in the appropriate resources early on in the process; this has the combined goal of helping injured claimants with early treatment plans and an expedient claims process, while simultaneously mitigating cost more effectively. Problematic claims could now be referred to senior claims management on a daily basis. What had been an exceedingly time-consuming and somewhat limited manual process of allocating resources to a claim was now automated with Nodal.
Nodal’s text mining capacity also enabled IRMA claims managers to gain insight into formerly vaguely coded injuries such as those involving multiple body parts. These claims, which could include injuries ranging from minor finger or hand sprains to complicated neck/back injuries, had extremely different risk profiles. In the past, identifying the costliest claims in this category had been a laborious process. But by scraping the data in adjusters’ notes, Nodal was able to uncover claims that involved a head injury, opioid use, or other high-cost triggers.
Nodal’s ability to extract from unstructured data information on comorbidities, scheduled surgeries, MRIs, or other planned procedures also meant IRMA’s claims adjusters often had prior notification of costly treatments before they were scheduled, presenting an opportunity both to help manage a claimant’s treatment and to plan for and mitigate the claims’ outcomes before incurring the cost. Beyond the cost-containment benefits, Nodal’s findings allow insurers and employers to search for patterns in various claims, delivering insight that could be used to help prevent against future incidents.
By reviewing IRMA’s complete claims history and analyzing its claims’ development, Milliman was able to provide IRMA’s claims team with customized cost scores based on the organization’s historical data. These weighted claims scores gave IRMA claims managers an unbiased, personalized measure of the potential cost for newly filed claims.
Before implementation, savings were expected to fall between 3% and 10%, or $530,000 to $1.8 million annually, but IRMA’s actual results over the past year far exceeded initial projections.
Since implementation, Nodal correctly identified 19 of the 20 largest claims for IRMA within 30 days, giving IRMA management an opportunity to provide additional oversight and allocate the appropriate resources to the most problematic claims.
As a result, in the past year, IRMA has seen:
These gains also reflect IRMA’s willingness to embrace a new technology, incorporate it into its processes, and consistently rely on the information provided by Nodal to inform its decision-making.
Looking ahead, IRMA expects to have:
For a risk pool like IRMA, continued improvements in claims frequency and severity isn’t just a matter of numbers. These improvements could mean lower premiums for its excess coverage and savings on claims that will give the pool the ability to reinvest in safety or other initiatives that further its mission.
Using predictive modeling to forecast large workers’ compensation costs: A case study
Nodal™, Milliman’s web-based predictive modeling and decision-support system, accurately identified 95% of high-cost claims for the Intergovernmental Risk Management Agency before they happened, reducing costs by over 15%.