More and more organisations are developing and using greater numbers of models to make and inform decisions across the business. And many of these models are becoming more complex due to the increasing ease of accessing and utilizing data in an automated way.  However, models can and do go wrong, and sometimes with disastrous consequences.  The Space Shuttle disaster in 1986, fatalities due to autonomous driving cars more recently, and even a biology text book priced on Amazon at $23 million – were all unintended consequences of models going wrong.    

Model Risk Management (MRM) is the practice of identifying and mitigating the potential of model error or wrongful model usage.  Where models are used by organisations to make or inform decisions, many industry regulators are demanding effective MRM practices to protect organisations, and their customers, prospects and partners.    

So why do models go wrong? There are many reasons why models can go wrong and the causes can be found throughout the model lifecycle.  From model development, to model implementation, model usage, model tracking and maintenance – if mistakes are made, incorrect assumptions overlooked, or steps missed then models may not work as planned.  It goes without saying that if things are done right the risks are minimised and the business value maximised.    

Doing the right things includes defining, documenting and adhering to best practice standards across the model lifecycle.  These standards will ensure the correct steps are taken, the correct reviews performed, the correct approvals made for model development, data used, model design, model validation, model implementation and ongoing model monitoring.  For businesses that fall under specific industry regulators, correctly defining and following these standards will help achieve compliance.  But more importantly, these practices will drive business benefits. 

An effective MRM system will comprise of:  

• a centralised model inventory  

• defined and documented modelling standards  

• implemented governance workflows across the model lifecycle  

• reporting of model risk statuses, actions, due dates, remediation plans etc.  

By having everything centralised, documents are easy to find, complete audit trails can be easily reviewed, and all actions, recommendations, plans and approvals are logged and reported ensuring that nothing gets missed.  

The MRM system can be described simply as a solution that:  

1. ensures we “do the right thing” 

2. easily demonstrates and proves that we “did the right thing”  

3. puts in place plans and processes to ensure that we will “continue to do the right thing”.  

There is a lot of detail that has to be included within a comprehensive MRM system which is not the purpose of this article.  But what is important to note is that MRM should not be seen as an overhead, an additional cost and box ticking exercise for compliance, but as a way to instill and ensure best practices, and gain business value and competitive advantages from investments in data and analytics.  Strong MRM minimises model related issues and enables the implementation and use of sound, robust and value-add analytic models across your organisation.    

To see how our software solution, Focus, addresses the needs of Model Risk Management take a look here https://www.credit-scoring.co.uk/focus