Roundtable on expert judgment and overrides in behavioral modelling

Aim

Share challenges and ideas regarding the application of expert judgment and overrides in behavioral modeling of prepayments and savings.

Context

P&P observes that banks are currently facing lots of challenges in predicting client behavior, e.g.:

  • Due to a long trend of declining interest rates, the historic data provides limited information about client behavior under an increasing interest rate;
  • COVID-19 has led to an increase in client savings despite a zero- or negative interest rate;
  • The elasticity of savings may change in the future, g., due to aggregators or PSD2;
  • Historical prepayment behavior may not be representative of the future, due to changes in portfolio composition, rate fixing tenors, legal changes, and housing market

P&P deems it is challenging to combine existing models, calibrated on historical behavior, with available expertise and expectations on future behavior. When should expert opinion be used, and how do you substantiate its use?

Organization

A total of 9 Dutch retail banks joined the roundtable. Each bank was invited to bring two ALM colleagues, one with experience in modelling and the other with experience in the first line. Two rounds of parallel sessions were held. During the first-round participants could choose between discussing prepayment- or savings modeling. The second round focused on governance of expert judgment and overrides. As the level of governance is related to the balance sheet size of a bank, a split was made between larger and smaller banks. Each parallel session was supported by several statements to guide the discussion.

Prepayments, Savings & Governance

Prepayments

Main discussions points on prepayment behavior for Dutch retail mortgages:

  • Participants deem that the effect of COVID-19 on prepayments is limited. With regard to prepayments, COVID-19 is not considered to be a more extreme macro-economic event than crises observed in the COVID-19 data should therefore not be excluded from modeling.
  • Participants deem it is difficult to say whether prepayment behavior on mortgages with a 20- year fixed rate is similar to clients that chose a long fixing tenor in the past. Whereas in the past behavior was related to the fixing tenor, clients are now expected to act roughly the
  • Participants do not necessarily think the balance sheet is becoming more sensitive to client behavior due to longer fixing One participant argues that the client incentive is becoming more pronounced, making it easier to predict what the client will do. Especially when interest rates rise it is expected that the majority of the portfolio has no financial incentive to prepay, and therefore becomes more predictable.
  • Some participants deem it is irrational that clients fix their mortgage for a very long period, as clients always need to pay more to fix for a longer period. Other participants deem that the additional premium for fixing longer is worthwhile, as the premium for a longer tenor is low in relation to the long-term financial security it provides to a
  • Participants think that client prepayment behavior is sensitive to more factors than changes in interest rates. A longer fixing tenor also increases the exposure to changes in other factors, such as changes in fiscal treatment and general economic
  • Participants observe that it is currently difficult to find a new home, which makes it more difficult for clients to act on their financial incentive to move and fully prepay without

Savings

Main discussions points for behavior of Dutch retail savings:

  • Participants deem that dealing with COVID-19 data is challenging. COVID-19 coincides with declining interest rates and excess liquidity in financial markets, and it is hard to disentangle which behavior is due to COVID-19 and what is caused by other effects. In addition, the duration of COVID-19 also affects the stickiness of
  • Some participants argue that uncertainty plays a large role in saving, which can be partly driven by COVID-19. It is suggested that additional savings volume obtained due to COVID-19 should not be attributed to extra stable funding and should have a different
  • Participants expect that term deposits will become more popular once interest rates start to rise again. However, a migration to term deposits mainly depends on whether rates stabilize. It is not expected that clients return to term deposits when clients expect that interest rates will continue to The attractiveness of term deposits will also depend on the aggressiveness of the bank’s pricing strategy of term deposits vs. on-demand deposits.
  • Most participants think that savings modeling should be predominantly expert based in an increasing interest rate environment. However, participants observe that there is a lack of experts and a risk of group think. Furthermore, participants consider it harder to defend an expert-based model towards the supervisor than a quantitative

Governance

Main discussions points on governance of expert judgment and overrides:

  • Participants deem that the validity of expert judgment should be closely monitored. It should be assessed whether the model continues to be exposed to outliers in its input data, and whether expert judgment sufficiently adjusts for the target variable. The intended validity of expert judgment should be longer than 3 The expert judgment should be reviewed at least annually.
  • Participants think it is not always possible to backtest expert However, the inability to perform a backtest is not necessarily a reason to avoid the application of expert judgment. One participant argues that not explicitly applying expert judgment in a changing environment is indirectly also an application of expert judgment which requires substantiation.
  • One participant argues it is important to be aware that agreeing experts can also be wrong (e.g., predicted decline of the housing market at the start of the COVID-19 pandemic).
  • Participants from smaller banks generally aim to avoid the application of expert judgment. Small banks have a limited number of potential experts, so it is hard to avoid group think. Expert judgment is mainly used by small banks when the data history is not sufficiently
  • Participants observe that the supervisor generally has an understanding attitude when expert judgment and overrides are applied. The governance surrounding expert judgment is often scrutinized more than the judgment
  • Participants have mixed experiences with using alternative data to support expert judgment Some participants deem that if relevant alternative data is available, it should be used in the regular model development process. Smaller banks consider that the use of alternative data and challenger models are a luxury for which priority, budget, or capacity are usually not available. Larger banks do apply challenger models.

Do’s and dont’s

Our high-level takeaways from the session are the following:

Do’s

  • Record your decisions.
  • Have experts form their opinion independently.
  • Frequently assess expert judgment, at
    least annually.
  • Monitor the added value of expert judgment in the model, by comparing performance against the same model without expert judgment.
  • Try to substantiate expert judgment with tangible information not used by the model.
  • Apply challenger models to quantify and
    justify expert judgment

Dont’s

  • Cherry pick or give the impression of cherry picking. Refrain from expert judgment that influences short term profits.
  • Fall into a status quo trap / avoiding expert judgment because it is hard to explain. Not applying expert judgment is also a choice which should be assessed.
  • Close your eyes for impactful macro, fiscal, or other changes that might influence behavior substantially.
  • Trust experts unconditionally. Stay critical of both models and experts.
  • Rely on a single dominant person. Avoid group think and organize an independent challenge.
  • Leave out COVID-19 data.

Contact

Gerd-Jan van Wiggen

Gerd-Jan van Wiggen