Reserve Uncertainty Characterization

[Gabriel]

Producing a full distribution of reserves is among the latest trends in reserving techniques. From my experience, the two most standard industry models are the Mack and the ODP methods. Whichever method is used, my question is: How should we as Actuaries characterize the results from these models to our (non-Actuarial) audience?

I checked the ASOP literature but it does not seem to have insight specifically on the characterization of a range (perhaps the techniques are too new?).

I have come across several suggestions from various presentations and sources, none of which I think is perfect. I also must add that I think communication of the limitations of these models is much more important. This communication begins, however, with an appropriate characterization.

Some suggestions I have seen, and my interpretation of each (debate encouraged, should you take issue):

"The range of reasonable estimates" - This, to me, is the most limiting. It conveys that if we took 10,000 Actuaries and compiled their best estimates, this would be the range of best estimates. I do not think that this is the distribution implied by the ODP/Mack processes.

"A range of reasonable outcomes" - First, replacing the "The" with "A" is a good start as it implies that there are other reasonable ranges (consistent with the fact that there are several methods to produce ranges). Second, I suggest that the ODP/Mack methods do produce "reasonable outcomes" if reasonable outcomes are those that are reasonably projected from the historical data. I may include a definition of reasonable as those that are "reasonably forseeable" based on the historical data. On the downside, however, can what the ODP/Mack methods produce as a 90% or 95% or 99% really be characterised as "reasonable"? I have my concerns...

"A distribution of reasonable outcomes" - Replacing "range" with "distribution" is another good change. One may suggest that a "range" of estimates is more limiting than a "distribution". A range is often used in a SAO to describe the difference to the best estimate within which an Actuary is willing to give a "within a reasonable range" opinion. A distribution perhaps implies that there are more extremes.

"A distribution of possible outcomes" - I have seen this terminology used in many places and it makes me the most uncomfortable, but I can be convinced otherwise. I suggest that there are many "possible" outcomes that are not reflected in a standard ODP/Mack method, specifically that reserves will develop in a way not sufficiently captured in the historical data. In some reports that use this characterization, they also include a limitation that the range or estimates captures "process and parameter error", but does not capture "model risk", "operational risk", "regulatory risk", "legal risk", "political risk", or "economic risk". I think that this is more accurate, but I am concerned that the initial characterisation of the reserves is displayed much more prominiently than the subsequent limitations.



What say you AO? Anyone care to share how they characterize these results?

[MountainHawk]

I think what you get out of Mack or other similar methods is 'a distribution of reasonable estimates.

If management wants a range, they should either set the parameters ("reasonable is the 30th to 70th percentiles"), or if it is open ended, you should make a selection but be clear on what you have selected.

For the SAO, I'm of the belief that we should be opining if the selected reserve is in a 'range of reasonable estimates', which I interpret as a much more narrow range, with the low end of the range either right at or just below select. If my best estimate is $x,xxxM of reserves, how can I consider it reasonable for the company to base it's financials on a number that is 5% short of that?

[TwistedMentat]

Originally Posted by Gabriel

"The range of reasonable estimates" - This, to me, is the most limiting. It conveys that if we took 10,000 Actuaries and compiled their best estimates, this would be the range of best estimates. I do not think that this is the distribution implied by the ODP/Mack processes.

I consider this to reflect parameter uncertainty included in the ODP or Mack output, but not the process variability. If stochastic methods are used, this is what the reserving actuary references in the PSAO.

Originally Posted by Gabriel

"A distribution of possible outcomes" - I have seen this terminology used in many places and it makes me the most uncomfortable, but I can be convinced otherwise. I suggest that there are many "possible" outcomes that are not reflected in a standard ODP/Mack method, specifically that reserves will develop in a way not sufficiently captured in the historical data. In some reports that use this characterization, they also include a limitation that the range or estimates captures "process and parameter error", but does not capture "model risk", "operational risk", "regulatory risk", "legal risk", "political risk", or "economic risk". I think that this is more accurate, but I am concerned that the initial characterisation of the reserves is displayed much more prominiently than the subsequent limitations.

This includes both process variability and parameter uncertainty. This is not referenced in the PSAO, but is used to estimate required economic capital for non-diversifiable insurance risk. As you mention, this does not capture all sources of insurance risk, so required economic capital should contain a provision for un-modeled risk which is inherently judgmental.

[Ganymede]

If you predict the distribution of possible outcomes, when the outcome is known you can go back and check to see how you did. It is difficult to draw a conclusion on a single outcome. You need to look at several outcomes. If you see that most of the outcomes that fall in the tails (over 95th percentile or under the 5th percentile) of your predicted distribution, you should question your methodology.

I really don't like any ranges that are characterized by "reasonable." No matter what the outcome, you can always rationalize your range as being "reasonable."