Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Property - Casualty / General Insurance
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


Upload your resume securely at https://www.dwsimpson.com
to be contacted when new jobs meet your skills and objectives.


Reply
 
Thread Tools Search this Thread Display Modes
  #11  
Old 04-11-2019, 03:40 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 134
Default

I did find This Stackexchange on discrete models useful. It goes over various metrics to evaluate your model.

There's also the DHARMa package that scales the Q-Q plot to work for poisson.
Reply With Quote
  #12  
Old 04-11-2019, 03:52 PM
MoralHazard MoralHazard is offline
Member
CAS
 
Join Date: Jul 2011
Favorite beer: Sam Adams Rebel Rouser
Posts: 110
Default

Quote:
Originally Posted by Actuarially Me View Post
Thanks, I'll give that a try too. I added my code for calculating the gini. Is that how you would do it for frequency? One of my biggest fears is that I've been calculating it incorrectly, which I'm thinking I am.
You should first be converting the claim count prediction to a frequency prediction (meaning, divide the predicted claim count by exposure). Use that as your sort order.
Reply With Quote
  #13  
Old 04-11-2019, 03:54 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: As far as 3 cups of sugar will take you
Studying for ACAS
College: Hard Knocks
Favorite beer: Most German dark lagers
Posts: 63,870
Default

You might also look at 50 and 75 buckets.

Looks better regarding the crunched residuals.

Seeing a trend in the residual graph usually means that something could be done better or that one of your assumptions (e.g., Poisson distributed) is off.

I agree with the suggestion to use the "quasiPoisson" option instead since it's very likely that your (true) variance is larger than the mean. That could "fix" the observed trend you're seeing with the crunched residuals with 100 buckets.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #14  
Old 04-11-2019, 03:54 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 134
Default

Quote:
Originally Posted by MoralHazard View Post
You should first be converting the claim count prediction to a frequency prediction (meaning, divide the predicted claim count by exposure). Use that as your sort order.
Ah that makes sense. So for Severity, my sort order should be Inc/Count

and Pure Premium, the sort order is just predicted since predictions are already pure premium?
Reply With Quote
  #15  
Old 04-11-2019, 04:00 PM
MoralHazard MoralHazard is offline
Member
CAS
 
Join Date: Jul 2011
Favorite beer: Sam Adams Rebel Rouser
Posts: 110
Default

Quote:
Originally Posted by Actuarially Me View Post
Ah that makes sense. So for Severity, my sort order should be Inc/Count

and Pure Premium, the sort order is just predicted since predictions are already pure premium?
Yes.
Reply With Quote
  #16  
Old 04-11-2019, 04:02 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 134
Default

Quote:
Originally Posted by Vorian Atreides View Post
You might also look at 50 and 75 buckets.

Looks better regarding the crunched residuals.

Seeing a trend in the residual graph usually means that something could be done better or that one of your assumptions (e.g., Poisson distributed) is off.

I agree with the suggestion to use the "quasiPoisson" option instead since it's very likely that your (true) variance is larger than the mean. That could "fix" the observed trend you're seeing with the crunched residuals with 100 buckets.
Thanks for all the help! I've read the Monograph and Practitioners guide a few times, but only some of it sticks each time and I forget it by the time I get to modeling it.


I found a formula that tests for overdispersion and it does show that my data is slightly overdispersed.

I also found a function influencePlot from the car package that shows what points are highly leveraging the model. Can investigate removing some of those.

I also tested a zeroinfl model and it is a better fit according to AIC.

As moralHazard pointed out, I'm probably calculating frequency gini incorrectly cause I forgot to divide by exposure
Reply With Quote
  #17  
Old 04-11-2019, 04:05 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: As far as 3 cups of sugar will take you
Studying for ACAS
College: Hard Knocks
Favorite beer: Most German dark lagers
Posts: 63,870
Default

gini is good for making a decision regarding if one model "does a better job" than another wrt discriminating differences in results (e.g., identifying higher premium/freq/sev obs from lower ones).

Residuals are good for making a determination of whether or not the underlying assumptions of the model are appropriate. I've seen situations where "odd" results from a residual plot actually indicated that there the assumption that "all data belong to the same population" was incorrect and we needed to either segregate the data or introduce another variable to recognize the differences. You won't get that from the gini.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #18  
Old 04-11-2019, 04:08 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 134
Default

Quote:
Originally Posted by MoralHazard View Post
Yes.
Hey, what do you know. That fix made it better! And now when I do the intercept only, the Gini is practically 0. You're a lifesaver!
Reply With Quote
  #19  
Old 04-11-2019, 04:18 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 134
Default

Quote:
Originally Posted by Vorian Atreides View Post
gini is good for making a decision regarding if one model "does a better job" than another wrt discriminating differences in results (e.g., identifying higher premium/freq/sev obs from lower ones).

Residuals are good for making a determination of whether or not the underlying assumptions of the model are appropriate. I've seen situations where "odd" results from a residual plot actually indicated that there the assumption that "all data belong to the same population" was incorrect and we needed to either segregate the data or introduce another variable to recognize the differences. You won't get that from the gini.
Interesting. Isn't that kind of the argument for splitting models by peril or territory?
Reply With Quote
  #20  
Old 04-12-2019, 01:56 AM
bjc2142's Avatar
bjc2142 bjc2142 is offline
Member
CAS
 
Join Date: Aug 2014
Location: Boston
Studying for Exam 9
Favorite beer: Asahi Draft
Posts: 1,104
Default

By peril model was difficult. I think Fire peril did pretty well but not the others..maybe severe storm did okay since zip code was huge factor.
__________________
ACAS | 7 | 8 | 9
Reply With Quote
Reply

Tags
glm, peril, pure premium

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 05:08 AM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.26627 seconds with 9 queries