Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Property - Casualty / General Insurance
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


Not looking for a job? Tell us about your ideal job,
and we'll only contact you when it opens up.
https://www.dwsimpson.com/register


Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 06-12-2018, 11:34 PM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,135
Default Retention model performance

For those who have designed a binary retention model before, what kind of AUC (or any other measurement) should be considered respectable? I'm sure there will be plenty of variation by company and line of business, but any general targets would be helpful in determining if I'm heading in the right direction. Thanks
__________________
ACAS 7 8 9
Reply With Quote
  #2  
Old 06-12-2018, 11:53 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,046
Default

What's the AUC of your current model?
Reply With Quote
  #3  
Old 06-13-2018, 12:05 AM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,135
Default

Quote:
Originally Posted by Marcie View Post
What's the AUC of your current model?
Currently ~0.74
__________________
ACAS 7 8 9
Reply With Quote
  #4  
Old 06-13-2018, 12:12 AM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,046
Default

Quote:
Originally Posted by Tacoactuary View Post
Currently ~0.74
A respectable AUC of a new model is > 0.74.
Reply With Quote
  #5  
Old 06-13-2018, 02:27 AM
tommie frazier tommie frazier is offline
Member
 
Join Date: Aug 2003
Favorite beer: The kind with 2 e's
Posts: 22,950
Default

Quote:
Originally Posted by Marcie View Post
A respectable AUC of a new model is > 0.74.
well done
__________________
Removed a dated athletic reference under pressure from a friend. You can still give money to help fund research on neurofibromatosis (nf).

General info at www.ctf.org

Team donation page here.
Reply With Quote
  #6  
Old 06-22-2018, 03:58 AM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is offline
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 48,121
Default

Quote:
Originally Posted by Marcie View Post
A respectable AUC of a new model is > 0.74.
You gotta factor in the salary of the modeler and the cost of implementation
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #7  
Old 06-22-2018, 04:36 AM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,046
Default

Quote:
Originally Posted by Colonel Smoothie View Post
You gotta factor in the salary of the modeler and the cost of implementation
Ok, then:
Quote:
Originally Posted by Marcie View Post
A respectable AUC of a new model is > 0.74.
.
Reply With Quote
  #8  
Old 06-13-2018, 11:00 AM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: As far as 3 cups of sugar will take you
Studying for ACAS
College: Hard Knocks
Favorite beer: Most German dark lagers
Posts: 62,817
Default

One might also argue that an AUC in the neighborhood of the current model could also be "reasonable" (or respectable) since there might be other benefits of the new model (e.g., greater interpretability) that is not captured by AUC.

Put another way, don't put all of your proverbial eggs in one basket.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #9  
Old 06-13-2018, 12:33 PM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,135
Default

Quote:
Originally Posted by Vorian Atreides View Post
One might also argue that an AUC in the neighborhood of the current model could also be "reasonable" (or respectable) since there might be other benefits of the new model (e.g., greater interpretability) that is not captured by AUC.

Put another way, don't put all of your proverbial eggs in one basket.
Well the previous model had AUC ~ 0.71, so at least it is an improvement in that regard.
__________________
ACAS 7 8 9
Reply With Quote
  #10  
Old 06-13-2018, 02:48 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: As far as 3 cups of sugar will take you
Studying for ACAS
College: Hard Knocks
Favorite beer: Most German dark lagers
Posts: 62,817
Default

Quote:
Originally Posted by Tacoactuary View Post
Well the previous model had AUC ~ 0.71, so at least it is an improvement in that regard.
A significant improvement, though?

Improvement in one measure isn't enough to say that one model is "better" (overall) than another. It can be evidence to support such a claim, but isn't enough by itself.

That would be like saying that the taller person is a better basketball player.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 02:59 AM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.20692 seconds with 9 queries