Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Property - Casualty / General Insurance
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 06-12-2018, 10:34 PM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,123
Default Retention model performance

For those who have designed a binary retention model before, what kind of AUC (or any other measurement) should be considered respectable? I'm sure there will be plenty of variation by company and line of business, but any general targets would be helpful in determining if I'm heading in the right direction. Thanks
__________________
ACAS 7 8 9
Reply With Quote
  #2  
Old 06-12-2018, 10:53 PM
Marcie's Avatar
Marcie Marcie is online now
Member
CAS
 
Join Date: Feb 2015
Posts: 7,427
Default

What's the AUC of your current model?
Reply With Quote
  #3  
Old 06-12-2018, 11:05 PM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,123
Default

Quote:
Originally Posted by Marcie View Post
What's the AUC of your current model?
Currently ~0.74
__________________
ACAS 7 8 9
Reply With Quote
  #4  
Old 06-12-2018, 11:12 PM
Marcie's Avatar
Marcie Marcie is online now
Member
CAS
 
Join Date: Feb 2015
Posts: 7,427
Default

Quote:
Originally Posted by Tacoactuary View Post
Currently ~0.74
A respectable AUC of a new model is > 0.74.
Reply With Quote
  #5  
Old 06-13-2018, 01:27 AM
tommie frazier tommie frazier is offline
Member
 
Join Date: Aug 2003
Favorite beer: The kind with 2 e's
Posts: 22,678
Default

Quote:
Originally Posted by Marcie View Post
A respectable AUC of a new model is > 0.74.
well done
__________________
Removed a dated athletic reference under pressure from a friend. You can still give money to help fund research on neurofibromatosis (nf).

General info at www.ctf.org

Team donation page here.
Reply With Quote
  #6  
Old 06-13-2018, 10:00 AM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: Hitler's Secret Bunker
Studying for ACAS
College: Hard Knocks
Favorite beer: Sam Adams Cherry Wheat
Posts: 61,775
Default

One might also argue that an AUC in the neighborhood of the current model could also be "reasonable" (or respectable) since there might be other benefits of the new model (e.g., greater interpretability) that is not captured by AUC.

Put another way, don't put all of your proverbial eggs in one basket.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #7  
Old 06-13-2018, 11:33 AM
Tacoactuary's Avatar
Tacoactuary Tacoactuary is offline
Member
CAS Non-Actuary
 
Join Date: Nov 2014
Location: Des Moines, IA
College: Vanderbilt, UIUC
Favorite beer: Yazoo Sue
Posts: 1,123
Default

Quote:
Originally Posted by Vorian Atreides View Post
One might also argue that an AUC in the neighborhood of the current model could also be "reasonable" (or respectable) since there might be other benefits of the new model (e.g., greater interpretability) that is not captured by AUC.

Put another way, don't put all of your proverbial eggs in one basket.
Well the previous model had AUC ~ 0.71, so at least it is an improvement in that regard.
__________________
ACAS 7 8 9
Reply With Quote
  #8  
Old 06-13-2018, 01:48 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: Hitler's Secret Bunker
Studying for ACAS
College: Hard Knocks
Favorite beer: Sam Adams Cherry Wheat
Posts: 61,775
Default

Quote:
Originally Posted by Tacoactuary View Post
Well the previous model had AUC ~ 0.71, so at least it is an improvement in that regard.
A significant improvement, though?

Improvement in one measure isn't enough to say that one model is "better" (overall) than another. It can be evidence to support such a claim, but isn't enough by itself.

That would be like saying that the taller person is a better basketball player.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 05:43 PM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.14161 seconds with 11 queries