Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > General Actuarial
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions

Browse Open Actuarial Jobs

Life  Health  Casualty  Pension  Entry Level  All Jobs  Salaries


General Actuarial Non-Specific Actuarial Topics - Before posting a thread, please browse over our other sections to see if there is a better fit, such as Careers - Employment, Actuarial Science Universities Forum or any of our other 100+ forums.

Reply
 
Thread Tools Search this Thread Display Modes
  #11  
Old 07-02-2018, 12:34 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: Hitler's Secret Bunker
Studying for ACAS
College: Hard Knocks
Favorite beer: Sam Adams Cherry Wheat
Posts: 61,992
Default

Quote:
Originally Posted by CuriousGeorge View Post
Whether you expect future data to also have significant missing fields may also be a consideration.


But very important to separate out "default" type missing values and true missing values, IMO.

Not that this will impact the resulting model so much as having a better understanding of how to interpret the model.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #12  
Old 07-02-2018, 01:26 PM
whoanonstop's Avatar
whoanonstop whoanonstop is online now
Member
Non-Actuary
 
Join Date: Aug 2013
Location: Los Angeles, CA
Studying for Spark / Scala
College: College of William and Mary
Favorite beer: Orange Juice
Posts: 5,812
Blog Entries: 1
Default

Lots of people responding that don't seem to have any sound advice beyond "maybe your data isn't really missing"?????

Except for this:

Quote:
Originally Posted by AMedActuary View Post
It would depend on why the data is missing. If it's missing at random, it should be okay to just remove. If there are certain types of people who have more missing data than others, then it may have an effect on your results. See the below page on Wikipedia for a brief intro to missing data.

https://en.wikipedia.org/wiki/Missing_data
OP. For each scenario of missing data there will be different answers. Read through the Wiki above to hone in on which type of "missingness" you're dealing with.

Also, if you're working with linear models a lot, just buy "Regression Modeling Strategies" by Harrell. It has a chapter on missing data that would be of great benefit to you. Of course, the book is valuable beyond that chapter.

Quote:
Originally Posted by kevinykuo View Post
Do both and see which does better on validation? Do you expect new data to be scored to be missing a bunch of variables? Do you need to use GLM? Hard to say without knowing what problem you're solving, but these are some things to think about.
After checking the first review, this might be a good book for you as well:

"If you want to move past the "just use cross validation" stage of your ML work and improve your model's generalization (and understand why and when to use techniques like bootstrapping) this is the book for you."

-Riley
__________________
Reply With Quote
  #13  
Old 07-02-2018, 02:00 PM
Vorian Atreides's Avatar
Vorian Atreides Vorian Atreides is offline
Wiki/Note Contributor
CAS
 
Join Date: Apr 2005
Location: Hitler's Secret Bunker
Studying for ACAS
College: Hard Knocks
Favorite beer: Sam Adams Cherry Wheat
Posts: 61,992
Default

Quote:
Originally Posted by whoanonstop View Post
Lots of people responding that don't seem to have any sound advice beyond "maybe your data isn't really missing"?????
You make it sound like it's a pointless exercise to identify what's really missing.

Or maybe I'm reading your post wrong.
__________________
I find your lack of faith disturbing

Why should I worry about dying? Itís not going to happen in my lifetime!


Freedom of speech is not a license to discourtesy

#BLACKMATTERLIVES
Reply With Quote
  #14  
Old 07-05-2018, 02:53 AM
whoanonstop's Avatar
whoanonstop whoanonstop is online now
Member
Non-Actuary
 
Join Date: Aug 2013
Location: Los Angeles, CA
Studying for Spark / Scala
College: College of William and Mary
Favorite beer: Orange Juice
Posts: 5,812
Blog Entries: 1
Default

Quote:
Originally Posted by Vorian Atreides View Post
You make it sound like it's a pointless exercise to identify what's really missing.

Or maybe I'm reading your post wrong.
I was making an assumption that the OP knows how to create a dataset and that the data is actually missing. Given he has passed a number of actuarial exams, that is a safe assumption, no? I just find "Well maybe the data isn't actually missing!" responses similar to that of an IT person telling one to turn the unit off and on again.

At least for me, I was hoping that the conversation would have laid out a few tactics on what to do when the data is actually missing. I've experienced a few scenarios in previous positions where some crazy approaches were used to fix missing values (lol, just fill them with zeros, anyone?) and I wanted to see if anyone would suggest anything by the book here.

I was disappointed is all.

-Riley
__________________
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 12:21 PM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.25619 seconds with 9 queries