Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Property - Casualty / General Insurance
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions

ACTUARIAL SALARY SURVEYS
Contact DW Simpson for a Personalized Salary Survey

Reply
 
Thread Tools Search this Thread Display Modes
  #11  
Old 02-20-2019, 11:33 AM
itGetsBetter itGetsBetter is offline
Member
CAS AAA
 
Join Date: Feb 2016
Location: Midwest
Favorite beer: Spruce Springsteen
Posts: 270
Default

Quote:
Originally Posted by Actuarially Me View Post
There's some things that can be scraped from websites, but accuracy is only as good as the website.
From a pricing perspective, scraping data from websites is often either illegal, in violation of site terms of service, or not approved by regulators.
Reply With Quote
  #12  
Old 02-20-2019, 11:36 AM
itGetsBetter itGetsBetter is offline
Member
CAS AAA
 
Join Date: Feb 2016
Location: Midwest
Favorite beer: Spruce Springsteen
Posts: 270
Default

Quote:
Originally Posted by ALivelySedative View Post
Yup. Been that way for a couple decades at least. I've convinced my actuarial superiors that this is true, but I haven't convinced anyone to actually do anything about it from the IT side. Adverse selection is rampant!
I empathize with you. Some ideas:
- Talk with data governance team.
- Explain to IT that requests for data will decrease if they just give you all the data up front so you can calculate transformations and bins yourself, as needed.
- Bring a vendor or consultant in to do part or all of the change to the systems and data collection process.
- Provide Cost Benefit Analysis to upper management.
Reply With Quote
  #13  
Old 02-20-2019, 11:56 AM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 191
Default

Quote:
Originally Posted by itGetsBetter View Post
From a pricing perspective, scraping data from websites is often either illegal, in violation of site terms of service, or not approved by regulators.
Good to know! Does that vary by line of business or personal vs commercial?

Makes sense from a personal lines perspective, but I work in commercial lines. For instance, property line data vendors have image detection machine learning algorithm to determine roof type from OpenMaps. Also scraping Forbes 500 and Wikipedia to get some company info.
Reply With Quote
  #14  
Old 02-20-2019, 12:05 PM
itGetsBetter itGetsBetter is offline
Member
CAS AAA
 
Join Date: Feb 2016
Location: Midwest
Favorite beer: Spruce Springsteen
Posts: 270
Default

Quote:
Originally Posted by Actuarially Me View Post
Good to know! Does that vary by line of business or personal vs commercial?

Makes sense from a personal lines perspective, but I work in commercial lines. For instance, property line data vendors have image detection machine learning algorithm to determine roof type from OpenMaps. Also scraping Forbes 500 and Wikipedia to get some company info.
No, I do not think this varies by line. If a vendor does it, they probably got permission from the site. If you do it, I am assuming that you did not get permission from the site or the regulator.
Reply With Quote
  #15  
Old 02-20-2019, 12:18 PM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 191
Default

Quote:
Originally Posted by itGetsBetter View Post
No, I do not think this varies by line. If a vendor does it, they probably got permission from the site. If you do it, I am assuming that you did not get permission from the site or the regulator.
True. I haven't done any scraping here. A lot of the vendors I saw used scraping from some sites to triangulate accuracy. Maybe those are sites that have publicly available APIs

Can't say I followed that advice when pulling fantasy football data lol.
Reply With Quote
  #16  
Old 02-20-2019, 01:54 PM
itGetsBetter itGetsBetter is offline
Member
CAS AAA
 
Join Date: Feb 2016
Location: Midwest
Favorite beer: Spruce Springsteen
Posts: 270
Default

Quote:
Originally Posted by Actuarially Me View Post
True. I haven't done any scraping here. A lot of the vendors I saw used scraping from some sites to triangulate accuracy. Maybe those are sites that have publicly available APIs

Can't say I followed that advice when pulling fantasy football data lol.
Insurance companies make a lot more than fantasy footballers so I'm guessing you're not going to get in trouble.
Reply With Quote
  #17  
Old 02-20-2019, 03:27 PM
bjc2142's Avatar
bjc2142 bjc2142 is offline
Member
CAS
 
Join Date: Aug 2014
Location: Boston
Studying for Python
Favorite beer: Asahi Draft
Posts: 1,191
Default

Thanks for sharing this. I should refer back to this once I'm done with May exam.
__________________
FCAS - Hawaii 2019
Reply With Quote
  #18  
Old 02-22-2019, 06:59 AM
oirg oirg is offline
Member
CAS
 
Join Date: Oct 2003
Posts: 8,589
Default

Note that if you model frequency and severity separately, you make the assumption that they are independent when you combine results by multiplying. This assumption is not necessary when you model pure premium directly.
Reply With Quote
  #19  
Old 02-22-2019, 09:13 AM
Actuarially Me Actuarially Me is offline
Member
CAS
 
Join Date: Jun 2013
Posts: 191
Default

Quote:
Originally Posted by oirg View Post
Note that if you model frequency and severity separately, you make the assumption that they are independent when you combine results by multiplying. This assumption is not necessary when you model pure premium directly.
Very true. Leads me to some questions for anyone:

For those with experience in building pure premium models, what structure do you use?
Separate Frequency and Severity
Single Pure Premium model
Hurdle

What distributions have worked best for your data?
Reply With Quote
  #20  
Old 04-08-2019, 01:02 PM
Death by Exam Death by Exam is offline
CAS
 
Join Date: Feb 2019
Posts: 7
Default

Thank you for putting all this together! Bumping so I can more easily find the post to continue reading in the future.
__________________
P, FM, IFM, C, MAS-I
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 08:53 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.44382 seconds with 9 queries