Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Software & Technology
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions



Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 11-04-2018, 12:36 PM
bigalxyz bigalxyz is offline
 
Join Date: Aug 2013
Posts: 10
Default Alternatives to Emblem/Earnix/SAS, etc.

I've used Emblem on & off for almost 20 years. My current work situation, however, means Emblem probably won't be available to me - nor will any other proprietary software such as Earnix, SAS, etc.

I will be doing some premium rating work soon and I'd like to base it on GLM models. I know my way around R, which of course can fit GLMs. The "Microsoft R Client" version seems to have access to the RevoScaleR package (previously sold by Revolution Analytics) which compared to the default fitting algorithms is claimed to be faster & run with bigger data sets, etc. - so that's a useful step forward and the bones of a solution.

However...where Emblem still scores over R is of course:
* Customisation to P&C applications
* Nice friendly GUI instead of command line

Hard to believe that nobody is trying to develop open source bolt-ons to R to try to address these issues, but it's proving hard to find out anything specific. Does anyone know about this stuff? Keen to get involved.

Thanks
A
Reply With Quote
  #2  
Old 11-04-2018, 01:23 PM
kevinykuo kevinykuo is offline
CAS
 
Join Date: Nov 2017
Posts: 22
Default

Could you elaborate on the "Customisation to P&C applications" bit? I'm sure if there's interest we have enough R + actuarial jobs around these parts to throw an open source package together.

Regarding the second point, one issue with doing modeling in a GUI is that it isn't as reproducible and auditable. Code can be checked into source control and is more transparent.

Also, if you're running into speed/data size issues, look into sparklyr (assuming you have access to Spark) and h2o, both of which are open source packages with GLM functionality.
Reply With Quote
  #3  
Old 11-04-2018, 01:31 PM
bigalxyz bigalxyz is offline
 
Join Date: Aug 2013
Posts: 10
Default

Quote:
Originally Posted by kevinykuo View Post
Could you elaborate on the "Customisation to P&C applications" bit? I'm sure if there's interest we have enough R + actuarial jobs around these parts to throw an open source package together.

Regarding the second point, one issue with doing modeling in a GUI is that it isn't as reproducible and auditable. Code can be checked into source control and is more transparent.

Also, if you're running into speed/data size issues, look into sparklyr (assuming you have access to Spark) and h2o, both of which are open source packages with GLM functionality.
Thanks. Re: customisation - I'm talking about Emblem features such as:
* joining exposure data to claims data and chopping up into time periods (in File Converter)
* the validation stages within the File Converter
* the Model Combiner feature

All of this stuff is perfectly do-able in other environments of course, but with less ease.

Point taken about audit trails - and yes command line work scores well on that front.

I'm not aware of sparklyr or h2o - I will investigate...

A
Reply With Quote
  #4  
Old 11-04-2018, 02:00 PM
kevinykuo kevinykuo is offline
CAS
 
Join Date: Nov 2017
Posts: 22
Default

The first point might just be a couple lines of dplyr code, but I'm not familiar with what File Converter does. What are "validation stages" and "model combiner"?
Reply With Quote
  #5  
Old 11-05-2018, 05:27 AM
bigalxyz bigalxyz is offline
 
Join Date: Aug 2013
Posts: 10
Default

The Emblem software has a few components - eg.
* File Converter
* Modeller
* Model Combiner

Used in that order.

File Converter is used to import data (from common db files, or csv, etc.), assign significances to certain fields ("this field is our exposure measure", "this field is our claim cost for claim type X", etc.), matches claims back to chunks of exposure in respect of individual policies, allows rating factors to be banded, or put in ranges, etc.

The Modeller is the "core" of the package - this is where GLMs are fitted.

The Model Combiner is used as a final stage for situations where a total risk premium needs to be created from several different models - for example if the claims data is analysed as a number of frequency and severity models, these need to be multiplied together and then added (RP = f1.s1 + f2.s2 + f3.s3 +...). Also allows adjustments to be made to allow for future claims inflation, IBNR loadings, and so on.

The File Converter and Model Combiner stuff could be done without too much trouble in something like MS Access (or, I would think, MS SQL/Server for bigger files). No doubt there are other ways too eg. I imagine it could be done in R as well.

Most of the effort is in using the Modeller to arrive at the "best" GLM for a particular data set. Emblem makes this easier with its GUI - quick to add factors or remove factors from models, band them, introduce factor interactions, use offsetting, etc. and lots of model diagnostics (to help with selection) in a nice graphical form - chi-squared and F statistics for nested models, AIC, BIC, standard errors of parameter estimates, all that stuff. It's this aspect of Emblem that I will miss the most I think. AFAIK all of the same underlying model info is readily available in R, but it's not in such an easily digestible layout.

My question I suppose is whether anyone knows of any "widgets" for R which can help plug this gap. Part of me thinks I'm just being lazy! Roll my sleeves up and get stuck into some old skool command line fun...
Reply With Quote
  #6  
Old 11-06-2018, 05:27 PM
kevinykuo kevinykuo is offline
CAS
 
Join Date: Nov 2017
Posts: 22
Default

I think all of the functionality you mentioned is available in R, more or less off the shelf in standard packages. It may be the GUI to code transition that takes some getting used to.

Is there a tutorial/textbook somewhere that covers the end-to-end process? E.g. https://www.casact.org/pubs/monograp...hare-Tevet.pdf covers a lot of technical modeling stuff but not much before/after. I think it would be helpful to get some realistic enough mocked up data and put some code together, but to do that I'll need help from folks who have done it a few times IRL.
Reply With Quote
  #7  
Old 11-07-2018, 12:03 AM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is offline
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 47,996
Default

It's probably possible to recreate what Emblem can do in Shiny, although some engineering would be involved to get it to perform as fast. Financially, the cost to do so probably isn't worth the licensing fees you pay for Emblem...if it takes you any longer than two months to set up an alternative (not including support costs!) than it won't cut it financially.

I would love to do the whole thing in R though.
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #8  
Old 11-07-2018, 12:03 AM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is offline
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 47,996
Default

BTW Kevin, I'll be looking for you at the CAS Annual meeting. I'll send you a PM.
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #9  
Old 11-07-2018, 05:07 AM
bigalxyz bigalxyz is offline
 
Join Date: Aug 2013
Posts: 10
Default

Quote:
Originally Posted by kevinykuo View Post
I think all of the functionality you mentioned is available in R, more or less off the shelf in standard packages. It may be the GUI to code transition that takes some getting used to.

Is there a tutorial/textbook somewhere that covers the end-to-end process? E.g. https://www.casact.org/pubs/monograp...hare-Tevet.pdf covers a lot of technical modeling stuff but not much before/after. I think it would be helpful to get some realistic enough mocked up data and put some code together, but to do that I'll need help from folks who have done it a few times IRL.
Thanks. The paper looks interesting so I've downloaded it & will read. I'm an experienced practitioner with this stuff, but it's good to revise my knowledge & perhaps learn a new trick or two along the way. I hadn't seen this paper before although I'm familiar with an earlier CAS paper:

https://www.casact.org/pubs/dpp/dpp04/04dpp1.pdf

On the "before & after" processing...I'm not aware of any detailed documentation. In truth it's not hugely complicated though. I have an old sample data set (exposure and claims data) which I have spent some time prepping this week - working out time intervals, chopping exposure periods up into chunks, attaching claim records to the relevant row in the exposure table, etc. and it's actually fairly straightforward to do in MS Access with a few bits of SQL code (MS Access wouldn't cope with really big data sets but for the sample data I have it works ok). It has taken me a while to do because (a) I've never done it before (I've usually had Emblem to do it for me in the past so I've got lazy) and (b) my SQL skills aren't too hot. But I've done it without losing too much of what remains of my hair.

Gonna try building some GLMs in R shortly and see how I get on without Emblem's GUI widgets, how much I miss them, etc. I'm treating this as a dry run to see how it goes before (hopefully) moving on to a live project.

Last edited by bigalxyz; 11-07-2018 at 06:35 AM..
Reply With Quote
  #10  
Old 11-16-2018, 08:25 PM
kevinykuo kevinykuo is offline
CAS
 
Join Date: Nov 2017
Posts: 22
Default

@bigalxyz let me know if you'd like to collaborate in putting a tutorial together. I'm pretty sure the whole thing can be done in R without having to write SQL or mess with Access.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 03:36 AM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.17182 seconds with 11 queries