Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Careers - Employment
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


Upload your resume securely at https://www.dwsimpson.com
to be contacted when new jobs meet your skills and objectives.


Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 01-27-2018, 12:51 PM
JMO's Avatar
JMO JMO is offline
Carol Marler
Non-Actuary
 
Join Date: Sep 2001
Location: Back home again in Indiana
Studying for Nothing actuarial.
Posts: 37,085
Default Bad news for AI and Data Scientists

Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
The study is here: https://www.documentcloud.org/docume...Algorithm.html

News articles about it are ubiquitous: https://www.google.com/search?hl=en&...pe=&as_rights=

Based on what I have read, there has been ongoing criticism of a certain proprietary model to predict recidivism, used by judges for setting bail and deciding on sentences. The latest study compared that algorithm with human judgement by means of "mechanical turk" (i.e., online people recruited to do various tasks). The also used a much simpler algorithm involving only age and prior convictions.

When the researchers looked at predictions from two years ago and the results since then, the proprietary algorithm was right about 2/3 of the time. So was the mechanical turk. And so was the far simpler algorithm.
__________________
Carol Marler, "Just My Opinion"

Pluto is no longer a planet and I am no longer an actuary. Please take my opinions as non-actuarial.


My latest favorite quotes, updated Apr 5, 2018.

Spoiler:
I should keep these four permanently.
Quote:
Originally Posted by rekrap View Post
JMO is right
Quote:
Originally Posted by campbell View Post
I agree with JMO.
Quote:
Originally Posted by Westley View Post
And def agree w/ JMO.
Quote:
Originally Posted by MG View Post
This. And everything else JMO wrote.
And this all purpose permanent quote:
Quote:
Originally Posted by Dr T Non-Fan View Post
Yup, it is always someone else's fault.
MORE:
All purpose response for careers forum:
Quote:
Originally Posted by DoctorNo View Post
Depends upon the employer and the situation.
Quote:
Originally Posted by Sredni Vashtar View Post
I feel like ERM is 90% buzzwords, and that the underlying agenda is to make sure at least one of your Corporate Officers is not dumb.

Last edited by JMO; 01-27-2018 at 01:01 PM..
Reply With Quote
  #2  
Old 01-27-2018, 01:44 PM
Woodrow's Avatar
Woodrow Woodrow is offline
Member
SOA
 
Join Date: Oct 2002
Posts: 6,305
Default

Quote:
Originally Posted by JMO View Post
Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
Maybe it depends on the topic, but for a lot of AI, being "as good as untrained people" would be an amazing success.

Think about AI tasks like "recognize a picture of a cat"

Last edited by Woodrow; 01-27-2018 at 03:46 PM..
Reply With Quote
  #3  
Old 01-27-2018, 02:06 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 29,013
Default

This really isn't bad news for AI/DS. Or even news. Complex models are not necessarily better than simple ones.

This is bad news for people who buy complex models without the requisite knowledge to evaluate what they are buying. Even that is overstating it. It is only bad news for ignorant buyers of overly complex models who may be held responsible for their bad decisions. I am guessing that the bureaucrats who rolled out this sentencing abomination won't be held to account.
Reply With Quote
  #4  
Old 01-27-2018, 02:09 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 29,013
Default

If anything, the abundance of ignorant buyers of overly complicated models is GOOD news for AI/DS practitioners.
Reply With Quote
  #5  
Old 01-27-2018, 02:23 PM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is online now
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 46,968
Default

Surely you remember this, OP:

https://en.wikipedia.org/wiki/AI_winter
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #6  
Old 01-27-2018, 02:30 PM
jas66Kent's Avatar
jas66Kent jas66Kent is offline
Member
SOA
 
Join Date: May 2012
Location: London
Favorite beer: Corona :)
Posts: 22,369
Default

There is no real substitute for human judgment. That will always be the case.
__________________
Quote:

"On a reflection of Tyranny. To abandon facts is to abandon freedom. If nothing is true, then no one can criticize power because there is no basis upon which to do so. Post-truth is pre-fascism.
Reply With Quote
  #7  
Old 01-27-2018, 02:31 PM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is online now
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 46,968
Default

Quote:
Originally Posted by jas66Kent View Post
There is no real substitute for human judgment. That will always be the case.
What if we encounter an alien species with better judgment?
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #8  
Old 01-27-2018, 02:32 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 29,013
Default

Quote:
Originally Posted by Colonel Smoothie View Post
What if we encounter an alien species with better judgment?
You mean like jas?
Reply With Quote
  #9  
Old 01-27-2018, 02:46 PM
JohnLocke's Avatar
JohnLocke JohnLocke is offline
Member
SOA
 
Join Date: Mar 2007
Posts: 16,127
Default

The fact that needlessly complex solutions to simpler problems can be generated does not imply that there aren't more complex problems in need of more complex models.
__________________
i always post when i'm in a shitty mood. if i didn't do that, i'd so rarely post. --AO Fan

Lucky for you I was raised by people with a good moral center because if that were not the case, you guys would be in a lot of trouble.
So be very, very glad people like me exist. Your future basically depends on it. --jas66kent

The stock market is going to go up significantly due to Trump Economics --jas66kent
Reply With Quote
  #10  
Old 01-27-2018, 03:20 PM
AbedNadir's Avatar
AbedNadir AbedNadir is offline
Member
CAS SOA
 
Join Date: Mar 2014
Studying for enlightenment
Posts: 2,104
Default

Quote:
Originally Posted by JMO View Post
Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
The study is here: https://www.documentcloud.org/docume...Algorithm.html

News articles about it are ubiquitous: https://www.google.com/search?hl=en&...pe=&as_rights=

Based on what I have read, there has been ongoing criticism of a certain proprietary model to predict recidivism, used by judges for setting bail and deciding on sentences. The latest study compared that algorithm with human judgement by means of "mechanical turk" (i.e., online people recruited to do various tasks). The also used a much simpler algorithm involving only age and prior convictions.

When the researchers looked at predictions from two years ago and the results since then, the proprietary algorithm was right about 2/3 of the time. So was the mechanical turk. And so was the far simpler algorithm.
I'm surprised an actuary thinks you can generalize from this legal example to AI/Data Science being on the decline or not all it's cracked up to be. Really, it's just an example of people misusing it and not using a simple model when they ought to because they just want to be fancy.

Algorithms, in the right hands, are used when they need to be, when simple ones fail to capture the complexity of the relationships involved and thus necessitate the use of more flexible and complex models.
__________________
Reply With Quote
Reply

Tags
data science, predictive analytics

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 02:05 AM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.40173 seconds with 9 queries