Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Careers - Employment
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions



Reply
 
Thread Tools Display Modes
  #1  
Old 01-27-2018, 01:51 PM
JMO's Avatar
JMO JMO is offline
Carol Marler
Non-Actuary
 
Join Date: Sep 2001
Location: Back home again in Indiana
Studying for Nothing actuarial.
Posts: 36,773
Default Bad news for AI and Data Scientists

Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
The study is here: https://www.documentcloud.org/docume...Algorithm.html

News articles about it are ubiquitous: https://www.google.com/search?hl=en&...pe=&as_rights=

Based on what I have read, there has been ongoing criticism of a certain proprietary model to predict recidivism, used by judges for setting bail and deciding on sentences. The latest study compared that algorithm with human judgement by means of "mechanical turk" (i.e., online people recruited to do various tasks). The also used a much simpler algorithm involving only age and prior convictions.

When the researchers looked at predictions from two years ago and the results since then, the proprietary algorithm was right about 2/3 of the time. So was the mechanical turk. And so was the far simpler algorithm.
__________________
Carol Marler, "Just My Opinion"

Pluto is no longer a planet and I am no longer an actuary. Please take my opinions as non-actuarial.


My latest favorite quotes, updated Feb 15, 2018. Hmmm. It's been quite a while.

Spoiler:
I should keep these four permanently.
Quote:
Originally Posted by rekrap View Post
JMO is right
Quote:
Originally Posted by campbell View Post
I agree with JMO.
Quote:
Originally Posted by Westley View Post
And def agree w/ JMO.
Quote:
Originally Posted by MG View Post
This. And everything else JMO wrote.
And this all purpose permanent quote:
Quote:
Originally Posted by Dr T Non-Fan View Post
Yup, it is always someone else's fault.
MORE:
Quote:
Originally Posted by PeppermintPatty View Post
But sometimes you get someone who has that charming mix of "don't care" and "don't get it", and there's just not much you can do.
Quote:
Originally Posted by Bro View Post
I recommend you get perspective.
Quote:
Originally Posted by Enough Exams Already View Post
Dude, you can't fail a personality test. It just isn't that kind of test.

Last edited by JMO; 01-27-2018 at 02:01 PM..
Reply With Quote
  #2  
Old 01-27-2018, 02:44 PM
Woodrow's Avatar
Woodrow Woodrow is offline
Member
SOA
 
Join Date: Oct 2002
Posts: 6,056
Default

Quote:
Originally Posted by JMO View Post
Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
Maybe it depends on the topic, but for a lot of AI, being "as good as untrained people" would be an amazing success.

Think about AI tasks like "recognize a picture of a cat"

Last edited by Woodrow; 01-27-2018 at 04:46 PM..
Reply With Quote
  #3  
Old 01-27-2018, 03:06 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 28,401
Default

This really isn't bad news for AI/DS. Or even news. Complex models are not necessarily better than simple ones.

This is bad news for people who buy complex models without the requisite knowledge to evaluate what they are buying. Even that is overstating it. It is only bad news for ignorant buyers of overly complex models who may be held responsible for their bad decisions. I am guessing that the bureaucrats who rolled out this sentencing abomination won't be held to account.
Reply With Quote
  #4  
Old 01-27-2018, 03:09 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 28,401
Default

If anything, the abundance of ignorant buyers of overly complicated models is GOOD news for AI/DS practitioners.
Reply With Quote
  #5  
Old 01-27-2018, 03:23 PM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is online now
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 45,896
Default

Surely you remember this, OP:

https://en.wikipedia.org/wiki/AI_winter
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #6  
Old 01-27-2018, 03:30 PM
jas66Kent's Avatar
jas66Kent jas66Kent is offline
Member
SOA
 
Join Date: May 2012
Location: London
Favorite beer: Corona :)
Posts: 21,813
Default

There is no real substitute for human judgment. That will always be the case.
__________________
Quote:
“Stupidity is a more dangerous enemy of the good than malice. Against stupidity we are defenseless. Neither protests nor the use of force accomplish anything here; reasons fall on deaf ears; facts that contradict one’s prejudgment simply need not be believed – in such moments the stupid person even becomes critical – and when facts are irrefutable they are just pushed aside as inconsequential, as incidental. For that reason, greater caution is called for when dealing with a stupid person than with a malicious one".
Reply With Quote
  #7  
Old 01-27-2018, 03:31 PM
Colonel Smoothie's Avatar
Colonel Smoothie Colonel Smoothie is online now
Member
CAS
 
Join Date: Sep 2010
College: Jamba Juice University
Favorite beer: AO Amber Ale
Posts: 45,896
Default

Quote:
Originally Posted by jas66Kent View Post
There is no real substitute for human judgment. That will always be the case.
What if we encounter an alien species with better judgment?
__________________
Recommended Readings for the EL Actuary || Recommended Readings for the EB Actuary

Quote:
Originally Posted by Wigmeister General View Post
Don't you even think about sending me your resume. I'll turn it into an origami boulder and return it to you.
Reply With Quote
  #8  
Old 01-27-2018, 03:32 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 28,401
Default

Quote:
Originally Posted by Colonel Smoothie View Post
What if we encounter an alien species with better judgment?
You mean like jas?
Reply With Quote
  #9  
Old 01-27-2018, 03:46 PM
JohnLocke's Avatar
JohnLocke JohnLocke is offline
Member
SOA
 
Join Date: Mar 2007
Posts: 16,102
Default

The fact that needlessly complex solutions to simpler problems can be generated does not imply that there aren't more complex problems in need of more complex models.
__________________
i always post when i'm in a shitty mood. if i didn't do that, i'd so rarely post. --AO Fan

Lucky for you I was raised by people with a good moral center because if that were not the case, you guys would be in a lot of trouble.
So be very, very glad people like me exist. Your future basically depends on it. --jas66kent

The stock market is going to go up significantly due to Trump Economics --jas66kent
Reply With Quote
  #10  
Old 01-27-2018, 04:20 PM
AbedNadir AbedNadir is offline
Member
CAS SOA
 
Join Date: Mar 2014
Studying for enlightenment
Posts: 1,848
Default

Quote:
Originally Posted by JMO View Post
Apparently, complex algorithms are not necessarily better than simpler ones. Or even untrained people.
The study is here: https://www.documentcloud.org/docume...Algorithm.html

News articles about it are ubiquitous: https://www.google.com/search?hl=en&...pe=&as_rights=

Based on what I have read, there has been ongoing criticism of a certain proprietary model to predict recidivism, used by judges for setting bail and deciding on sentences. The latest study compared that algorithm with human judgement by means of "mechanical turk" (i.e., online people recruited to do various tasks). The also used a much simpler algorithm involving only age and prior convictions.

When the researchers looked at predictions from two years ago and the results since then, the proprietary algorithm was right about 2/3 of the time. So was the mechanical turk. And so was the far simpler algorithm.
I'm surprised an actuary thinks you can generalize from this legal example to AI/Data Science being on the decline or not all it's cracked up to be. Really, it's just an example of people misusing it and not using a simple model when they ought to because they just want to be fancy.

Algorithms, in the right hands, are used when they need to be, when simple ones fail to capture the complexity of the relationships involved and thus necessitate the use of more flexible and complex models.
__________________


Quote:
Originally Posted by Colonel Smoothie View Post
Maybe, if there were some magical 4th dimension where they did their work in hypercubicles. Otherwise, I don't see them.
Reply With Quote
Reply

Tags
data science, predictive analytics

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 02:37 AM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.19858 seconds with 9 queries