Actuarial Outpost
 
Go Back   Actuarial Outpost > Exams - Please Limit Discussion to Exam-Related Topics > CAS > CAS Exams - General
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions


Fill in a brief DW Simpson Registration Form
to be contacted when new jobs meet your criteria.


Reply
 
Thread Tools Search this Thread Display Modes
  #31  
Old 12-27-2018, 02:39 PM
Transitive Tangerine's Avatar
Transitive Tangerine Transitive Tangerine is offline
Member
COPA
 
Join Date: Oct 2018
Studying for FAP (hehehe nice)
Posts: 1,659
Default

yup, less diversification benefits now when the correlation coefficients among a great proportion of the exam has increased
Reply With Quote
  #32  
Old 12-27-2018, 03:40 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,774
Default

Quote:
Originally Posted by Mud View Post
Agreed. I have long advocated for the CAS to just write a straight-up exam, no tricks, see what happens. If that results in a high MQC score, fine; if that means 80% of the candidates show proficiency and should pass, fine. (Quit laughing, the CAS has said it could happen!) Candidates are always going to screw up and make mistakes, there's no need to throw more stuff into questions or try to be cute with questions to trip up candidates.

I liked the idea of writing a question that combines a couple of papers so that candidates understand how they tie together. Those who do understand the material will be able to do this pretty easily; those who don't understand the material will struggle. I've also long advocated for having those writing questions learn how to write those questions such that it's pretty easy to pick out the candidates who know the material from those who don't. That's probably a different discussion, though. The goal of the exams is supposed to be to distinguish those who know the material from those who don't, and only pass those who clearly know the material. ("Clearly" is a term subject to interpretation.)

Throwing a bunch of papers into a mass question and lobbing it out, though? That's overkill; it's complexity for the sheer sake of trying to be overly difficult. If it's a great idea, just have one massive question that hits every paper on the syllabus. It's like the ongoing move to fewer points on the exam; if that's really such a great tool for splitting the candidate pool, just have one (1) question for like five (5) points. (Which is clearly an absurdity, but underscores that the tighter you group potential candidate scores, the less variation you potentially have and the harder it is to split the pool accurately.)
I pretty much agree with all of this. As someone who has written questions, I never tried to be tricky with them.

I also like the idea of IQs but I've struggled enough with writing quality non-IQs that I can only imagine how difficult getting a good IQ can be. Writing good questions -- whether IQs or non-IQs -- that do a good job of distinguishing the "Minimally Qualified" from the "Maximally Unqualified" is a much trickier task than you might imagine.

I also disagree with the ongoing trend of fewer points. I don't think that's on purpose by the exam committee -- at least, I haven't heard of anything that fewer points is a goal.


ETA: I hope this goes without saying, but I feel the need to add: these are solely my opinions; I do not speak for the exam committee nor do my opinions reflect the opinions of the exam committee.

Last edited by Marcie; 12-27-2018 at 03:43 PM.. Reason: disclaimer
Reply With Quote
  #33  
Old 12-27-2018, 03:43 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,774
Default

Quote:
Originally Posted by PeppermintPatty View Post
I would make the exam shorter, and make it more total points (so more points per amount of work). Making the time longer just allows more test-creep.
Reply With Quote
  #34  
Old 12-27-2018, 03:55 PM
act_123 act_123 is offline
Member
CAS
 
Join Date: Dec 2013
Posts: 4,899
Default

Quote:
Originally Posted by Marcie View Post
I pretty much agree with all of this. As someone who has written questions, I never tried to be tricky with them.

I also like the idea of IQs but I've struggled enough with writing quality non-IQs that I can only imagine how difficult getting a good IQ can be. Writing good questions -- whether IQs or non-IQs -- that do a good job of distinguishing the "Minimally Qualified" from the "Maximally Unqualified" is a much trickier task than you might imagine.

I also disagree with the ongoing trend of fewer points. I don't think that's on purpose by the exam committee -- at least, I haven't heard of anything that fewer points is a goal.


ETA: I hope this goes without saying, but I feel the need to add: these are solely my opinions; I do not speak for the exam committee nor do my opinions reflect the opinions of the exam committee.
Do you know why the exam questions have been devalued? Does the CAS think they are making the test shorter when they make each question worth less?

It seems to me like you add a question or two on the exam and all the questions become worth less.

Exam 7 in 2017 there was a 2 point question where you needed to do four different methods (collective, individual, benktender, and optimal) for three different accident years. Besides for the fact the question took forever, there were just so many places to lose points.

Edit:
Looked it up it was 2 points it was question 1 on the exam.
__________________
ACAS 7 8 9

Last edited by act_123; 12-27-2018 at 04:04 PM..
Reply With Quote
  #35  
Old 12-27-2018, 04:02 PM
JMBNYC JMBNYC is offline
Member
CAS
 
Join Date: Sep 2018
Posts: 144
Default

When the exam is too long, the quality of all answers decrease. This seems obvious on its face, but is very much missing from the CAS announcement on the fall pass rates. The exam 8 examiners report essentially said, "well we warned you about IQ's, and we made a time adjustment." Fine, but they are missing the point. If everyone has to rush through the exam trying to snatch up random quarter points everywhere they can find because they don't have time, of course the end result is that the candidate pool appears unprepared.

These are remarkably low pass rates for everything but exam 6, and they've implied that the reason is that candidates didn't read prior examiners' reports. I imagine that at the very least almost all exam 8 takers learned long ago that they should read those reports and did so in preparation for this year's exam.
Reply With Quote
  #36  
Old 12-27-2018, 04:05 PM
OnLevel OnLevel is offline
Member
CAS
 
Join Date: Nov 2016
Posts: 1,634
Default

I just came here to say that I very much dislike the way the CAS does a lot of things, exams included. I'm too angry to actually offer any constructive answers at this time.
Reply With Quote
  #37  
Old 12-27-2018, 04:42 PM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 42,121
Default

Quote:
Originally Posted by act_123 View Post
Do you know why the exam questions have been devalued? Does the CAS think they are making the test shorter when they make each question worth less?

It seems to me like you add a question or two on the exam and all the questions become worth less.

Exam 7 in 2017 there was a 2 point question where you needed to do four different methods (collective, individual, benktender, and optimal) for three different accident years. Besides for the fact the question took forever, there were just so many places to lose points.

Edit:
Looked it up it was 2 points it was question 1 on the exam.
Historically, there've been a couple of times when the CAS tried to make the exams shorter by reducing the number of points on the exam. And it probably worked for a year or two. The exam chair had a robust sense of what a 2 point question looked like, and by removing 20 points, the exams was probably shorter.

But there's always pressure over time to jam more in. We need to cover all this material, after all. And the result has been (imo) that the value of "one point" has increased over the years, and I think that leads to bad outcomes.
Reply With Quote
  #38  
Old 12-27-2018, 05:24 PM
Marcie's Avatar
Marcie Marcie is offline
Member
CAS
 
Join Date: Feb 2015
Posts: 9,774
Default

Quote:
Originally Posted by PeppermintPatty View Post
Historically, there've been a couple of times when the CAS tried to make the exams shorter by reducing the number of points on the exam. And it probably worked for a year or two. The exam chair had a robust sense of what a 2 point question looked like, and by removing 20 points, the exams was probably shorter.

But there's always pressure over time to jam more in. We need to cover all this material, after all. And the result has been (imo) that the value of "one point" has increased over the years, and I think that leads to bad outcomes.
I would probably change "bad" to something like "sub-optimal," Sir, viewed from the big picture. It could lead to bad outcomes for some individuals.

I still believe the process is overall quite fair and mostly gets it right. But there are ways that it could be better.
Reply With Quote
  #39  
Old 12-27-2018, 07:08 PM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 42,121
Default

Fair enough.
Reply With Quote
  #40  
Old 12-27-2018, 08:17 PM
Maphisto's Sidekick's Avatar
Maphisto's Sidekick Maphisto's Sidekick is offline
Member
CAS
 
Join Date: Nov 2001
Location: South Park Genetics Lab
College: Ardnox
Favorite beer: The kind with alcohol
Posts: 3,094
Default

Quote:
Originally Posted by PeppermintPatty View Post
And the result has been (imo) that the value of "one point" has increased over the years, and I think that leads to bad outcomes.
I would suggest a recalibration of the value of "one point":

One point should represent something that the minimally-qualified candidate is expected to be able to answer in [a unit of time].

The value of [a unit of time] is debatable. When I started drafting this post, my inclination was to prescribe 2.25 minutes, which would indicate 100 points for a 4-hour exam, allowing for an extra 15 minutes to review... but the precise time/effort-per-point probably isn't particularly important as long as it's consistent.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 06:30 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.23673 seconds with 9 queries