Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > General Actuarial
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions

Salary Surveys
Property & Casualty, Life, Health & Pension

Health Actuary Jobs
Insurance & Consulting jobs for Students, Associates & Fellows

Actuarial Recruitment
Visit DW Simpson's website for more info.
www.dwsimpson.com/about

Casualty Jobs
Property & Casualty jobs for Students, Associates & Fellows


General Actuarial Non-Specific Actuarial Topics - Before posting a thread, please browse over our other sections to see if there is a better fit, such as Careers - Employment, Actuarial Science Universities Forum or any of our other 100+ forums.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 12-27-2018, 03:20 AM
almost_there almost_there is offline
Member
 
Join Date: Jan 2017
Posts: 328
Blog Entries: 1
Default IFoA marking criticised

Over the last few years the IFoA have been criticised on social media increasingly for their marking. Students have been increasingly submitting data subject access requests to the IFoA to obtain their marks and therefore can see the breakdown per question and how marker 1 and marker 2 scored them. End of 2016 the IFoA even sent round a student newsletter defending its marking and claimed it was in accordance with accepted Uni standards in that for 90% scripts that marker 1 & 2 were within 10 marks of each other. One can see from the acted forum and elsewhere that students are still experiencing marking discrepancies between marker 1 & 2 that are way outside the 10 mark tolerance IFoA finds acceptable. IFoA are also being criticised for not taking any serious action on these matters; a 3rd marker all too often presents a mark at the average of marker 1 & 2. This in itself presents problems when markers 1 & 2 are far out.

Last edited by almost_there; 12-27-2018 at 08:08 AM..
Reply With Quote
  #2  
Old 12-27-2018, 07:53 AM
therealsylvos's Avatar
therealsylvos therealsylvos is offline
Member
CAS
 
Join Date: Mar 2014
Posts: 16,919
Default

Interesting, can any CAS graders weigh in? 10 seems like a lot, I think for CAS exams graders need to reconcile much closer than that.
__________________
Quote:
Originally Posted by ao fan View Post
i like laying in the tube and pretending that i'm dead.
Reply With Quote
  #3  
Old 12-27-2018, 08:06 AM
almost_there almost_there is offline
Member
 
Join Date: Jan 2017
Posts: 328
Blog Entries: 1
Default

Quote:
Originally Posted by therealsylvos View Post
Interesting, can any CAS graders weigh in? 10 seems like a lot, I think for CAS exams graders need to reconcile much closer than that.
I agree with you that 10 is a lot, far too much.

Look at what this person has suffered:
https://www.acted.co.uk/forums/index...46/#post-58644

This was the student letter of Dec 2016 I was referring to:
http://instituteandfacultyofactuarie...9558A201773426
We have carried out a thorough assessment of the quality of examination marking. We have looked at almost 15,000 scripts from across all the April 2016 examinations. The outcome is that, for each examination, over 90% of scripts have a variance between first and second independent markers which is within the normal academic expectation of independent marking (0-10 marks on a 100 mark total paper).
Reply With Quote
  #4  
Old 12-27-2018, 08:59 AM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 39,780
Default

I have graded CAS exams. These days, there is a different grading pair for every question. The pair that grades question #5 will ordinarily grade question #5 for every candidate. (For very long questions, there are sometimes 4 graders who each grade a bit more than half the candidates, with overlap. Back in the old days, when we had fewer candidates, one grading pair would grade two or more questions across the exam.)

So there's no way to add up the scores from markers 1&2 across the exam.

We used to be asked to reconcile every candidate's score to within a quarter point before we went to Central Grading, where we would review the grades of every candidate close to the pass mark. Today we are asked to reconcile exactly. So there would be no difference, however you added up the scores.

We still regrade the candidates who are close. The idea is that the precise application of the grading rubric might shift slightly as one works through the stack, and the candidates who are close to the pass mark should be graded by the final, best rubric. Each grading pair must reconcile exactly. There are often productive discussions between the two as to exactly what this paper says and what sort of error that is.
Reply With Quote
  #5  
Old 12-27-2018, 09:01 AM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 39,780
Default

Oh, it's worth pointing out that our exams used to have 100 points, but now have something like 50-60 points. I think this creates problems in grading, and hope the CAS can renormalize to 100 points. But that's not really related to this discussion.
Reply With Quote
  #6  
Old 12-27-2018, 09:10 AM
bsanders33 bsanders33 is offline
Member
CAS
 
Join Date: Jun 2018
Posts: 150
Default

I wouldn't worry about it too much. As a student, just focus on preparing as much as you can, so hopefully you're not on the borderline where subjective decisions of the graders determine if you pass or fail.

Students who don't pass will always complain about the graders (and the exam questions), just like fans of a losing sports team will always complain about the refs.
Reply With Quote
  #7  
Old 12-27-2018, 09:18 AM
therealsylvos's Avatar
therealsylvos therealsylvos is offline
Member
CAS
 
Join Date: Mar 2014
Posts: 16,919
Default

Quote:
Originally Posted by bsanders33 View Post
I wouldn't worry about it too much. As a student, just focus on preparing as much as you can, so hopefully you're not on the borderline where subjective decisions of the graders determine if you pass or fail.

Students who don't pass will always complain about the graders (and the exam questions), just like fans of a losing sports team will always complain about the refs.
I've been quite successful in passing exams, with just one more to go. That doesn't stop me from complaining about exam questions and grading. I think people who finished a long time ago tend to minimize issues since they're so far removed from it affecting them. But bad practices should be fixed, and if candidates don't complain they won't get fixed.
__________________
Quote:
Originally Posted by ao fan View Post
i like laying in the tube and pretending that i'm dead.
Reply With Quote
  #8  
Old 12-27-2018, 09:32 AM
Gandalf's Avatar
Gandalf Gandalf is offline
Site Supporter
Site Supporter
SOA
 
Join Date: Nov 2001
Location: Middle Earth
Posts: 31,044
Default

Quote:
Originally Posted by PeppermintPatty View Post
I have graded CAS exams. These days, there is a different grading pair for every question. The pair that grades question #5 will ordinarily grade question #5 for every candidate. (For very long questions, there are sometimes 4 graders who each grade a bit more than half the candidates, with overlap. Back in the old days, when we had fewer candidates, one grading pair would grade two or more questions across the exam.)

So there's no way to add up the scores from markers 1&2 across the exam.

We used to be asked to reconcile every candidate's score to within a quarter point before we went to Central Grading, where we would review the grades of every candidate close to the pass mark. Today we are asked to reconcile exactly. So there would be no difference, however you added up the scores.

We still regrade the candidates who are close. The idea is that the precise application of the grading rubric might shift slightly as one works through the stack, and the candidates who are close to the pass mark should be graded by the final, best rubric. Each grading pair must reconcile exactly. There are often productive discussions between the two as to exactly what this paper says and what sort of error that is.
The SOA is extremely similar. There are different graders for each question. (For an exam with few candidates, some graders may grade more than one question, but even then any grader grades far under 50% of the questions.) If those two give different grades, they must reconcile extremely closely. Not necessarily exactly, but extremely closely. For candidates who end up very close to the pass mark, the paper is reviewed again in its entirety.
Reply With Quote
  #9  
Old 12-27-2018, 09:44 AM
PeppermintPatty's Avatar
PeppermintPatty PeppermintPatty is offline
Member
CAS
 
Join Date: Sep 2001
Posts: 39,780
Default

Oh, it's perhaps worth mentioning that there is no designation of "grader 1" or "grader 2". Because they need to reconcile exactly, there is no need to say that one is first or more important. I'd guess they are listed alphabetically on some master sheet somewhere, but there is truly no way to add up the "first" or "second" score. (except that since they are the same, I suppose you could do so at random and be correct.)
Reply With Quote
  #10  
Old 12-27-2018, 09:55 AM
therealsylvos's Avatar
therealsylvos therealsylvos is offline
Member
CAS
 
Join Date: Mar 2014
Posts: 16,919
Default

Quote:
Originally Posted by PeppermintPatty View Post
Oh, it's perhaps worth mentioning that there is no designation of "grader 1" or "grader 2". Because they need to reconcile exactly, there is no need to say that one is first or more important. I'd guess they are listed alphabetically on some master sheet somewhere, but there is truly no way to add up the "first" or "second" score. (except that since they are the same, I suppose you could do so at random and be correct.)
Just curious, what happens if the graders can't agree with each other to reconcile? What's the procedure? Is there some higher authority?

I ask because I see plenty of questions in examiners reports where they say they took off points for some inane reason, (like not explicitly saying a retro plan is balanced even though the question itself stated it) and can't see myself ever giving in if I was a grader and agreeing to dock points for that.
__________________
Quote:
Originally Posted by ao fan View Post
i like laying in the tube and pretending that i'm dead.
Reply With Quote
Reply

Tags
complaints, ifoa, issues, marking

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 10:35 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.15045 seconds with 12 queries