Pass Marks

[mlschop]

http://futureactuary.soa.org/soaf/issues/2007-09-06/2.html

This might be old news for some (it was new to me!), but I think it's great for the SOA/CAS to release pass marks. I never really understood why they had to be so secret in the first place. Glad to see Bruce was able to get this change done!

Quick question about this. The posting here says "Multiple Choice exams." Since FAP-1 is MC, I'd assume it would include FAP-1, but wanted to see if that could be confirmed. Also - if it does include FAP-1, will this start with December 2007's sitting of FAP-1?

Thanks!

[sundwarf]

Yup, I totally agree. It's a giant step foward. Thanks Bruce!

[twig93]

:clap:

Way to go Bruce!

The system certainly should apply to FAP-1; hopefully it does.

Let's just hope this doesn't get repealed like the whole "the pass mark shall not be higher than 60%" fiasco of the November 2005 sitting.

[Gandalf]

:clap:

Way to go Bruce!

The system certainly should apply to FAP-1; hopefully it does.

Let's just hope this doesn't get repealed like the whole "the pass mark shall not be higher than 60%" fiasco of the November 2005 sitting.
Unlikely, at least not before pass marks get announced this fall. The "pass mark shall not be higher than 60%" was never announced by the SOA or CAS, or even adopted by the CAS. This has been announced in a publication of both organizations, which says it has been adopted by the Boards of both.

[bdschobel]

Yeah, it took a little longer but should be more lasting! And I expect that the passmark for FAP-1 on 12/12 will be released, but I'll confirm that.

Bruce

[Hukleberry]

I love the part that says:


Moreover – and much more troubling – Bruce was concerned that some clever candidates (aren’t they all?) were able to discern the pass marks through various post-exam analyses, and this created a non-level playing field.

:lol: It made me chuckle.

[atomic]

I love the part that says:



:lol: It made me chuckle.

Well, it's amusing, if only because what that passage really means is that the compilation of PAKs and detailed analyses (by a group of individuals from whom one could practically count on to do such things), made the SOA policy of withholding the pass mark irrelevant.

[bdschobel]

Not so. The passmarks derived here were not always correct, and everyone didn't know where to find them.

Bruce

[Gandalf]

From reading discussion in the various exam sections here, especially PAK threads, my impression is:

1. There is so much agreement in cases where the exam is released that the students must almost always be right about the passmark (I suspect there's 95% agreement among students in about released exams, and it's quite unlikely so many could be wrong.)
2. For unreleased exams, the PAK-derived passmark cannot be so reliable. For one thing, the PAK choice may not be the correct answer to some problems. For another, you wouldn't know if multiple answers were accepted to some questions, or questions were thrown out, etc.

Even for unreleased exams, the PAK-derived passmarks must be close, giving people who know it an advantage over those who have no clue about the passmark.

[bdschobel]

In any event, transparency is almost always good, and secrecy is usually ineffective and unnecessary. Just my 2 cents.

Bruce

[Gandalf]

Releasing exams and final answer keys is more transparent than releasing passmarks. I think you're for that, too, but the SOA hasn't committed to releasing them all. Releasing grading outlines on written answer exams would be another big step for transparency.

[bdschobel]

As we move toward computer-based testing for all the multiple-choice exams -- and all the advantages that CBT provides -- we need a large bank of unpublished problems. This is a case where transparency would be counterproductive, unfortunately.

Bruce

[Swan]

Let's just hope this doesn't get repealed like the whole "the pass mark shall not be higher than 60%" fiasco of the November 2005 sitting.
No kidding. I don't think I can stand to witness another Mannie Fresh-style meltdown on the AO. :shudder: If I were a veteran truck driver I probably would have increased my vocabulary of cuss words two-fold by listening in on that dude's diatribe against Bruce on that infamous day.

[bdschobel]

Yeah, that really was wild. Like I was trying to do harm or something! :rolleyes:

Bruce

[twig93]

Oh God, I'd suppressed a lot of the details, but now they're all coming back. That was a crazy thread, but well worth forgetting!

[yanz]

Just wanted to make sure - the article in the link only talks about how pass marks will be released, not what they actually were/are (ie, there's no actual release available, right)?

Also, the release is only for MC exams?

[bdschobel]

Yes on both counts. We have no plans to release historical passmarks. And no changes are planned with respect to written-answer exams.

Bruce

[User34]

As we move toward computer-based testing for all the multiple-choice exams -- and all the advantages that CBT provides -- we need a large bank of unpublished problems. This is a case where transparency would be counterproductive, unfortunately.

Bruce

Bruce, has the SOA thought about releasing an exam or a sample of FAP1 for the 12/12 sitting, in the event that the new "interim assessment" proposal passed in Oct, since this will be the last multiple choice FAP1 exam and they do not have to worry about keeping a large bank of unpublished problems anymore.