Analysis of the SOA's question making paradigm.

[retaker]

Today we will analyze 3 standardized techniques the SOA’s psychologists currently use when making a question in order to take a simple concept and attempt to make it more difficult by changing the linguistics of the question:

Fairly good example:

Company X needs to decide whether or not to develop and market a pill to cure a certain disease. Once developed, then the pill needs to be approved before it can be sold.

If the pill is approved, the company will immediately incur marketing costs of 1000. The
probability that the pill will be a success is 75%.

If the pill succeeds, the company will receive a net cash flow perpetuity of 500 at the end of each year. If the pill fails, the company will receive a net cash flow of 500 at the end of year 1, declining by 100 each year for the following 4 years, and no cash flow thereafter.

Using an annual effective rate of 8%, calculate the probability of approval so that the expected net present value of the project is 1400.



Strategy 1: Superfluous bull Sh_t.
Note the very first sentence in the question. Developing and marketing have nothing to do with the problem.

Strategy 2: Use ordering of sentences. Note: this is an especially popular one.There are several instances of this technique in this problem.
1)As we just mentioned, the very first sentence in this question, in some
sense the anchor of the problem, is unnecessary.
2)“If the pill is approved, the company will immediately incur marketing
costs of 1000. The probability that the pill will be a success is 75%.”
By giving the probability of “success” immediately after the sentence
describing “approval” the student could be temporarily misled to think
that the “success” refers to the “approval”, after all, there is no mention
of contingency on success or failure to this point in the question.

Strategy 3: Using vague wording, and being intentionally vague in the about the temporal sequence of events in the question.
They never tell you how much time elapses between approval and success or failure, let alone the sequence of the superfluous info at the start of the question. Could be a year in between approval and determination of sucess, or at the same time, as they meant in this case.


Rewording of question so as to test actuarial ability rather than linguistic ability!


Company X has developed a pill to cure a certain disease. Now the pill needs to be approved before it can be sold.

If the pill is approved, the company will immediately incur marketing costs of 1000.

Once approved, however, success of the product is not guaranteed. The determination of the potential for success will be made at the same time as approval.
There is a 75% probability that the pill will be a success.

If the pill succeeds, the company will receive a net cash flow perpetuity of 500 at the end of each year. If the pill fails, the company will receive a net cash flow of 500 at the end of year 1, declining by 100 each year for the following 4 years, and no cash flow thereafter.

Using an annual effective rate of 8%, calculate the probability of approval so that the expected net present value of the project is 1400.



If the SOA wishes to make the problems more difficult to solve, I recommend digging deeper into the material to find more difficult concepts or changing to more difficult material all together rather than playing with language.

[Michael]

Are we sure that their motives are as we suspect? Do they really sit down and attempt to convolute straightforward problems using tangled language? Or, are they simply math types stuggling with language? Just a question....

[The Drunken Actuary]

So why is the 'exams' area the wrong place for discussing, um, exams?

[Moderator1]

So why is the 'exams' area the wrong place for discussing, um, exams?

retaker started identical threads in two sections. It is better to have all the discussion of a specific thread in one place. At that time, there was already one response here and no responses in 'exams' so I left this one alive and locked the other.

If retaker wishes, I will delete the thread in 'exams' and move this one there.

[retaker]

Yes, it is certainly possible that it is not intentional.

[retaker]

I don't care. I just hope I haven't p_ssed off Tracy? :oops:

[retaker]

Sorry, I meant Traci.[/i]

[Lex]

:viola:

Hey Retaker,

Cry me a river Argentina...you're so bitter.

I will agree that some questions the SOA puts could be worded better, but come on, is it so bad that you have to spend half a day writing your post?

[moot]

One of the many objectives of the education process is to enable the candidate to apply principles and techniques to real-world problems. Real-world problems come with lots of extraneous, irrelavant information attached. Questions with superfluous data test the candidate's ability to distill what is necessary and discard the rest from a problem. Where's the rub, retaker?

[retaker]

Half a day?

[retaker]

I don't recall seeing anybody else take the time to point this out. I was trying to you myself and you a favor by maybe illuminating the problem of poor wording of some of the exam questions with hopes that more care may be given in the future.

It's not easy to be revolutionary. :roll:

[Michael]

Noble intention, but I doubt the question makers spend much time here. Plus, it is hard to read it as altruistic couched as it is in somewhat cynical language. Matter of style, I suppose....

[moot]

I'm not trying to yank your chain, retaker, but if all twists and turns were removed from the exams, they'd end up looking like something that was slapped in front of you during the college calculus sequence - with tests like that, the value of ASA, ACAS, FSA and FCAS would be greatly reduced. One of the reasons actuaries are (generally) paid so well is that the prerequisites for the work and sometimes the work itself are HARD.

[retaker]

Well, I was hoping it might do some good.
Mr. White and Mahler are here, and although they are course 3, I believe they are important people. Even though I may be coming across as an a_s, I hope they might objectively consider what I have said. I don't desire to upset or insult anyone on the exam committees. I know they are volunteering and are no doubt very busy.

I am really not that bitter. I should have never chosen retaker. I will change it next month. (I hope)

[retaker]

Yeah, spud, i agree. I just think they should make it hard not by using language tricks, but difficult abstract concepts.

[Michael]

And that's fine. I would just suggest taht you could write it a little more gingerly if that is your intent. When you start off talking about "SOA psychologists", it kind of skews one's reading of it........

[retaker]

You're absolutely right, and actually I think they have gotten better about this in general. I only remember seeing 1 questionable question on Course 2 last time. There may have been 2, though. With Course 3, I don't remember seeing any. There were definitely some tricky ones with course 3, but they were tricky based on the material.

[Bama Gambler]

You're absolutely right, and actually I think they have gotten better about this in general. I only remember seeing 1 questionable question on Course 2 last time. There may have been 2, though. With Course 3, I don't remember seeing any. There were definitely some tricky ones with course 3, but they were tricky based on the material.

If there are only one or two poorly worded questions per exam, then why should they effect the outcome of your results. If you REALLY understand the material you should be able to get the other 47-49 questions right and still make a 10.

Bama Gambler

[Michael]

I'm with bammma on this one, for sure......I've always been surprised how people come to see these exams as conspiracies against eager students. We are far more in control here than is often assumed. And, again, there's no substitute for 500 study hours.........

[Cynic]

Come on, you guys! An actuarial student's life has little entertainment beside making fun of the SOA and other actuaries. :)

I think the exam writers want to prepare you for the real world by writing in obscure language. Remember that you are going to deal with actuaries for the rest of you life. Be prepared!

[Steve White]

Well, I was hoping it might do some good.
Mr. White and Mahler are here, and although they are course 3
On Course 3, and all the other committees I've been on, we spend a great deal of time trying to make sure the wording is not ambiguous. I sympathize with committees like Course 2 which have some non-numeric questions. Those are much harder to ask than numeric ones. You would be amazed at how much we on occasion discuss relatively minor differences in wording.

I can't recall a single occasion where it was suggested that we order the information to make a problem more challenging. It's fairly often that we reorder, compared to the way the original committee member proposed it, to make it more understandable. I don't think we have a standard that it must be presented in the easiest order to comprehend, but we're trying for challenging content, not challenging reading.

Howard Mahler knows the non-life Course 3 material better than any of us, I bet, but he has no role in the preparation of the exam.

[retaker]

Yeah, Cynic. :toast: Hey other guys :moon:

Mr. White, Thanks for your comment. I don't really believe that the wording is done just to mess us up, but it just seemed like since I had seen so many problems like this, that they might use the wording to make the problem harder to solve instead of some other method.
I attempted to be conciliatory concerning the way in which I presented my concern. Given that you say there is no malevolence on the part of the exam committees, please forgive me if I was overly critical.

Yeah, I believe Mahler is the supremum.

Why can't Batten and Mahler be charged with making course 3 exams?

[moot]

retaker said:
"Yeah, I believe Mahler is the supremum."

Given that the supremum is the least upper bound of a set (all other upper bounds are greater than it), I would think that Mr. Mahler might be slightly less than flattered by retaker's remark.

[retaker]

Not when the set is the set of all those who have knowlege of the non-life material on course 3!

Since the set is finite, sup = max. He is the Max.

[Alya]

Why can't Batten and Mahler be charged with making course 3 exams?
What do you mean "charged"? They most likely don't want to do it -- I don't think they would be able to continue their seminars if they where on the exam committee for that particular exam.

[urysohn]

Strategy 1: Superfluous bull Sh_t.
Note the very first sentence in the question. Developing and marketing have nothing to do with the problem.

"nothing to do with the problem"? God forbid any "superfluous" information is put in that actually relates the question to an actual work-related situation. That's the entire point of the question. It could just as easily, but more wordily, be rephrased as:

You are considering marketing and designing a new product. Here's the information you will base your decision on...


Not every problem on the exams can be written as "What's the present value of a $500 perpetuity, calculated at 8% interest [hint: ax = (1-v)^n....]?"

[retaker]

.Yes yes yes, but I am afraid you are a bit late, I have already been pimp slapped into submission by others, but thanks for the added reprimand.

Of course superfluous information is no problem, but in this case the superfluous info seems(ed) to be strategically placed (which it wasn't) to cause one think there is more going on than there is and mislead you concerning what you solving for.

[The Drunken Actuary]

So why is the 'exams' area the wrong place for discussing, um, exams?

retaker started identical threads in two sections. It is better to have all the discussion of a specific thread in one place. At that time, there was already one response here and no responses in 'exams' so I left this one alive and locked the other.

If retaker wishes, I will delete the thread in 'exams' and move this one there.I see.

[Budder]

I have already been pimp slapped into submission by others


and let the weekend begin!
:horse: :rofl:

[Howard Mahler]

I spent a dozen years on the CAS Exam Committee, the last 3 years as head of the whole Committee. We spent a lot of time trying to make the questions clear and unambigious. This does not say how often we succeeded.

I have never heard of anybody trying to write a poor question on purpose.
Having written exam questions for over 2 decades, I can say it is very easy to write a poor question and very hard to write a good question.

While I was somewhat involved with joint exams, I never served on a joint exam committee.

I have not been on the exam committee since 1993. However, I believe they have continued to try hard. Of course the volunteers on the exam committees are not professional exam writers. The CAS has hired the Chauncey group which is training question writers (on nonjoint exams.)
This will hopefully help in the future on CAS Exams.

Howard Mahler

P.S. Teachers of seminars can not serve on committees on the exams they teach. Seminar teachers can not see the exam before it is given, in order to avoid conflicts of interest. I do currenetly serve on the CAS Educational Policy Committee, which is not directly involved in putting together the exams.

[retaker]

Thanks, Mahler.

I think the fact that the CAS hired the consultants, to me anyway, illustrates that the CAS is more concerned with how we students feel concerning the exams. Since the exams are a big part of our lives right now, this should mean something to us. I wonder why more people don't migrate towards CAS?
Mr. Mahler, Do you have any rough idea on how the CAS system will change, if at all, in 2005, and whether next years new CAS course 4 will count towards the SOA?

[aces219]

Regarding why more people don't migrate towards CAS:
The exams are one part of your professional career, but you have to examine the work you will actually be doing to choose a track. I work in pension, and I really like doing so. I interned in medical and didn't like it nearly as well, so I believe I would feel the same way about the casualty side. It's also a question of what jobs are available in one's chosen location.

[retaker]

you mean you left medical, or even finished medical school and started taking exams?????? :-?

[retaker]

I know I am probably going to regret this, but I have just spent the day doing sample tests, and I couldn't help but notice a pattern in the way the, interest theory questions were worded.

Disclaimer:
Now, first off, I am not claiming that any of them were worded as poorly as the one which started this thread. Let me be clear. I do not think this is that big of a deal, it is just that I find it annoying and that it wastes time.

That being said, I find it hard to believe that these questions weren't purposely written to give you specific information in certain orders so that it takes you longer to set up the problem.
Ex:

Joe deposits 10 today and another 30 in five years into a fund paying simple interest of 11% per year.

Tina will make the same two deposits, but the 10 will be deposited years n years from today and the 30 will be deposited 2n years from today. Tina’s deposits earn an annual effective rate of 9.15%.

At the end of 10 years, the accumulated amount of Tina’s deposits equals the accumulated amount of Joe’s deposits.

(A) 2.0 (B) 2.3 (C) 2.6 (D) 2.9 (E) 3.2


In order to do this fast, telling us that Joe deposits 10 today and another 30 in five years into a fund paying simple interest of 11% per year , and, Tina will make the same two deposits, but the 10 will be deposited years n years from today and the 30 will be deposited 2n years from today. Tina’s deposits earn an annual effective rate of 9.15%. ,
does us no good if we don't know if we are accumulating it or taking the PV of it, or what. You notice that that isn't said until the end of the problem.
So basically you have to read through the whole problem and then go back and set up the equation rather than being able crank right from the beginning. Either that or you could scan through the problem, hoping you didn't miss some detail along the way, to get to the part where it finally tells you that you are accumulating first, and then start writing down the equation as you read through carefully. My point being, that your reading -language strategies are almost as important to doing the problem quick as knowing all the concepts.


Ex.
Bob has a choice of two loans when borrowing money to purchase a home. Loan X is a 30-year adjustable rate mortgage for 100,000. The interest rate for the first year is 4.5% convertible monthly. In the second year, the mortgage rate increases to 6.5% convertible monthly. In the
third year, the mortgage rate increases to 8.5% convertible monthly and remains at 8.5% convertible monthly for the remainder of the loan. Loan Y is a 30-year fixed rate mortgage for 100,000. The interest rate is 7.5% convertible monthly. Calculate the difference in the outstanding loan balances immediately after the 36th monthly payment for Loan X and Loan Y.
(A) 275 (B) 503 (C) 698 (D) 829 (E) 904


Granted they give you the interest rates as monthly, but they don't tell you the payments are monthly until the last line, and then it is even couched in the sentence: "Calculate the difference in the outstanding loan balances immediately after the 36th monthly payment for Loan X and Loan Y. "

It seems obvious that they are trying to trick you into writing this out as a yearly annuity. Again, I am not saying there is anything wrong with this! I just wanted to point out these facts, which are probably well known. However, as such, I don't think it can be correctly said that the wording of the questions is not very specifically chosen in some of the problems so as to not make them too easy. AGAIN, I AM NOT COMPLAINING.

[Gandalf]

:horse:

Granted they give you the interest rates as monthly, but they don't tell you the payments are monthly until the last line, and then it is even couched in the sentence: "Calculate the difference in the outstanding loan balances immediately after the 36th monthly payment for Loan X and Loan Y. "In practice in the US, home mortgages are monthly. It's nice that they later confirm it, but it shouldn't come as a surprise.
It seems obvious that they are trying to trick you into writing this out as a yearly annuity.What a dirty trick!! If you fall into that trap, you get exactly the same answer, to whatever number of decimals places you use.

[retaker]

This is my last beating of the dead horse. :horse:

It is not a dirty trick - not even a trick in this case, apparently. It is just annoying.

I didn't know you would get the same answer. I guess that is partly because you solve for the premium in this case.

[Valentina]

Why can't Batten and Mahler be charged with making course 3 exams?

Sorry - i just found this. :)


I was Bob Batten's student at GSU the year he retired; and my Life Con class was the last one he tought at GSU. He is truly the best teacher I have had and one of my most favorite persons in the entire world. I still try to keep in touch with him. But, trust me guys, you don't want his questions on the SoA exam. :) His tests were sooooo difficult, the SoA exams seemed like a breeze. :) Of course, the pass rate on the SoA exams for his students was over 90% for a reason. :)