PDA

View Full Version : IQ Tests

llcooljabe
02-12-2007, 12:31 PM
A good friend of mine is doing grad work in psychology. He's got to administer an IQ test to 3 children aged 6-16. But he wanted to practice on me before a "live" test, so that he would be a tad more polished.

Each age group takes the same test, but has different starting points in each subtest. E.g. the 6 year olds would start at #1, whereas 16 year olds would start at #10. Also each age group is compared against itself.

Some components:
1. Repeat a string of numbers after the proctor. starts at 2 digits, goes up to 8. What makes this hard is that the proctor is instructed to say the string of numbers at a very slow pace--1 digit, wait a full second, 2nd, wait a second, etc. If you tell me your phone number quickly, there's a very good chance I'd remember it area code and all a couple hours from now. However, I found this more difficult than I expected. I did get them all right.

2. Hear a string of numbers, but repeat them backwards. I only got up to 7 digits with this one. Same format as above, the slow reading really throws you off.

3. Hear a string of numbers and letters, repeat in numerical and alphabetical order, numbers first. This one I only made to 6 digits. Very hard.

4. 2 or 3 groups of pictures. Pick 1 from each set that are connected. E.g. (Dog, tree, chair) and (fork, phone and cat), the answer is dog and cat, since they're both animals.

This one got very obscure at the end. We both couldn't figure out why the answer was the answer in many cases.

The other aspects were fairly simple.

Now to the point of this very long post: Most of the test was not culture specific. But for the 4th part described above, how do you keep culture out of it? For one or two of the answers, you could argue that less financially well off would not know the connection. But for the most part, this test did a good job of keeping culture specific questions out of it.

Even still culture will sneak in, no?

yanz
02-12-2007, 12:43 PM
4 is just weird. One can think of all sorts of combinations that could work. Aside from [dog, cat] I can argue [tree, fork], [chair, fork], and [chair, phone]. And if you loosen the constraint to picking 2 'connected' items, and not items from the same category (however you define category), you easily get [tree, cat] also.

The first 3 seem to be memory tests. Not sure how that relates to IQ...
Additionally, #3 requires that you know numbers/letters and can order them.

llcooljabe
02-12-2007, 01:02 PM
4 is just weird. One can think of all sorts of combinations that could work. Aside from [dog, cat] I can argue [tree, fork], [chair, fork], and [chair, phone]. And if you loosen the constraint to picking 2 'connected' items, and not items from the same category (however you define category), you easily get [tree, cat] also.

The first 3 seem to be memory tests. Not sure how that relates to IQ...
Additionally, #3 requires that you know numbers/letters and can order them.

I just made that example up. But for the "harder" sets of pictures, the connections were very obscure.

Phil
02-12-2007, 01:05 PM
Surely you must have higher IQ if you see the connection between dog and telephone, right? Anyone who picks dog, cat should lose points.

great3981
02-12-2007, 01:44 PM
See "The Mismeasure of Man" by Stephen Jay Gould for a clear explanation of why this is all hogwash.

In short, the test assumes that intelligence:
1) Exists as an objective entity
2) Can be reified to a unitary linear scale
3) Scale in (2) can be measured using a test
4) Results of (3) can be used to compare people in a group

Not novel concepts. Also completely baseless...

llcooljabe
02-12-2007, 02:34 PM
I'll look up that gould article.

My only concern was how to separate the tests from culture.

For example, if you told a poorer person to pick the match for "Cup" from the following list: "table, saucer, fork, plate", the poorer person may not choose "saucer" because he has no idea that a saucer exists.

The memory tests, and some of theother tests were (in my limited view) successfully removed cultural cues. The "which ones match" and "which ones belong" have more of a risk.

dlwktb
02-12-2007, 02:35 PM
See "The Mismeasure of Man" by Stephen Jay Gould for a clear explanation of why this is all hogwash.

In short, the test assumes that intelligence:
1) Exists as an objective entity
2) Can be reified to a unitary linear scale
3) Scale in (2) can be measured using a test
4) Results of (3) can be used to compare people in a group

Not novel concepts. Also completely baseless...

I agree with everything you said, but I still think that these test measure a quantifiable "skill" or set of "skills" and that they have their place in education. Perhaps IQ tests are just mislabled and overly hyped.

great3981
02-12-2007, 04:45 PM
I'll look up that gould article.

The book is well worth reading, as are several of his others.

My only concern was how to separate the tests from culture.

Indeed, you have hit on the biggest problem. Gould describes one of the tests where a tester was told to draw in the missing part of an object (a person with only one arm, a chair with three legs). Upon seeing a house (with no chimney), one candidate placed a cross at the peak of the gable (such was the decor on all the houses in the candidate's hometown in southern Italy). Needless to say, the candidate was declared more "feeble-minded" for this response.

Its one example where the culture bias can be more subtle than one may believe at the beginning.

I agree with everything you said, but I still think that these test measure a quantifiable "skill" or set of "skills" and that they have their place in education. Perhaps IQ tests are just mislabled and overly hyped.

I agree that they have their place. The inventor of the test (Binet) was trying to develop a test to identify struggling students whom would benefit from extra assistance at school, and indeed cautioned against using the test to try to measure general intelligence. Gould argues that the mislabelling and misapplication to which you refer were an easy and powerful perversion of his original intent.

4sigma
02-12-2007, 06:26 PM
I agree that #4 in your friends intended testing arsenal has culture bias.
"Connected" can entail all sorts of reasoning.

atomic
02-12-2007, 06:48 PM
The book is well worth reading, as are several of his others.

I second that statemetn--Stephen Jay Gould's books are considered required reading in many high school biology classrooms. They are easily accessible, well-written, thought-provoking, and discuss case studies of the ways in which science is often misappropriated for social or political purposes.

Gould was particularly critical of the book The Bell Curve, which stirred up quite a controversy when it was published.

QMO
02-12-2007, 11:52 PM
Using things (numbers and letters) that are associated with literacy are biased.

At Ellis Island there is an interesting exhibit on intelligence testing. For example, if you're testing someone that has never held a pencil, a test that requires use of a pencil won't work.

Mark Cavazos
02-13-2007, 01:19 AM
4 is just weird. One can think of all sorts of combinations that could work. Aside from [dog, cat] I can argue [tree, fork], [chair, fork], and [chair, phone]. And if you loosen the constraint to picking 2 'connected' items, and not items from the same category (however you define category), you easily get [tree, cat] also.

Back in HS, I was at a lecture by some prof. He talked about a teacher giving a "next term in the series" question. The series was "1, 2, 3, 5, ...." Most students answered "7" or "8". One answered "1" - she said that her series was "1, 2, 3, 5, 1, 2, 3, 5, ...."

While I was in college, Mensa would periodically put multiple-choice tests in the paper to see if you might qualify. Depending on how ornery I felt, I would try to justify as many answers as possible. (One time, the only mention of that the test was supposed to be timed was in answers that were carried on a separate page. Another time, none could qualify because there were not a sufficient number of questions to get up to the required point total. I hoped these were mistakes of the editors and not Mensa.)

The first 3 seem to be memory tests. Not sure how that relates to IQ...

My antecdotal experience is that people associate memory with intelligence. I have a very good recall that can impress the socks off people. At times when I have recall some minor detail, others have commented that I am really smart. I try to correct them by saying that I have a good recall. (I do think that I am intelligent, but I do not think that recall is a demonstration of my intelligence.)

Mark Cavazos
02-13-2007, 01:26 AM
Indeed, you have hit on the biggest problem. Gould describes one of the tests where a tester was told to draw in the missing part of an object (a person with only one arm, a chair with three legs). Upon seeing a house (with no chimney), one candidate placed a cross at the peak of the gable (such was the decor on all the houses in the candidate's hometown in southern Italy). Needless to say, the candidate was declared more "feeble-minded" for this response.

This always reminded me of a Monty Python skit. They were trying to test the intelligence of penguins. The test stood outside a zoo penguin enclosure, read the questions and took down the responses. The testers realized that there was a flaw in the testing, so they put a group of non-English speaking humans in the same enviornment. The next scene had a tester reading the questions to people standing inside the penguin enclosure. They found that the two groups' tests were consistently the same.

great3981
02-13-2007, 09:12 AM
Gould was particularly critical of the book The Bell Curve, which stirred up quite a controversy when it was published.

The interesting aspect of The Mismeasure of Man is that he republished the original text shortly after The Bell Curve was published and sublines it "The Definitive Refutation to The Bell Curve".

The Mismeasure of Man was originally published 15 years prior to The Bell Curve.

I second that statemetn--Stephen Jay Gould's books are considered required reading in many high school biology classrooms. They are easily accessible, well-written, thought-provoking, and discuss case studies of the ways in which science is often misappropriated for social or political purposes.

Best one for me was Full House. His essay collections (Dinosaur in a Haystack etc.) are awesome as well.

This always reminded me of a Monty Python skit. They were trying to test the intelligence of penguins. The test stood outside a zoo penguin enclosure, read the questions and took down the responses. The testers realized that there was a flaw in the testing, so they put a group of non-English speaking humans in the same enviornment. The next scene had a tester reading the questions to people standing inside the penguin enclosure. They found that the two groups' tests were consistently the same.

:lolup:

CyberGuy2004
02-13-2007, 09:41 AM
Back in HS, I was at a lecture by some prof. He talked about a teacher giving a "next term in the series" question. The series was "1, 2, 3, 5, ...." Most students answered "7" or "8". One answered "1" - she said that her series was "1, 2, 3, 5, 1, 2, 3, 5, ...."
Examples like this and the one about incomplete houses (chimney vs cross) are among the reasons that an IQ test should never be administered in a strictly paper and pencil fashion. AFAIK, Binet's original plan was an oral exam, allowing the student to explain "nonstandard" answers if necessary. A standardized paper-and-pencil test can only say right or wrong. Standardized tests have their uses, but they aren't the answer to everything.

BTW and IMHO, the best definition of "Intelligence" is "What IQ tests measure,":rimshot: or "I'll know it when I see it." :judge:

Heywood J
02-13-2007, 10:23 AM
Is Stephen Jay Gould really that highly regarded? I was under impression that scientists considered him someone whose writing ability far outstripped his knowledge.

llcooljabe
02-13-2007, 10:36 AM
Examples like this and the one about incomplete houses (chimney vs cross) are among the reasons that an IQ test should never be administered in a strictly paper and pencil fashion. AFAIK, Binet's original plan was an oral exam, allowing the student to explain "nonstandard" answers if necessary. A standardized paper-and-pencil test can only say right or wrong. Standardized tests have their uses, but they aren't the answer to everything.

This implies that the test givers are intelligent themselves. From the stories my friend has told me, that's not always the case.

Maine-iac
02-13-2007, 10:44 AM
Interesting use of an IQ test that I ran across this morning . . . .

http://www3.ambest.com/frames/FrameServer.asp?Site=news&Tab=1&RefNum=91287&AltSrc=13

llcooljabe
02-13-2007, 10:48 AM
Interesting use of an IQ test that I ran across this morning . . . .

http://www3.ambest.com/frames/FrameServer.asp?Site=news&Tab=1&RefNum=91287&AltSrc=13

I literally just finished reading the article and came here to post it.

For those of you who aren't registered with AM Best:

BlueCross of Tennessee Eliminates IQ Test for Weight-Loss Surgery

OLDWICK, N.J. February 12 (BestWire) — Following criticism from a group that represents obese people, BlueCross BlueShield of Tennessee recently eliminated an IQ testing requirement for members seeking costly weight-loss surgery. Although the situation highlights Blue Cross' practice, what often goes unnoticed is that employers are looking to health insurers on how to cut their health benefits costs.

The Obesity Action Coalition does not have a problem with psychological evaluations being administered to people seeking the surgery, said James Zervios, the organization's director of communications. The coalition did take issue with BlueCross's IQ test, which "was blatant discrimination against those affected by obesity," he said. "It seems that obesity is the last disease where it's socially acceptable to discriminate against those individuals."

The coalition learned of the IQ test requirement when a BlueCross member told it they wanted the weight-loss operation, known as gastric bypass surgery, but that BlueCross was requiring the member to take the test, Zervios said. "Bottom line: An IQ test is not something that needs to be included in an insurance policy. That really perpetuates the negative stigma that's so often associated with those affected by obesity."

About 127 million U.S. adults are overweight, 60 million are obese, and 9 million are severely obese. Obesity is defined as being at least 30 pounds heavier than one's ideal weight. Those who are obese have a body mass index of 30 or greater and morbid obesity is indicated by a BMI of 40 or greater. Gastric bypass surgery involves reducing the size of a person's stomach and intestines, which then limits the amount of food a person eats.

The claim that the IQ screening policy, part of an overall psychological evaluation for bariatric candidates, was discriminatory, is "absolutely false," said Mary Thompson, a spokeswoman for the Tennessee Blues. However, because the coalition "misconstrued" the IQ test requirement, the company is no longer requiring it for bariatric candidates, she said.

"There was nothing in our policy that indicated that if your score…was below a specific level that you could not have the procedure," Thompson said, explaining that BlueCross enacted the IQ screening policy in April 2006 for all morbidly obese patients looking to undergo bariatric procedures.

The purpose of the test was to measure a patient's knowledge of the surgery and the eating and lifestyle changes a patient would need to follow after undergoing the operation, said Thompson. BlueCross considered the IQ test "critical" in terms of the patient's pre- and post-operative care, she said.

But in the United States, an IQ test is not required for any other disease, Zervios countered. "We don't tell a cancer patient, 'no chemotherapy until you take an IQ test.' " Similarly, "we don't IQ test somebody before they have skin cancer removed to make sure they're not going to go out in the sun after the operation is finished."

Rick Mayes, assistant professor of public policy at the University of Richmond, said he's never heard of a health insurer having an IQ test requirement for bariatric surgery.

However, BlueCross' medical team attended the 2006 Bariatric Summit, where experts stressed the importance of pre-operative weight loss and the need for well-designed studies focusing on the responsibilities of the individual having this surgery, and the outcomes of pre-assessment and behavioral assessment of the individual, said Thompson. "After hearing the concerns expressed at the summit, our medical team engaged in their own research and determined that the IQ screen was an effective tool in protecting members," she said.

A government report last year showed that almost six out of 10 bariatric patients land back in the hospital six months following the bariatric surgery due to dehydration and malnutrition, Thompson said.

"Requiring a formal IQ test as part of the evaluation process for insurance coverage does seem odd," said Brian D. Boyle, co-chair of the Health Care Group at O'Melveny & Myers LLP, Washington, D.C. Nevertheless, according to BlueCross, the purpose of its screening process wasn't to decide whether coverage would be provided, but rather to identify those patients who require extra counseling and attention in the post-surgical period, where the standard of care suggests that close attention to diet, "coping skills," and lifestyle changes is required, he said.

BlueCross of Tennessee "should be able to meet this important objective without having to administer a test of the three R's — reading, 'riting, and 'rithmetic," Boyle said.

BlueCross does not cover bariatric procedures among its fully insured plans, Thompson pointed out. The IQ screening was online specifically for BlueCross' employer customers who self-fund their health insurance plans and who elect to cover the bariatric procedure for their employees, she said.

Roughly half of BlueCross' accounts are self-funded and a relatively small percentage of those accounts, no more than 10%, offer bariatric coverage to their employees, Thompson said.

Employers are always turning to health insurers for ways to ensure that their dollars are being spent wisely and that those dollars result in the improved health of their employees, she noted. However, in the case of its former IQ test requirement, Thompson said she didn't know whether BlueCross' self-funded employer customers requested it.

Health insurers and employers try to legally avoid being "overwhelmed" with a disproportionate share of that 20% of the population that drives 80% of the health spending in the United States, said Mayes.

llcooljabe
02-13-2007, 10:48 AM
Incidentally, most med mal professionals are terrified of bariatric surgery (gastric bypass surgery and the like)

great3981
02-13-2007, 02:38 PM
Is Stephen Jay Gould really that highly regarded? I was under impression that scientists considered him someone whose writing ability far outstripped his knowledge.

In which field and to whom? He was a Professor of Zoology at Harvard for most of his life, if that speaks to his professional credentials. His PhD dissertatation was on the punctuated equilibria version of evolution, which has been a very influential theory (not sure how much of it was original and ground-breaking during his dissertation). Not sure how his other major academic works were perceived by his colleagues in the field.

He definitely had a gift for explaining complex evolutionary concepts to the interested layperson and clearly enjoyed doing so. Could this contribute to a view among his peers that his knowledge was subpar? Perhaps, and, if true, I would argue this view is unfair. Understanding the subtle differences that are lost when one must explain more complicated concepts to a non-expert audience doesn't necessarily mean that the author doesn't appreciate those differences as well.

Could his popularity from writing such books cause some of his peers to believe him to be overrated? Probably, and, if true, I would again argue that this is unfair. He wrote to various audiences. The value that he gives to one audience shouldn't be equated with the value he gives another.

Perhaps I am biased. If you can't tell, I happen to be a big fan. :)

great3981
02-13-2007, 02:43 PM
The purpose of the test was to measure a patient's knowledge of the surgery and the eating and lifestyle changes a patient would need to follow after undergoing the operation, said Thompson.

And they thought the best way to do this was by an IQ test (independent of the allegations of bias/racism)? Am I missing something??

Maine-iac
02-13-2007, 03:48 PM
Well, they claim that IQ was one of the tools to identify those that would need more close post-op monitoring and supervision to ensure that they were doing what they need to do to avoid complications, and that it was not used to determine who does and does not qualify for coverage.

I suppose it's possible that IQ could be a legitimate marker for the ability to understand and follow a set of complex instructions. But they would have been a lot better off to use some independent "psychological screening" assessment than to rely on on the IQ test with all the potential for negative associations.

great3981
02-13-2007, 03:53 PM
Well, they claim that IQ was one of the tools to identify those that would need more close post-op monitoring and supervision to ensure that they were doing what they need to do to avoid complications, and that it was not used to determine who does and does not qualify for coverage.

IQ as an indication of whom would be more likely to take responsibility for their own post-operative care? If you need a behavioral indication, I'd probably rely more heavily on a credit score (unless you are trying to avoid the poor/racism critique).... :shake: