The shonkiness of the MMPI-2

I just did the Minnesota Multiphasic Personality Inventory – 2; highly respected within the Psychiatric community. If a professional were to analyse my results, I'd be pinned as a paranoid schizophrenic with psychopathic traits. In my personal case I suffered extreme bullying to which I have MANY witnesses; (thereby ruling out the paranoid and schizoid personality traits), and every reason to want to get justice (personally, I fight back – I won’t allow people to trample on me). But an ignorant shrink would most probably treat me as a potentially 'dangerous individual' with delusional tendencies and shove me on Haloperidol or the like, or have me locked up etc. The questions are just too broad and without scaled answers (maybe rated from 1 to 10) as opposed to the simplicity of 'true' vs 'false' it's hard to take this sort of diagnostic test seriously.
As an example: 'Once in a while I think of things too bad to talk about'. Isn't that a little too subjective? For example, if I had been brought up in a family of fundamentalist Christians who opposed the sexual liberation of women, and I honestly answer 'true' to that question because I've contemplated sex a couple of times, it will skew my answer toward an ASPD diagnosis. Another example: 'At times I have fits of laughing and crying that I cannot control'. Sometimes we just get the giggles after someone has said something hilarious - it by no means questions what they are genuinely trying to get at - the people who just randomly laugh with no apparent reason...... That I don’t do – there’s always a good reason I laugh, but answering that question honestly has me classified as someone not responding to environmental stimuli normally. And that was just question 23 out of 567 questions and I was just skimming through picking up on their flawed semantics…….

Hey Interested Observer,

I think there's a common misunderstanding of psychometrics that responses to test items somehow quantifies certain measurable attributes that a person has when really we don't know a darn thing about what these measurable attributes are supposed to be and how the act of circling an answer on a pen and paper test would do anything like what we understand as measurement (i.e. thermometers, weighing scales, light meter, etc.).

Instead, when it comes to psychological measures just ask two questions: "what do the scores of the measure predict?" and "what do the scores of the measure correlate with?".

The rest is just mythology and confusion imo.

Minnesota has been kicking out a plethora of these indexing schedules. I've been subjected to a number of them. Been classed a high risk violent offender very likely to repeat by one of them and I spent 3 days crying over a mouse that accidentaly got its head caught in my recyling bin hole over how I could have prevented the death. My wife will attest I am gentle, and have a great respect for life, have disdain for violence... yet these tests admistered why going through a process that says I was a square peg, despite being round... they put my in a box regardless of whether I fit the data or not to back the success rate of the study. It's all about money, and what looks good on paper...

The questions are beyond vague. They can be interpreted incorrectly by the indiviudal taking the test, and they can be equally interpreted incorrectly by the person issuing the test. Minnesota is big on tests and very small on doing anything really credible. They have a corrupt justice system and have also issued such things as the duluth model of domestic violence. Yet everyone watches them and treats them like great innovators of knowledge and wisdom. These processes are dangerous and put innocent people at high risk for misclassification and people that should be classed not even in the radar. These tests repeated miss serial killers and the most dangerous people of our society. That right there should tell you they aren't worth the paper they're printed on.

These tests remind me of a polygraph test. Since those aren't even considered acceptable and many sets of questioning can be similar to what's found on those tests. Primarily the psycho/sociopathic index things like the tolam scale for violence and all sort of other hogwash the community thinks very highly of.

Scales are very often taken out of context by the person interpeting the scale. When asked how much pain you're in an emergency situation most poepl scream 10 to get help. When it might only really be a 4 or 5. They need the massive number of questions to judge truthfulness, and accuracy to repeat pattern response to similar question pools to rule out someone faking. It's a serious headgame is what it is. Since some of the questions despite being an effort to word something the same but appear different can actually have a different meaning.

IE: I have a phobia of lines... and a problem with showering. But I'm OCD with cleanliness.

When I was in the psyche ward the bathroom had a small tiles on the floor with lots of lines. When I shower I have to clean everything 4 times. I was at war with cleaning myself, and freaking out from having to stand on the lines on the floor to shower. I was at war with two states of controlling problems. Solution was to spread bedsheets on the tiles so I could wash while in the hospital. These tests cannot even remotely begin to take situations like this into consideration. The hospital and the tending psychiatrists and nurses didn't understand the problem. The Janitor though was nice enough to get me spare bed sheets. Ultimately it was the Janitor that saved my sanity.

I remember the tolman test clearly only being administered against criminals convicted of/charged with violence... and specifically telling people not to use the test on people that have not been subject to criminal process because it may generate a false positive. Hmmm... scientific process would at least require a placebo/non-positive failsafe test state to give the normal state. When you only include one side and proclaim it tracks violence, but only in people charged with violence... it's like a modern day witchhunt. There's nothing credible to balance the test statistically.

What's scarier is the N=?? test subjects for many of these tests is very very very small. Often not even enough to get a 95% check. Which against Z-test or Chi-Square or anything else that would be credible to produce a statistical precedence.

Having studied most of the studies that have been conducted, they are not credible. But they are held to be credible, because they are backed by professionals on a present day witchhunt. Just be thankful they're not labotomizing us anymore.

To both Infinite-Jester and Vandel,
Thanks for your responses. Popper would be turning over in his grave if he knew anything about modern Psychiatric assessment. Hardly scientific is an understatement. As, I think Vandel said, when only 30% of your sample groups' responses (usually composed of a minimum of 30 subjects being the minimum number needed to get a peer-reviewed journal entry) support your hypothesis, yet that is considered enough to push through a theory, something is drastically wrong!
Take care and big cuddles,
Emma <3

InterestedObserver wrote:Popper would be turning over in his grave if he knew anything about modern Psychiatric assessment. Hardly scientific is an understatement.

That's an interesting question about what Ol' Papa Popper would think about measurement practices in psychology/psychiatry. Claiming an intelligence test measures intelligence isn't really falsifiable because it's something of a tautology in the sense that you're defining the construct by how you're measuring it.

As for measurement practices being unscientific, I think they're better understood as being normative rather than empirical. The real question is what relation the normative practices have to behaviour and experience.

Also, I don't know how much increasing sample size would improve the problems with measurement. Even if you had a population of 100 000 complete your measure and participate in a longitudinal study looking at what the test scores predict or correlate with I don't know if that would answer the question of "what does it measure?" because that's a conceptual question. Not an empirical one.

Again, the question is "what does it do?" not "what does it measure". As for MMPI, I have no idea. I know there are clusters of test items that correlate well which is pretty.

Sample size and time the sample is taken can also have drastic consequences on the outcome of the study.

I do a number of studies against the North Carolina public offender database. Take something like prostitution. When you plot the crimes you find bands forming where the arrests plummet during winter months... and sharply rise when temperatures are warmest. A summer study would give a false positive for winter months... and vice versa. So the whole of the year needs to be taken into consideration to show that there is a decline and to generate a hypothesis as to why this might be occurring. Comparing it against a state like Florida you don't get that banding. And looking at a state like Nevada you need to look at counties, since in some it's legal. Polling prostitutes seriving prison sentences and looking at the states where they're from, you might conclude that Florida in this case has a larger prostitution problem because they're active all year round. But what might be missed in this case is that Nevada has a larger number of escorts and prostitutes, and that another problem all together is generated in North Carolina in the off season is that you get a spurious increase in sexual assault in the winter months when the prostitutes aren't available. So if you merely studied the North Carolina data and promoted it for the country you'd have a seriously incorrect skew and an unobjective analysis. This single state study process being applied accross the nation for studies done solely in Minneosta and finding them applied to the world quickly becomes apparent as to why this is a bad practice.

It's like looking at speeding tickets in the state. Looking at specific weeks and time periods. As a whole the first and last sunday of each month have the highest tickets produced. This would suggest to some that there's a quota. Higher samples for breakins, you get a funky stat that no BE has occured in the state on a tuesday between the hours of 2-3am in the morning... I don't remember if these are all correct, it's been a while since I looked at my data and such but these are examples of things that kind of of tip off in the back of my mind. I'm working with some 2.7 million records. Seems like a lot, but when you start splitting by name, date of birth, and crime by year... and hoping for an n=5 for a given day of the year and your numbers start getting really small to mean N=5. The more you're checking the vastly greater the sample size needs to be to meat the minimum requirements.

In the case of these tests using N=30 and grouping all violent crimes, or all classes of a mental illness in a given schedule you get an even smaller sample subset when you break down that the test covers 10 segments... you have an actual N = 3 nominal for a given subject segment.

They claim high accuracy. I remember reading through the collective works of CGY in his early type inidication studies where his N=2 for Introverted... arguably it may have been N=1.

Some FDA drug testing can be a little as N=10. This meets the double blind minimum of 5/5 for what is classed as minimum needed for an accurate statistical measure. This to me is a joke. That 10 people can represent 7 billion.

Given the MMPI-2 can be used to guage viablity for security clearance. This is an interesting article for malignant attempts to fake out the MMPI and MMPI-2... using a semi-decent sample size in comparison...

http://maamodt.asp.radford.edu/Research%20-%20Forensic/97%2012-2-Aamodt-42-47.pdf

There's been a lot of studies conducted on the conclusiveness of these studies and they all basically conclude that a lot more research needs to be conducted before serious conclusions can be made as to the validity of the tests.

Just for kicks I checked the 1979 mortality data from the CDC and they still haven't corrected it. August 10, 2010 I sent them a fix and it's still not done. That's a good case for larger sample size can generate a problem. If you looked at all the records in that file, because of an error in the data, you get 20k worh of babies dying in mining accidents. This illustrates another important problem is that a lot of the data used in these studies is inaccurate or incorrect.

I wish there was something more conclusive, but right now there's not. I don't think there will be. If you want stat testing I'd argue banking corporations have compiled the largest amount of data over the years. I know the MBTI was compiled largely based on the banking surveys, or at least followed the model of the banking surveys. Doesn't mean the MBTI, or Kersey Temperment Sorters or any of these other tests are accurate. Again, study upon study I conclude the same thing... you've got as good a chance at having your palm read to determine what's going on as you do with these tests sometimes. It's psuedo-science at its finest.

Need serious ZZZ's... Be well everyone... Interesting disucssion.

Hi Infinite Jester,
I agree with you about defining a construct in the way that you measure it. Criticisms of most IQ tests usually result in the Bell curve being bandied around as proof IQ tests are too socioeconomically and culturally relative to be of any substance.
I’ve always understood psychology as being a social science, in that progression within the paradigm is made using a hypothesis that is presumably refutable and if it can’t be nullified, it’s understood as a ‘breakthrough’ or verification of the like. The steps taken in all scientific methodology, whether that be the social sciences or ‘pure’ sciences are:
1. We observe
2. We present a possible hypothesis for what we have observed
3. We make deductions from our observations that are both explanatory and predictive
4. We test the hypothesis
Using a 30% success rate as ‘voiding the null hypothesis…’ is sure idiocy in my mind. It would never happen within the ‘truly’ scientific community - thus my comments about Popper. In the world of physics at least, that would be a joke.
As for your criticisms of my comments about using such a small sample and accepting a supposedly scientific principle on the basis of the results from that sample, even ‘normative’ ideas rely on a few more than 30 people – what may be culturally acceptable to you may not be culturally acceptable to me (especially when you only need 30% of the group to confirm your hypothesis) the same ideas standing with ideology, religion etc. I believe the wider the sample group, the more accurate the results; that in itself is inductive reasoning within a ‘normative’ framework. The smaller the group, the more open to bias it is.
My understanding of empiricism is that it is knowledge gained from experience – It may have different connotations within the Psychiatric / Psychological community – I’ll have to look that one up. In my experience, people are varied. Taking the MMPI-2 tomorrow is most likely to be different to taking it today….
I understand that in order to understand people more clearly, we need to ‘define’ them to some extent. However, within the discipline, I think it’s better to take a leaf from the likes of R.D Laing’s work, and treat each individual as an individual as opposed to a person defined by a label – especially given the numerous false positives that come from largely defunct tests such as the MMPI-2.

Best wishes,
Emma <3
PS. If I'm losing track of the argument it's because I'm always pretty drunk by this time of night lol

Hi Vandel,
I just saw your response and think we're pretty much coming from the same point of criticism, though I skimmed through so I might end up editing this as I'm pissed right now lol.
Anyway,
Take care,
Emma <3

Hey Interested Observer & Vandel,

I understand that having a larger sample size increases the external validity of a study and to answer the question of how reliable a measure is requires large sample sizes. However, increasing sample size does not answer questions like "what does a measure measure" because that is a conceptual question. Not an empirical one. For example, look at the following questions and think about whether they are could possibly be answered by systematic observation.

(1) What is intelligence?
(2) How do you measure intelligence?
(3) How well does the Wechsler intelligence scale measure intelligence?

I don't see how experimental research could possible answer questions (1) and (2) because (1) is a question about the meaning of a concept and the rules of it's instantiation and (2) is a question about ontology in the sense that you're asking what kind of thing intelligence is and would you quantify it. As for (3), provided you're asking about the properties of the Weshsler, that's empirical and certainly sample size will be important.

Apophenia is the experience of seeing meaningful patterns or connections in random or meaningless data.

The term was coined in 1958 by Klaus Conrad, who defined it as the "unmotivated seeing of connections" accompanied by a "specific experience of an abnormal meaningfulness", but it has come to represent the human tendency to seek patterns in random nature in general, as with gambling, paranormal phenomena and religion.

Edith Keeler: What... what on earth is that? Spock: I am endeavoring, ma'am, to construct a mnemonic memory circuit using stone knives and bearskins.