Tuesday, July 26, 2011

Thanks for the Questions!

Dear Readers,

Aunt Louise has kicked the following question over to Madame L, who thanks her, Jeff, and Josie and Wayne. The questions are:

Isn't it true that 78% of statistics are made up on the spot? I've read elsewhere that Utah (particularly the women) have the highest rate of depression. Do you have a reliable source on that? I do see where you are coming from though and I can definitely see how Utah should have the lowest rates of all those things you mentioned.

 
Dear Jeff, Aunt Louise, Josie and Wayne, and All Dear Readers,
 
Madame L, who is also rolling on the floor laughing, guesses that 100% of that statistical question was made up on the spot!
 
Thanks for asking, because it points out the difficulty many of us have when faced with statistics: Should we believe them? Where did they come from? And, come on, you just made that up, didn't you? Didn't you?
 
Madame L hopes you'll be reassured to learn that, in her experience, most statistics that we see in scientific research are accurate and truthful. Problems arise when the questions we want answered are not the ones the statistics purport to answer; when researchers are being paid by drug companies or political pollsters, for instance, to throw out the negatives and give undue importance to any positive results, no matter how paltry or unconvincing; and when statistics are portrayed graphically in a misleading fashion.
 
Here are some examples of graphically misleading statistics:
 
First, an example of a statistical answer to a question that was worded misleadingly: 
 
This clipping from US News and World Report on 1/29/01 suggests that Alaskans are terrible parents.  Is this true? 


Answer: No. The difference in the abuse rates probably stems from different definitions for abuse in the various states.  For example, Alaska (the "worst" state) says that a child is abused if his or her health or welfare is harmed or threatened.  Pennsylvania (the "best" state) defines it as a recent act or failure to act.

Here's an example of a graph that seems to show something which is not at all true: 

The following statistics suggest that 16-year-olds are safer drivers than people in their twenties, and that octogenarians are very safe.  Is this true?

Is this true?
Answer:  No.  As the following graph shows, the reason 16-year-old and octogenarians appear to be safe drivers is that they don't drive nearly as much as people in other age groups. 

Now, to get to your second question, about depression rates in Utah compared to other states. Madame L has also seen statistics about this issue, and Madame L is very sorry to say that she believes these "statistics" are inaccurate and misleading, and that some of them are being propounded by hateful anti-LDS groups.
 
The reasoning goes: Mormons should be perfectly happy since they claim they have some kind of inside track to God. But they're not uniformly and perfectly happy, so there must be something wrong with their religion. And then the finger-pointing and name-calling begin. In fact, when Madame L Googled "statistics Utah depression," her first hits included several websites which specialize in doing that kind of thing.
 
Even USA Today, another quick hit from this search, reports a 2007 result of a study claiming Utah has the highest depression and among the highest suicide rates. The Deseret News reported these results, too, but with a thoughtful and interesting commentary on possible reasons for the results.
 
Some BYU professors responded in 2010 to these statistics, pointing out that LDS people are not self-medicating with alcohol and illegal drugs, but seeking professional medical and counseling help, which may raise the statistical average. Also, these professors pointed out that because LDS people are better educated than the average people in the average non-LDS-majority state, they may recognize symptoms of depression earlier and be more willing to seek help than others.
 
Madame L likes these points, but she would go further: Madame L would like to see how "depression" is reported and recorded in all the states. As in the above examples, it is not only possible but is quite likely that the way that depression results are reported in Delaware, say, as opposed to Utah, may make a difference in the results.
 
Also, Madame L checked to see how this study was funded, and guess what! The research was funded by a grant from Wyeth Pharmaceuticals, one of the biggest drug companies in the world. This disinclines Madame L to trust the results. Madame L wouldn't trust these results, in fact, unless she could personally look at the results for each state, what questions were asked and of whom, how they were asked, what each state defines as clinically depressed, how results were reported, and so on.
 
Madame L teaches her own students that, when reading any scientific or medical research, one should disregard the Abstract, Introduction, and Discussion sections. One should instead scrutinize very closely the Methods and Results sections. There one will find how the study was conducted, what statistical parameters were applied, and what biases were noted by researchers. One will thereby be able to judge for oneself the truth of the reported results.
 
Madame L emphasizes to her students that science and medical news reporters operate under deadlines and under depressing (pun intended!) requirements to explain complicated concepts in simple words, both of which factors often lead to inaccurate and misleading reporting.
 
Madame L also advises her students, and you, Dear Reader, to be careful when talking to doctors or counselors about feelings of sadness or depression --- and to be careful when listening to their responses. They are often under time and financial constraints, as well as direct and indirect pressure to diagnose depression and to prescribe drugs to treat said depression, drugs which more recent, non-pharmaceutical-funded research, has shown to be largely ineffective except as placebos.
 
Madame L wouldn't go so far as to say Tom Cruise was right about anti-depressants, but she would like to point out to all her readers that anti-depressant studies are notorious for suggesting that drugs are needed which later turn out to be ineffectual and sometimes dangerous, sometimes even leading the patient to suicidal thoughts and alarming weight gains.
 
Finally, Dear Jeff, Aunt Louise, Josie and Wayne, and All Dear Readers, Madame L hopes that if any of her answers have been unclear in any way, you will ask more questions.
 
Take care,

Madame L

3 comments:

Josie and Wayne said...

Thanks!

AskTheGeologist said...

This is wonderful! I learned a lot from that first link... like (#7): I will buy a bunch more guns. No. Wait. That means I'm just scared and paranoid. Wait. What if I just scream at the neighbors across the street? Wait. No! Then they'll just blast me! Wait...

Statistically, I think I've just proven I'm a loony, I am. Huh? Me?
~~~~~

AskTheGeologist said...

En serio, there is one line in that message that bears repeating: "One should instead scrutinize very closely the Methods and Results sections."

And I'll add to that: NEVER trust a study paid for by pharmaceutical companies. Too many times money has trumped science - ALWAYS look to see who benefits from any "statistical" study. Are they motorcycle clubs? Pharmaceuticals? Health Insurance companies? Karl Rove?

If you don't have time or energy to sort the details out, then find someone without a dog in the fight who DID*. There's not much left that one can trust in this world when people manipulate science for gain.
~~~~~

* Madame Elle has made it her business to sleuth out the honest, analytical web-sites. Perhaps her Dear Readers can beg her to share some of these? I trust HER.
~~~~~