Why Most Published Research Findings Are False

Abstract: Ioannidis JPA (2005), PLoS Med 2(8): e124: “There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.” (PLosMedicine… thanks, adam)
Technorati tags:

Counting the Civilian Dead in Iraq

“Thousands of Iraqi civilians have died as a result of violence since the war began in 2003. But it’s not known exactly how many died, where and at whose hands.

There have been several efforts to count the war’s toll on civilians, yielding reports ranging from 24,000 to 128,000 from last fall through last month. Compounding the complexity, all of these numbers were collected differently and count different things, so they aren’t directly comparable. For example, the widely cited number last month of about 25,000 counts only violent deaths that have been reported to the media. Meanwhile, a study conducted last fall that found 100,000 deaths arrived at that figure by calculating ‘excess’ deaths — all deaths, including those from illness and accidents, were included, but deaths from a comparable prewar period were subtracted out.

The uncertain and inconsistent numbers help explain why the civilian death toll — caused by criminals, terrorists, insurgents and soldiers from all sides — hasn’t been given much attention in major U.S. media, even as many newspapers report every death of U.S. soldiers in Iraq and, last December, headlined incomplete tsunami death-toll numbers for weeks.” (WSJ Online)


I was recently pointed to the Wikipedia article on this fascinating Japanese phenomenon. Hikikomori is a Japanese term referring to acute social withdrawal by adolescents or young adults.

“While there are mild and extreme degrees, the Japanese Ministry of Health defines a hikikomori as an individual who refuses to leave their parents’ house, and isolates themselves away from society and family in a single room for a period exceeding six months, though many such youths remain in isolation for a span of years, or in rare cases, decades. Many cases of hikikomori may start out as school refusals, or tohkohkyohi in Japanese. According to estimates by psychologist Saito Tamaki, who first coined the phrase, there may be 1 million hikikomori in Japan, 20 percent of all male adolescents in Japan, or 1 percent of the total Japanese population. Surveys done by the Japanese Ministry of Health as well research done by health care experts suggest a more conservative estimate of 50,000 hikikomori in Japan today. As reclusive youth by their very nature are difficult to poll, the true number of hikikomori most likely falls somewhere between the two extremes.

…Sometimes referred to as a kind of social problem in Japanese discourse, the hikikomori phenomenon has a number of possible contributing factors — young adults may feel overwhelmed by modern Japanese society, or be unable to fulfill their expected social roles as they have not yet formulated a sense of personal tatemae and honne needed to cope with the daily paradoxes of adulthood. The dominant nexus of the hikikomori issue centers around the transformation from young life to the responsibilities and expectations of adult life — indications are that advanced capitalist societies such as modern Japan are unable to provide sufficient meaningful transformation rituals for promoting certain susceptible types of youth into mature roles within society.”

Middle class affluence allows many families to support their isolative child indefinitely. There may be a contribution from the particularly Japanese codependent collusion (amae) between mother and son, making an effective response to the isolationism more difficult. And the decreasing job security in the Japanese corporate environment, combined with extreme performance pressure, may contribute to making social withdrawal rampant.

There appears to be considerable debate about whether hikikomori should be considered a sociological or psychological phenomenon. A variety of psychological diagnoses could contribute to its phenomenology, including anxiety disorders including agoraphobia and social phobia (social anxiety disorder); avoidant personality disorder; and depression. I think the social vs. psychological debate is a specious one, as there is likely a coalescence of internal and social factors at play here. There are a number of so-called “culture-bound syndromes” in which a behavioral symptom pattern appears to be particularly prevalent, and unique, in a given culture. These include amok, latah, wendigo, etc. I usually see them as variants of common psychiatric disturbances brought to the fore by the particular social stresses of a given culture.