So glad to see Nate Silver weighing into the mix on the validity of the DADT survey that the military is distributing to the troops (at a cost of $4.4 million). His conclusion: it’s fairly useless – hardly asking the relevant questions and therefore yielding dubious conclusions.
Rather than getting an assessment about the troops’ impressions of the DADT policy, the survey is mainly aiming to test specuation about homosexuals serving in the military, with questions geared toward whether troops believe there are gays among them. Nate identifies a couple of disturbing biases that this could yield:
What further seems likely is that the distribution of false positives will be biased. In particular, it seems likely to be biased in two ways:
— Soldiers in units with low morale are probably more likely to accuse one another of being gay based on hearsay evidence, i.e. rumors or innuendo. Although this might be changing some, there’s usually no better way for one young man to undermine another than to allege that he is gay. And these (usually spurious and false) accusations would presumably be more common in units where the troops weren’t getting along with one another. Thus, the survey will probably attribute lower morale to the presence of homosexuality within the unit, when in fact it’s low morale that triggers the suspicion (but not the actual fact) of homosexuality. [emphasis mine]
— Homophobic soldiers are probably more likely to accuse one another of being gay based on stereotypes. Our troops range from relatively worldly young men and women to others who are teenagers and who have barely seen the world outside of their hometowns. They might mistake behaviors (one’s choice of music, for instance) that in fact reflect socioeconomic or cultural differences for instead being indicators of sexual orientation. Or their homophobia might be less benign: they see gays lurking around every corner. If they don’t get along with their commander, for instance, they might put one or two facts together (he isn’t married and likes watching Sex and the City!) and conclude that must be gay.
I added the emphasis because I think it is a salient point about a very negative implication of distributing a survey like this that actually has adverse policy consequences. Biases that undermine the integrity of the survey are one thing, but biases that yield results that could lead to perverse policy prescriptions are especially onerous. Indeed, signs seem to point to the survey being intrinsically inclined to make overturning DADT more difficult, for spurious reasons.