"How the Fuck Did This Get Published?" might become a regular feature on this blog since I often end up asking myself that question while reading source material related to whatever I'm writing about.
Today, it is
this article (and the
study referenced within) on Huffington Post. As you might imagine, I am not the biggest fan of HuffPo. Although it is (arguably) at the top of the hierarchy of click-bait garbage (Buzzfeed<UpWorthy<Thoughtcatalog<Gawker<HuffPo), it is still click-bait garbage; this article is no exception with a headline of "Nearly One-Third of College Men In Study Say They Would Commit Rape" when the "study" consisted of a self-report questionnaire given to a convenience sample of only
86 men. That is a sample size and collection method of an undergraduate research project, not a study published in a peer-reviewed journal. The fact that
anyone is trying to draw larger conclusions about even the student population from which the sample was drawn shows a fundamental lack of understanding of statistics and the scientific method.
I'm half watching the Green Bay game right now so I'm not really too interested in good transitions. So I'll just go right into explaining my problems with this study. This will make more sense if you read along with me.
INTRODUCTION
The first sentence of the introduction cites the dubious "one in five" statistic. Are you fucking kidding me? It doesn't even cite it correctly. The
citation is in reference to the second half of the sentence, which is that the majority of sexual assaults are committed by people who are known by the victim (which is true and supported by data). The full version of the "one in five" statistic (see some of my previous posts for full explanation of why it is bullshit) is that about 19% of college-age women will be the victim of
attempted or completed sexual assault during college, with the majority of that 19% consisting of the "attempted" variety, which included vague situations such as "unwanted touching", which could be anything from an unwanted kiss (example: go in for a kiss after a date, she says no, you stop) to getting groped in a crowded place. Furthermore, this isn't even a study; it is a factsheet compilation of many different studies. If you want to quote a quantitative figure like that, you better cite the exact study from which it came.
Further down in the introduction, this passage:
"For example, commentators have suggested that rape only occurs if a woman labels it as such, while experiencing an unwanted sexual experience under the influence does not constitute rape. This perspective demonstrates stereotypical assumptions many people still hold about rape, considering it to be an act of extreme violence perpetrated by an unknown assailant upon an unsuspecting woman who is willing to defend her sexual purity with her life. Such sentiments contribute to a culture that continues to put at least partial responsibility for sexual assault on the victim and normalizes sexual aggression as part of the male gender role."
I cannot read this without seeing
<-- WHO???? CITE!?!? angrily scrawled above "commentators". Which is exactly what would be written had this been turned in for a grade. Furthermore, if you are going to call something a "stereotypical assumption", why not cite an example? If it is that widespread, there should be plenty of studies demonstrating how widely held the belief is. Also, since normalization of "sexual aggression as part of the male gender role" is a central research question examined by this study, why are you assuming it already exists rather than presenting it as a hypothesis? Also, WHERE IS THE CITATION ANYWHERE IN THESE STATEMENTS?
The next section explains some of the differences that can arise when asking participants virtually identical questions with different wordings. This has been demonstrated in studies dating back to the 1960s and 1970s when the field of quantitative psychology started to gain significance with their focus on researching the research methods themselves. "Sexual assault" sounds less harsh and carries less stigmatizing connotations than "rape". But here are where semantics get in the way of clarity- all rapes are sexual assaults, but not all sexual assaults are rape. So although "rape" does indeed carry a more stigmatizing connotation, it is also naturally going to be reported less because it is a more specific offense than "sexual assault".
Callous Sexual Attitudes and Hostility Towards Women
I will try to avoid going down the rabbit hole here, for unless you're like me and nerd out over research methods, you probably won't care. I will say that I think the experimental operationalization of these two behavioral constructs is sound. The expected links between callous sexual attitudes (CSA) with coercive behavior and hostility towards women (HTW) with aggressive behaviors respectively makes sense, however I don't entirely buy into the incorporation of hypermasculinity as a key variable, especially since the CSA is only one subsection of a larger measure for hypermasculinity. The hypermasculinity measure contained numerous subscales, which when considered together gave an aggregate indication of hypermasculinity. Basing a measure of hypermasculinity on this subscale alone is like coming to the conclusion that anything with four wheels is an automobile (A-->C), and anything with an engine is a automobile (B-->C), both of which are untrue statements if an automobile's distinguishing characteristic from other vehicles is defined as having four wheels powered by an engine (A&B-->C).
I'll take a minute here to distill the hypothesis, because it isn't really clearly stated (more red ink). The researchers think that the results will separate the men into three distinct groups;
- Men who express no intentions to use sexually aggressive behavior; strongest variable relationship between individuals in this subset would be a low score on the HTW scale
- Men who openly endorse intentions to rape women
- Men who endorse intentions to use force/coercion but deny intentions to rape; difference between this group and the group above is expected to be found in the CSA scale- this group will score higher than group above
Materials and Methods, or Where I Completely Lose My Shit
This study consisted of 86 heterosexual men. With a sample size that small, there is no way that this study should have
ever been published in a peer-reviewed journal, especially since it is defined as a convenience sample. Do you know when you resort to a convenience sample? When shit hits the fan and you accept the fact that your study is going nowhere. I don't know how University of North Dakota runs things, but at UNC, PSYC 101 students are required to participate in a certain number of research studies for completion of the course. This is a common practice at many other schools as well. The small sample size coupled with the fact that the participants were offered extra-credit for their participation and "most" were juniors tells me that participants were most likely drawn from students enrolled in a
single course, possibly a
single section of a single course. Additionally, the measures used are self-report surveys, which are the weakest form of data collection. So why such a shitty sample? Most likely because the IRB didn't approve the study for use on the PSYC 101 participant pool (if it exists) for any of the reasons that I've described so far.
To fully understand why such a small sample size should pretty much invalidate the whole study, read
this. If you are going to try to use the results of a study to make conclusions about a larger population (which is arguably
the whole fucking point of doing research in the first place), you need to have a certain sample size for those conclusions to be valid. According to
Wikipedia, UND has an enrollment of 15,000. I don't feel like finding the demographic data, but let's just assume it is 50% male, 50% female. That would make your total population of male college students at UND about 7,500. To conclude anything about the population from the sample, you would need to have a sample size of around n=365, which is about four times larger than this study's n=86.
EDIT: The study had 86 total participants. I originally read that only one participant's responses were excluded. I didn't mention it because it is standard practice to exclude responses from participants who didn't complete all items on the survey. However, upon further examination, a total of 13 were excluded from analysis, thus the results only represent 73 individuals.
Furthermore, the decision to sample only males is curious for a few reasons. Firstly, the researcher most likely falls into the completely unscientific school of thought (thanks third-wave feminism!) that gender is a social construct, so why are characteristics of "hypermasculinity" only being examined along the heterosexual cis-male spectrum? Since gender is ostensibly completely independent from biological sex, why aren't females also included in the sample? While I don't believe gender is entirely a social construct, individuals' chosen form of expression of their gender role is very much constructed; some men behave more "feminine" than others, some women behave more "masculine" than others. Notions of masculinity and femininity have a basis in biological differences, but the behavioral manifestation is a result of culture. So if you want to examine how increased masculine traits may lead to sexually aggressive/coercive behaviors, this finding would be even more supported if women were included in the sample.
Also, the researcher is interested in semantics and word choice- for example the difference created by a behavioral description of "rape" versus a direct question addressing "rape". According to the makeup of the sample, I guess only cis-males are capable of being rapists. This is ridiculous, especially considering the definition of rape presented in the paper's introduction. Both men and women are equally capable of sexually coercive behaviors, though I would say men are much more likely to exhibit aggression. But these differences are important and not widely studied. Again, women should be included in the sample, if for no other reason a control group.
Results/Discussion/Conclusion
Not the statistical analysis I would have used, but I guess it works. I'm just worried that the method used increases Type I error rate by lumping all the variables together.
All you need to know about this is that under the "Limitations/Future Research" section, the researchers do not mention the sample even ONCE. Yet again, imagine me vomiting red ink all over this. You ALWAYS talk about the sample because
there is no such thing as a perfect sample. The First Law of Research is "Shit Happens". In fact, that is just about every law of research when it comes to running a study. Eighty-six is an exceptionally small sample. Furthermore, a sample entirely composed of straight men (90% of whom identified as Caucasian) doesn't exactly scream "representative of the population". This section is where you acknowledge its shortcomings and suggest how future research can improve on the current study (larger sample, include women, etc). It is a vitally important practice in science because it enables others to replicate and expand upon current findings. So as a short list, future studies should examine if there are better ways to measure the variables, should have a much larger sample size composed of men and women, and possibly include a variable of whether or not participants have already received some sort of educational intervention on sexual assault and whether or not that influences their responses. Are any of these things suggested? Nope. Not a single one of them. I came up with those in the amount of time it took to type them. This is basic stuff if this is what you were trained to do. But there is nothing of that nature here.
Well, what
is suggested?
Mandatory gender and sexuality studies classes for male college students.
How the fuck did this get published?
