Dispassionate Meditations on Mass Shootings (3)


The Lankford study … is it credible?

Does the United States Have the Most Mass Shootings (1)?

In the previous post I confirmed that the incidents and associated death toll of mass shootings in the United States is on a precipitous increase.  Since we live here this turn of events is at top of mind (e.g., note the recent Times Square panic).  And, since most of our news is about the United States it’s not unreasonable to fear that there is something uniquely wrong here, resulting in a higher incidence of mass shootings than occur elsewhere in the world.

So, when an academic study by Adan Lankford  (University of Alabama) was published showing that the United States is indeed unique (in a bad way), many people’s emotional response was confirmation of their worst fears.  And, since that study was cited by major media organizations and politicians the presumed credibility of these results was confirmed.

But there was something strange about this study.  It turned out that Dr. Lankford’s paper was vague/inconsistent about key issues like definitions and methodology.  He also refused to release the data set upon which his conclusions were based.  This behavior is extremely odd for an academic who speaks on key public policy issues.  The generally expected behavior is clarity on definitions and methodology and complete transparency on the data set.  In this way other researchers have the opportunity to replicate or challenge the findings.

For example, the New York Times story based on the Lankford paper consistently refers to “mass shootings” as the topic under study.  Lankford’s paper uses multiple terms interchangeably, such as “mass shooters,” “rampage shooters” and “active shooters.”  The Lankford paper’s title uses the term “Mass Shooters.”  The New York Times article uses the term “mass shooting” twenty-four times but never mentions “active shooter.”

The definition of a “mass shooting” is uncertain and highly variable as discussed by the Washington Post.

But “mass shooting” is a term without a universally-accepted definition, which complicates news coverage of events such as Sunday’s massacre in Las Vegas.  …

The FBI does not officially define “mass shooting” and does not use the term in Uniform Crime Report records. In the 1980s, the FBI established a definition for “mass murder” as “four or more victims slain, in one event, in one location,” and the offender is not included in the victim count if the shooter committed suicide or was killed in a justifiable homicide, according to a Congressional Research Service report detailing the definitions.

After the 2012 shootings in Newtown, Conn., Congress defined “mass killings” to mean “three or more killings in a single incident.” Some media outlets and researchers still use the four-fatality definition, and have adopted the CRS definitions of “mass shooting” and “mass public shooting.” Other researchers include injuries in the victim count. Some researchers include acts of terrorism, drug deals gone wrong or gang conflict in their research. Others don’t.

Some media reports, such as those of our Wonkblog colleagues, and advocates use a broader definition used by the Mass Shooting Tracker maintained via Reddit, an online forum. In this case, mass shootings are incidents in which four or more people, including the gunman, are killed or injured by gunfire. By this count, the San Bernardino shooting is the 355th mass shooting this year. (In comparison, CRS counted 317 mass shooting incidents from 1999 to 2013.)

Note that none of these definitions place a limit on the number of shooters.  And, the San Bernardino incident in which there were two shooters is explicitly included under any definition of a “mass shooting.”  This distinction is, as we will see, central to assessment of Lankford’s results.

Well into the Lankford paper a definition for the data utilized is provided (emphasis added):

Data for this study were drawn first from the New York City Police Department’s (NYPD) 2012 Active Shooter report. This report employs the Department of Homeland Security’s definition of “active shooter”: “an individual actively engaged in killing or attempting to kill people in a confined and populated area” (Kelly 2012, p. 1). More commonly, these offenders are referred to as rampage shooters or public mass shooters. According to the formal definition, their attacks must have (a) involved a firearm, (b) appeared to have struck random strangers or bystanders and not only specific targets, and (c) not occurred solely in domestic settings or have been primarily gang-related, drive-by shootings, hostage-taking incidents, or robberies (Kelly, 2012). For this study, attackers who struck outdoors were Public Mass Shooters and Firearms included; attackers who committed sponsored acts of genocide or terrorism were not. This is consistent with the criteria employed by the Federal Bureau of Investigation (FBI) in its 2014 active shooter report (Blair & Schweit, 2014).

It turns out that even the FBI acknowledges that the definition of an “active shooter” is uncertain and open to interpretation (emphasis added).

The agreed-upon definition of an active shooter by U.S. government agencies … is “an individual actively engaged in killing or attempting to kill people in a confined and populated area.” Implicit in this definition is that the subject’s criminal actions involve the use of firearms. For purposes of its study, the FBI extended this definition to include individuals, because some incidents involved two or more shooters. Though the federal definition includes the word “confined,” the FBI excluded this word in its study, as the term confined could omit incidents that occurred outside a building. Whether inside or out, these incidents still posed a threat to both law enforcement and the citizens they seek to protect.

So, what we have is a confused situation in which terms with varying definitions are used interchangeably by the paper’s author (Lankford), with the term “mass shooting” used in the paper’s title and “mass shooting” used exclusively by the New York Times in its article based on this paper.

The consequences of this confusion and the author’s complete refusal to answer questions or simply share his data set significantly complicated the process of review and replication that normally occurs for academic research results.

This assessment continues with the next post.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s