James Scanlan is a Washington attorney specializing in the use of statistics with respect to employment discrimination litigation and compliance. He has forwarded a copy of the letter he has submitted to Minneapolis Mayor Betsy Hodges and Chief of Police Janeé Harteau regarding the recent American Civil Liberties Union Minnesota study of the racial impact of Minneapolis policing practices.
I have referred to the ACLU study at several points in this series; John wrote about it at length here. Mr. Scanlan observes that the ACLU Minnesota study has the same misconception as the Department of Justice Ferguson disparities report (and a vast array of other things written about racial disparities). Mr. Scanlan’s home page and related materials are accessible here.
In formatting the letter for this post, I have bracketed footnote numbers in the text and appended the notes to the bottom of the letter. Below this introduction is Mr. Scanlan’s letter on the ACLU Minnesota study to Mayor Hodges and Chief Harteau. Members of the Minnesota media who may wish to follow up with Mr. Scanlan are invited to contact us at [email protected] Mr. Scanlan writes:
June 8, 2015
The Honorable Betsy Hodges, Mayor
Janeé Harteau, Chief of Police
City of Minneapolis
Minneapolis City Hall
350 S. 5th Street Minneapolis, MN 55415
Re: Misunderstanding in ACLU Minneapolis Arrest Disparities Study of the Relationship of the Frequency of Arrests to (a) Relative Racial Differences Between Arrest Rates and (b) Proportion African Americans Comprise of Persons Arrested
Dear Mayor Hodges and Chief Harteau:
On occasion I write to institutions or organizations whose activities involve the interpretation of data on demographic differences in the law or the social or medical sciences alerting them to ways in which their interpretations are undermined by the failure to understand patterns by which standard measures of differences between favorable or adverse outcome rates of advantaged and disadvantaged groups – or differences between the proportion a group comprises of persons potentially experiencing an outcome and the proportion it comprises of persons actually experiencing the outcome – tend to be systematically affected by the overall frequency of an outcome. Recipients of other letters involving issues of the type discussed in this letter include Robert Wood Johnson Foundation (Apr. 8, 2009), National Quality Forum (Oct. 22, 2009), Institute of Medicine (June 1, 2010), The Commonwealth Fund (June 1, 2010), United States Department of Education (Apr. 18, 2012), United States Department of Justice (Apr. 23, 2012), Board of Governors or the Federal Reserve System (Mar. 4, 2013), Harvard University (Oct. 9, 2012), Harvard Medical School, Massachusetts General Hospital, et al. (Oct. 26, 2012), Mailman School of Public Health of Columbia University (May 24, 2013), Investigations and Oversight Subcommittee of House Finance Committee (Dec. 4, 2013), Education Trust (April 30, 2014), Annie E. Casey Foundation (May 13, 2014), Institute of Medicine II (May 28, 2014), IDEA Data Center (Aug. 11, 2014), Education Law Center (Aug. 14, 2014), Financial Markets and Community Investment Program, Government Accountability Office (Sept. 9, 2014), Wisconsin Council on Families and Children’s Race to Equity Project (Dec. 23, 2014), Portland, Oregon Board of Education (Feb. 25, 2015), Vermont Senate Committee on Education (Feb. 26, 2015), United States Department of Justice and City of Ferguson, Missouri (Mar. 9, 2015), Senate Committee on Health, Education, Labor and Pensions II (Mar. 20, 2015), and Texas Appleseed (Apr. 7, 2015). An amicus curiae brief I filed on November 17, 2014, in Texas Department of Housing and Community Development, et al. v. The Inclusive Communities Project, Inc., Sup. Ct. No. 13-1371 (TDHCD brief), might be deemed a similar communication to the United States Supreme Court.
This letter is prompted by the May 29, 2015 release of an American Civil Liberties Union (ACLU) study on the racial impact of Minneapolis police practices, “Picking Up the Pieces, Policing in America, A Minneapolis Case Study,” as well as actions city officials have taken or are considering taking in response to the study. A premise of the study is that high frequency of arrests for less serious offenses causes large relative racial differences in arrests for such offenses and causes African Americans to comprise a much higher proportion of persons arrested than they comprise of the city’s populations.
That premise, though apparently unchallenged in discussions concerning the ACLU study, is the exact opposite of reality. Reducing the frequency of arrests for less serious offenses will tend to both (a) increase relative racial difference in arrests for such offenses and (b) increase the proportion African Americans comprise of persons arrested for such offenses.
The point is illustrated in Table 1 below, which is a variation on Table 1 (at 9) of the above-referenced TDHCD brief, Table 1 (at 329) of my recent article “Race and Mortality Revisited,” Society (July/Aug. 2014) (a fairly comprehensive treatment of the pertinent statistical issues and the implications of the widespread failure to understand them), and Table 2 (at 29) of a September 5, 2014 methods workshop given to the demography and epidemiology arms of the University of Minnesota (which contains many graphical and tabular illustrations of the issues). The data in the table also underlie the hypothetical illustrations of the key statistical point in a number of recent, more succinct treatments of the subject, including a Minneapolis Star Tribune commentary regarding misperceptions about other racial differences in Minnesota. See “It’s easy to misunderstand gaps and mistake good fortune for a crisis,” Minneapolis Star Tribune (Feb. 8, 2014), “Things government doesn’t know about racial disparities,” The Hill (Jan. 28, 2014), “The Paradox of Lowering Standards,” Baltimore Sun (Aug. 5, 2013), and “Misunderstanding of Statistics Leads to Misguided Law Enforcement Policies,” Amstat News (Dec. 2012).
The table is based on normally distributed test scores where the mean scores of the higher-scoring group (AG, for advantaged group) and the lower -scoring group (DG, for disadvantaged group) differ by half a standard deviation. Moving from the first to the second row one can observe the implications of lowering a test cutoff from the point where 80% of AG passes to the point where 95% of AG passes. The table shows that lowering the cutoff, thereby increasing overall pass rates while reducing overall failure rates, decreases the relative difference in pass rates and increases the proportion DG comprises of persons who pass the test. But lowering the cutoff also increases the relative difference in failure rates and increases the proportion DG comprises of persons who fail the test.
Table 1. Illustration of effects on relative differences in pass and fail rates of lowering a cutoff from a point where 80% of AG passes to a point where 95% of AG passes, with proportions DG comprises of persons who pass and of persons who fail (when mean scores differ by approximately half a standard deviation and DG comprises 50% of test takers).
The pattern shown in the table is not peculiar to test score data or the numbers I chose to illustrate it. Rather, inherent in the shapes of other than highly irregular risk distributions is a pattern whereby the rarer an outcome the greater tends to be the relative differences in experiencing it and the smaller tends to be the relative difference in avoiding it (a corollary to which is a pattern whereby the rarer an outcome the greater tends to be the proportion groups most susceptible to the outcome comprise of persons experiencing the outcome and of persons avoiding the outcome). The pattern can be illustrated with virtually any data where one is able to observe rates at which advantaged and disadvantaged groups fall above and below various points on a continuum of values associated with experiencing an outcome or its opposite. And it is found in a wide range of data on actual situations where the frequency of an outcome changes over time or varies from setting to setting, as shown by the many examples in “Race and Mortality Revisited” and its references as well as the University of Minnesota workshop.
Some recent examples that are particularly pertinent to the local situation may be found in the Minneapolis Disparities, St. Paul Disparities, and Minnesota Disparities subpages of the Discipline Disparities page of jpscanlan.com. These pages discuss data showing that recent reductions in public school discipline rates in the referenced jurisdictions were accompanied by increased racial/ethnic differences in discipline rates. This would necessarily mean that the proportion groups with higher discipline rates comprise of students who are disciplined increased.
The points addressed above and in the referenced materials do not pertain merely to issues about whether observed differences should be attributed to the frequency or infrequency of an outcome and whether particular policies should be deemed to have large or small racial impacts. The points also pertain to perceptions about the strength of the forces causing outcome rates of advantaged and disadvantaged groups to differ with respect to the way such perceptions influence interpretations regarding the nature of those forces. Thus, for example, it must be understood that there exists no rational basis for maintaining that the strength of the forces causing pass or fail rates of AG and DG to differ varies between the two rows in Table 1 above (or among the four rows in Table 5 (at 335) of “Race and Mortality Revisited,” Table 2 of the TDHCD brief (at 15), and Table 15 (at 56) of the University of Minnesota workshop). Further, any inference one might draw about underlying processes, whether based on the larger relative difference in the favorable outcome in the first row of Table 1 or the larger relative difference in the adverse outcome in the second row of the table, would lack a statistical foundation. With respect to the mistaken drawing of such inferences, see especially “Race and Mortality Revisited” (at 339-341), the letter to Harvard University mentioned in the first paragraph (at 40- 41), and “The Perverse Enforcement of Fair Lending Laws,” Mortgage Banking (May 2014) (at 92-93). More generally regarding discrimination issues, see my September 13, 2013 University of Kansas School of Law Faculty Workshop paper “The Mismeasure of Discrimination” and January 20, 2015 University of California, Irvine Methods Workshop “The Mismeasure of Discrimination.”
Academics in the Minneapolis area [who] are knowledgeable about these issues include Professor J. Michael Oakes of the Department of Epidemiology and Community Health of the University of Minnesota’s School of Public Health and Professor John Robert Warren of the University’s Department of Sociology (who together organized the September 2014 methods workshop mentioned above). Professor Milo Shields, Director of the W.M. Keck Statistical Literacy Project of Augsburg College, is also knowledgeable about these issues, being familiar with the March 9, 2015 letter to officials of the Department of Justice and the City of Ferguson, Missouri mentioned in the first paragraph. That letter addresses a closely related matter respecting misperceptions underlying the Department of Justice’s March 4, 2015 report on the racial impact of police and court procedures of Ferguson, Missouri.
/s/ James P. Scanlan
James P. Scanlan
 To facilitate consideration of issues raised in letters such as this I include links to referenced materials in electronic copies of the letters. All such letters may be found by means of the Institutional Correspondence subpage of the Measuring Health Disparities page of jpscanlan.com.
 “The Mismeasure of Association: The Unsoundness of the Rate Ratio and Other Measures That Are Affected by the Prevalence of an Outcome.” Methods Workshop at Minnesota Population Center and Division of Epidemiology and Community Health of the School of Public Health of the University of Minnesota (Sept. 5, 2014)
 The item was titled “The proof is in the pattern” in the Star Tribune print edition.
 Some older works of particular pertinence to the instant situation include “An Issue of Numbers,” National Law Journal (Mar.5, 1990), and “The Perils of Provocative Statistics,” Public Interest (Winter 1991) (regarding, inter alia, the mistaken belief that the stringency of National Collegiate Athletic Association academic eligibility standards was responsible for the high black representation among persons disqualified by the standards), and “Mired in Numbers,” Legal Times (Oct. 12, 1996) (regarding, inter alia, the fact that changing a three-strikes rule to a four-strikes rule would increase the racial disproportionality in application of the rule).
 The illustration is cast in terms of (a) the ratio of AG’s pass rate to DG’s pass rate and (b) the ratio of DG’s fail rate to AG’s fail rate. The relative differences would be the rate ratios (also termed risk ratios) minus 1.
 Some recent treatments of the pattern by others include Mackenbach JP (Should we aim to reduce relative or absolute inequalities in mortality? Eur J Pub Health 2015;25(2):185), Lambert PJ and Subramanian S (Disparities in Socio-Economic outcomes: Some positive propositions and their normative implications. Soc Choice Welf 2014;43:565-576), and Lambert PJ and Subramanian S (Group inequalities and “Scanlan’s Rule”: Two apparent conundrums and how we might address them. Working Paper 84/2014, Madras School of Economics (2014)).
 Other subpages of the Discipline Disparities page discuss like situations in jurisdictions across the country.
￼￼￼￼￼￼￼￼￼￼￼￼￼￼￼￼￼ The letter to Texas Appleseed mentioned in the first paragraph, which involves a study reflecting a misunderstanding of the relationship between the frequency of truancy proceedings in Texas courts and the proportion disadvantaged groups comprise of person subject to such proceedings, is also closely related to the subject of this letter. All the letters mentioned in the first paragraph, however, address the key statistical point.