Discriminatory design

Identifying discriminatory design

One goal of ME 30 is to empower you to use tools of electronics and robotics to design new technologies. As we wrap up this semester, we acknowledge that with that power comes much responsibility, including the responsibility to understand how robotic systems can negatively impact some individuals and groups even as they positively impact the lives or work of others — and the responsibility to learn how to proactively avoid this kind of discriminatory design.

Luckily we have sociologist Ruha Benjamin, a professor in Princeton’s Department of African American Studies, to help us. Benjamin’s scholarship uncovers the ways in which racial discrimination and marginalization are designed into technology — often unintentionally, but also at times intentionally. If you’re interested in learning more about Dr. Benjamin’s work, you can view her 2019 talk on discrimination and algorithms here, and her 2015 TEDx Talk “From Park Bench to Lab Bench: Whose future are we designing” here. (These videos are optional resources. You do not need to watch these videos for this assignment.)

For Video Response 11, you’ll be reading an excerpt from Dr. Benjamin’s 2019 book Race After Technology: Abolitionist Tools for the New Jim Code. Each chapter of this book includes a case study of a technology that produces (or reproduces) racist outcomes. Our excerpt, from Chapter 1, motivates the need for race-conscious design (pp. 59-63) with a case study of an automated hotel soap dispenser (pp. 64-69).

Questions for your Video Response (choose one)

For VR 11, read the excerpt (posted in the Flipgrid assignment). Then choose one of the following questions to answer. If you don’t feel comfortable answering in video form, it’s fine to type your answer and submit it as an “Attachment Link” on Flipgrid instead.

Option 1: Research shows that it can be easier to talk about race when we work to be more aware of the emotions that such discussions generate in ourselves.
What was your emotional state like after you read the opening question, “So, are robots racist?” And how did your emotional state evolve over the course of your reading of the chapter?

Option 2: Benjamin calls on engineers to learn how to be race-conscious designers because “colorblind” engineering perpetuates racist realities. She asserts that “colorblind, gender-neutral, and class-avoidant approaches to tech development are another avenue for coding inequity.” (p. 63). How does the soap dispenser example (explained most directly on p. 68) illustrate this point?

Option 3: Benjamin disagrees with the assumption that “self-conscious intention is what makes something racist” (p. 60). On the contrary, she points out, a machine can be characterized as racist if it produces racially discriminatory effects, no matter what the designers’ intention. What does Benjamin mean when she writes that “so much of what is routine, reasonable, intuitive, and codified reproduces unjust social arrangements” (p. 60)? Discuss another technology (other than the problematic soap dispenser) that illustrates this point.