8.1 Get to know the data

Eyewitness testimony is an important tool for establishing whether a suspect may or may not have committed a crime. If an eyewitness identifies a suspect as the person they saw at the scene of a crime, this is usually taken as strong evidence of the suspect’s potential culpability. These identifications are usually done within a lineup, where the eyewitness is shown a suspect among several other non-suspects known as “fillers”. If the witness identifies the suspect as the person they saw, this information can be used to investigate that person and to provide evidence in court.

Despite the importance of eyewitness testimony, human memory is far from perfect. It can be difficult for people to (a) remember details of a person’s face in the midst of a stressful event like experiencing a crime; and (b) identify whether a suspect matches what they remember even when people are encountered in different settings (e.g., different light conditions, different clothes, different facial hair/makeup, etc.). Because of these difficulties, eyewitnesses will sometimes make false identifications. Moreover, subtle details of how the police present the lineup to the witness may influence how accurately they can pick out a suspect. As a result, there is considerable interest in understanding how accurate eyewitness testimony is and how best to conduct police lineups to minimize false identifications.

The data for this session were collected as part of a long-term study of the use of police lineups (Wixted et al., 2016). The study was conducted in the field in 2013, in the Robbery Division of the Houston (Texas) Police Department. The participants were eyewitnesses in various robbery cases. For each case, the investigating officer constructed a lineup consisting of six photographs. One photograph was of the suspect in the case, the remaining five were fillers, defined as non-suspects who still match the physical description of the suspect. The lineups were shown either simultaneously (all six photos at once) or sequentially (one photo at a time).

Examples of simultaneous and sequential photo lineups (Gepshtein et al., 2020).

It is important to note that the investigators who showed these lineups to the eyewitnesses were blind as to which person in the lineup was the suspect. As a result, the investigators would not be able to systematically bias the identifications that witnesses made.

Here are the first few rows of the data, to give a sense of what it looks like:

Each row represents the response from a single witness to a single six-person photo lineup. There are just two variables:

• presentation: Whether the six photos were presented to the witness in “Sequential” or “Simultaneous” format.
• ID: Whether the witness identified the “Suspect” as being the person they saw, or whether they identified one of the “Filler” people instead.

8.2 Confidence interval (bootstrapping): How accurate are eyewitnesses?

An important thing to keep in mind is that even if the witness identified the suspect in the lineup, that does not necessarily mean the suspect is guilty or even that they were present at the scene of the crime. On the other hand, identifying one of the fillers is definitely a mistake, since these people are known to have nothing to do with the crime being investigated.

lineup %>%
    specify(___) %>%
    calculate(___)

boot_prop_id_dist <- lineup %>%
    specify(___) %>%
    generate(___) %>%
    calculate(___)

boot_prop_id_ci <- boot_prop_id_dist %>%
    get_confidence_interval(level = 0.95)

boot_prop_id_dist %>%
    visualize() +
    shade_confidence_interval(boot_prop_id_ci)

8.3 Hypothesis test 1 (normal distribution): Are eyewitnesses better than chance?

In the previous exercise, you constructed a 95% confidence interval for the proportion of suspect ID’s from photo lineups. It is important to consider whether this proportion is higher than would be expected by chance. If eyewitnesses were merely picking a person at random from the lineup, they would still pick the suspect some of the time just by chance. We can frame this as a research question that can be tackled with a hypothesis test.

null_prop_id <- ___
sample_size <- ___
obs_prop_id <- ___

null_se <- sqrt(null_prop_id * (1 - null_prop_id) / sample_size)

z_score <- (obs_prop_id - null_prop_id) / null_se

pnorm(z_score, lower.tail = FALSE)

8.4 Hypothesis test 2 (permutation): Is there a difference between sequential and simultaneous lineups?

Recall that the lineups in this study were presented in one of two ways: either simultaneously or sequentially. Laboratory studies of eyewitness memory (using mock crimes instead of real ones) have found that simultaneous lineups tend to produce more “correct” identifications of the suspect. Will the same be true in the real lineups we have been analyzing?

obs_diff <- lineup %>%
    specify(___) %>%
    calculate(___, order = c("Sequential", "Simultaneous"))

null_diff_dist <- lineup %>%
    specify(___) %>%
    hypothesize(___) %>%
    generate(___) %>%
    calculate(___, order = c("Sequential", "Simultaneous"))

null_diff_dist %>%
    visualize() +
    shade_p_value(obs_stat = obs_diff, direction = ___)

null_diff_dist %>%
    get_p_value(obs_stat = obs_diff, direction = ___)

8.5 Hypothesis test 3 (permutation): Does it matter if police know who the suspect is?

Even though these lineups were conducted in the field, they were conducted in a well-controlled setting where the police investigator who assembled and presented the lineup was not working on that same case. As a result, the investigator was “blind” in the sense that they did not know which of the pictures was the suspect. This was to make sure that the investigator would not accidentally or intentionally bias the witness toward identifying the police’s suspect.

What if the investigator who prepared the lineup was involved in the case and therefore knew who the suspect was? The photos would still be shuffled prior to presenting them to the witness, so the investigator still wouldn’t know which position the suspect is in. In a simultaneous presentation, the suspect could appear in any location and in a sequential presentation, the suspect could appear at any spot in the order. Even so, because the investigator is working the case, they might choose filler photographs in such a way as to make their suspect “stand out”, biasing the witness to pick that person.

To study how much this matters, the authors of the study prepared a set of “mock” lineups. In these cases, real photo lineups were presented to people who were not witnesses to the crime. As a result, these people could only guess who the suspect was, since they had no information about the crime. The authors of the study compared lineups that were prepared either by the investigating officer (“Not blind”) or by an officer unfamiliar with the investigation (“Blind”—this is the type of lineup we analyzed above).

This is what the first few rows of the resulting data look like, which are stored under the name fairness:

There are two variables:

• condition: Whether the lineup was “Blind” or “Not blind”
• ID: Whether the mock witness picked the “Suspect” or a “Filler”.

obs_fairness_diff <- fairness %>%
    specify(___) %>%
    calculate(___, order = c("Not blind", "Blind"))

null_fairness_diff_dist <- fairness %>%
    specify(___) %>%
    hypothesize(___) %>%
    generate(___) %>%
    calculate(___, order = c("Not blind", "Blind"))

null_fairness_diff_dist %>%
    visualize() +
    shade_p_value(obs_stat = obs_fairness_diff, direction = ___)

null_fairness_diff_dist %>%
    get_p_value(obs_stat = obs_fairness_diff, direction = ___)

8.6 Wrap-up

In this session, we revisited many of the core concepts in statistical inference. We used both simulation and the normal distribution to find confidence intervals for individual proportions and to conduct hypothesis tests for comparing proportions.