A Powered by Osteons reader, Judy Barr, alerted Kristina Killgrove to a new article in the journal Science and Justice called “Cognitive bias in forensic anthropology: Visual assessment of skeletal remains is susceptible to conformation bias” by Nakhaeizadeha, Dror & Morgan. Kristina then posted it to BioAnthropology News (if you’re not a member, click on the link and join!), and got into a back-and-forth with fellow bioarchaeologist Alison Atkin, who also writes the Deathsplanation blog. We decided to try something new: opening up our ideas by cross-posting our conversation on our respective blogs.
|Nakhaeizadeha et al. 2014, Fig. 1 -- The skeleton used in the study|
- They don’t fully discuss which portions of the skeleton were available for study. Was the pubic symphysis -- by far the best portion of the skeleton for accurately estimating both age and sex -- present or did the participants rely on less accurate techniques?
- The skeleton was archaeological, not forensic, and the participants didn’t know this. This context -- bioarchaeological or forensic -- is very relevant for completing a full demographic profile because some methods may be affected by secular or cross-cultural differences in populations.
- The participants were largely students. Students outnumbered professionals (using the terms of the authors) 2 to 1. In the control group C, the students outnumbered the professionals 3 to 1. This makes it impossible to control for experience in assessing bias. (I have nothing against students, of course, since I was one just a few short years ago. But I am also certain that I am better at my job now than I was then.)
- Participants were from different fields. In the UK, osteology and forensic anthropology are different disciplines. We learn the same information, but the aims of our investigations are very different. I wonder if all of the participants would have been considered qualified to stand as an expert-witness in court, given the main aim of this study?
- DNA analysis of sex is a unique biasing factor. I was curious about the decision to include DNA results with sex-specific markers (the authors used the word gender, but we won't get into that here). It is very rare that you see DNA used to determine sex in archaeological material - and it would always be done after the skeletal assessment. Using both precise methods (like DNA analysis) and less precise methods (like skeletal markers) to form the false contexts would likely have provided different levels of bias for different aspects of this study.
- Could participants answer probable male/female? The archaeological skeleton in this study was recorded in the database as 'probable female'. The authors stated they are especially concerned about bias in ambiguous cases - however as far as I can tell, participants only had the option to state at the end of their assessment whether the individual was: male, female, undermined - and not probable male/female.
- Sex (Probably Female)
- Group A (male context) - 71% said male, while 29% said female
- Group B (female context) - 100% said female
- Group C (no context) - 30% said male, while 70% said female
- Age-at-Death (36-45)
- Group A (25-30 context) - 0 at 18-25, 11 at 26-35, 3 at 36-45, 0 at over 46
- Group B (50-55 context) - 0 at 18-25, 1 at 26-35, 7 at 36-45, and 5 at over 46
- Group C (no context) - 1 assessed 18-25, 6 at 26-35, 5 at 36-45, and 1 at over 46
- Ancestry (Caucasian)
- Group A (Caucasian context) - 100% got it right
- Group B (Asian context) - 50% got it right, 29% said Asian, and 21% didn’t know
- Group C (no context) - 100% got it right
Smith, A.C. and A. Boaks. In press. How “standardized” is standardized? A validation of postcranial landmark locations. Journal of Forensic Sciences.
**For example: Based on the non-metric analysis of the os coxae, which showed good preservation allowing for a full assessment, these methods were used of scoring these skeletal indicators (sub-pubic angle, sciatic notch, etc). Peer-reviewed studies have shown these methods to be accurate up to 98% in correctly identifying the biological sex of an individual (provide relevant citations). I estimate, based on the results of these methods that this individual is a probable female. This suggests, given the different categories of classification of biological sex within an osteological framework that it is most likely these skeletal remains are from an female individual. BAM!