Skip to content

stanford-policylab/asr-disparities

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Racial Disparities in Automated Speech Recognition

Files and code to reproduce results found in our PNAS paper can be found here.

Setup

Install the necessary python modules

pip3 install -r requirements.txt

We assume prior generation of audio snippets for both VOC and CORAAL; this process is provided in src/utils/snippet_generation.py with CORAAL mp3 snippets contained in input/CORAAL_audio.

Clean and standardize all (ground truth and ASR) transcriptions and calculate WER on all snippets

(Only CORAAL data are displayed, using the ground-truth and ASR transcripts contained in input/CORAAL_transcripts.csv)

python3 src/clean_WER.py

Match audio between black and white speakers, and perform analyses

Note that input files include: VOC_WER.csv (which contains VOC error rates for all 5 ASRs, without transcriptions given privacy constraints), and DDM.csv (which contains the random sampling of 150 CORAAL snippets for DDM encoding). The full R code is provided in src/analysis.Rmd, which compiles to src/analysis.html.

Rscript src/analysis.R

Additional analyses are provided in src/utils, including n-gram matched samples, lexicon share, and language modeling