Removing Bias from Hiring
with Natural Language Processing
Resumes are inconsistent. Even the best OCR parsing leaves you with lots of messy and unstructured data. Then, as a candidate moves through the application process, humans get involved. Add to the data set free form text reviews of the applicant and both linguistic and personal biases. In addition, each data source is siloed providing limited analytical opportunity.
Natural Language Processing allows machines to
understand and classify free form text.
After assessing multiple companies hiring processes, we have found 3 consistent opportunities to systematically improve hiring outcomes using NLP machine learning. The problem areas are: correctly structuring candidate resume data, assessing job fit, and reducing human hiring bias.
Blue Orange Digital started with the imperative issue, isolated data sources. We designed a custom ETL to blend, clean and standardize the different data sets.
We then worked to parse and categorize free form string text to intelligently search and analyze unstructured documents. Using keyword detection classifiers, optical character recognition, and cloud based NLP engines, we were able to scrub string text and turn it into relational data. With structured data, we provided a fast, interactive and searchable Business Analytics dashboard in AWS Quick Sight.
With a cleaned and structured data set, we were able to perform both sentiment analysis on the text and subjectivity detection to reduce candidate bias in human assessment.
With structured data, we provided a fast,
interactive and searchable Business Analytics
dashboard in AWS QuickSight.
We have used ML to automate resume screening and shortlist and grade candidates by learning from existing employees’ resumes. First we used a natural language processing ML algorithm to turn the unstructured resume text into relational data. Then we built another ML algorithm that trained itself on prior employees to learn which resume data points (inputs) are correlated with successful employees to produce a shortlist of qualified candidates for the position (output). Instead of just scanning for keywords, we are able to make predictive hiring suggestions to HR. In addition, for firms who use digitized interviews, we can use machine learning technology to assess candidates’ personality and job fit by learning from successful candidates’ facial expressions and word choices.
We used sentiment analysis to identify potentially biased language in job descriptions. The program is fed inputs with words like “aggressive” that are perceived as masculine-sounding and words such as “collaborative” that are perceived as feminine-sounding. By analyzing the words used in a job posting, the program creates an output of suggested replacement words in order to help solve the problem that these words may be discouraging female candidates from applying.