“Artificial Intelligence (AI) will be either the best or worst thing ever to happen to humanity.” I believe these words by Stephen Hawking are a reminder that AI, like any other technology, is a reflection of its inputs. Molded by the right set of hands, AI can bring about profound changes in our lives by improving social co-existence, protecting the environment, and augmenting human capabilities and interactions. At the same time, inherent human follies such as biases can significantly impact the output, leading to disruption or even destruction.
Identified in facial recognition systems, online search algorithms, and even hiring programs, bias is all-pervasive in AI. Let me elaborate with an example. A few years ago, Allegheny County in the U.S. launched the Allegheny Family Screening Tool (AFST). The tool uses predictive risk modeling to rate incoming calls on general child protective issues. AFST uses upwards of 100 variables to generate a score from 1 to 20 – based on which calls may be flagged for further investigation of families, where a child’s welfare could be at risk. However, AFST has also received criticism for bias against low-income families who might be scrutinized more. This creates a conundrum – does the county continue to use the tool with underlying concerns or leave the task up to the interpretation of a sole unchecked human? The issue of bias in AI is not straight-forward as we think, and requires a deeper understanding of various factors at play.
According to a WNS DecisionPointTM report on the ethical use of AI, there are two primary causes for prejudice, namely:
Let me dive into these two aspects a little more.
Dataset Challenges
When datasets are not all-pervasive, the ability of AI models to accurately predict outcomes is compromised. For example, a study of facial analysis techniques found that unrepresentative training data led to higher error rates as far as minorities were concerned, especially women. Voice recognition tools such as Siri and Alexa, while trained on large datasets, are known to understand the commands from white, upper-class Americans more easily than others due to the over-representation of this segment in the training data.
Natural Language Processing (NLP) and Machine Learning (ML) technologies are designed to help machines learn by observing patterns of human behavior. As a result, patterns of human biases also creep in to negatively impact AI-driven, decision-making. Take the example of a hiring tool built on AI that a leading company experimented with to review resumes and shortlist candidates. The tool leveraged 10-year training data that was skewed heavily in favor of men in the system. The AI tool, in extension, showed the same bias and downgraded the resumes of women.
Targeted Output Selection
Not assigning the right goal or objective to an AI model can also result in unintended bias. Case in point: grading essays through automated scoring that has its own biases. These tools consider parameters such as the length of an essay and sophisticated words as key criteria to assign high scores. They are unable to evaluate creativity or detect gibberish. Designers will therefore need to reconsider the objectives assigned to such tools to mitigate erroneous outputs.
The power of AI is both exciting and worrisome. Therefore, detecting biases and building an ethical AI framework is a moral obligation for everyone involved. It is a goal we simply cannot afford to miss!
To know more about creating ‘trust’ within an AI ecosystem, read the WNS DecisionPointTM report