Knowing when to trust a model’s predictions is not always easy for workers who use machine-learning prototypes to help them make decisions, especially because these models are often so complex. Hence, users may use a technique known as selective regression, in which the model estimates its confidence level for each prediction and rejects if it is too confident. A human can then manually examine those cases, gather additional information, and make decisions about each one.
While selective regression has been shown to improve overall model performance, researchers at the Massachusetts Institute of Technology (MIT) and the MIT-IBM Watson AI Lab discovered that it can have a reverse impact on underrepresented groups of individuals in a dataset. With selective regression, the model’s certainty grows, as does its chance of making the correct prediction, but this does not always happen for all subgroups.
A model predicting loan approvals, for example, may make fewer errors on average, but it may make more incorrect predictions for Black or female applicants. One reason for this is that the model’s confidence measure was trained on overrepresented groups and may be inaccurate for underrepresented groups.
After identifying the problem, the MIT researchers developed two algorithms to address it. They demonstrate, using real-world datasets, that the algorithms reduce performance disparities that have harmed marginalised subgroups.
Regression is a method for estimating the relationship between a dependent and independent variable. Regression analysis is commonly used in machine learning for prediction tasks such as predicting the price of a home based on its features (number of bedrooms, square footage, etc.) With selective regression, the machine-learning model has two options for each input: make a prediction or abstain from making a prediction if it lacks confidence in its decision.
When the model abstains, the coverage—the portion of samples on which it bases predictions—decreases. The model’s overall performance ought to increase by restricting its predictions to inputs about which it is quite certain. However, this can potentially accentuate dataset biases, which happen when the model lacks sufficient data from subgroups. Underrepresented people may make mistakes or poor forecasts because of this.
The goal of the MIT researchers was to guarantee that, as the performance for each subgroup improves with selective regression, so does the overall error rate for the model. This threat is identified as a monotonic selective risk. To deal with the issue, the researchers designed two neural network algorithms that impose this fairness criterion.
One algorithm ensures that the model’s features contain all important information regarding sensitive factors like race. Sensitive qualities can’t be used for judgments owing to laws or policies. The second procedure uses calibration to ensure the model generates the same prediction for an input, regardless of sensitive properties.
The researchers tested these algorithms on high-stakes real-world datasets. A crime dataset uses socioeconomic data to forecast the number of violent crimes in communities. An insurance dataset predicts total annual medical expenses invoiced to patients. Both databases have personal information.
Implementing their techniques on top of a standard machine-learning method for selective regression reduced inequities by lowering error rates for minority groupings in each dataset. This was done without considerably increasing errors.
The researchers want to adapt their answers to other challenges, such as predicting property values, student GPA, or loan interest rate. To prevent privacy risks, they intend to use less sensitive information during model training.
They also want to enhance selective regression confidence estimates to avoid scenarios where the model’s confidence is low, but its prediction is true. This could reduce human workload and simplify decision-making.
