New Method Peeks Inside the “Black Box” of Artificial Intelligence

Pathologies of Neural Models Make Interpretation Difficult

A new method to decode the decision-making processes used by "black box" machine learning algorithms works by finding the minimum input that will still yield a correct answer. In this example, the researchers first presented an algorithm with a photo of a sunflower and asked "What color is the flower?" This resulted in the correct answer, "yellow." The researchers found that they could get the same correct answer, with a similarly high degree of confidence, by asking the algorithm a single-word question: "Flower?" (source: Rice University)

December 3, 2018 | Source: University of Maryland, College of Computer, Mathematical & Natural Sciences,, Matthew Wright, 5 November 2018

Artificial intelligence—specifically, machine learning—is a part of daily life for computer and smartphone users. From autocorrecting typos to recommending new music, machine learning algorithms can help make life easier. They can also make mistakes. 

It can be challenging for computer scientists to figure out what went wrong in such cases. This is because many machine learning algorithms learn from information and make their predictions inside a virtual “black box,” leaving few clues for researchers to follow.

A group of computer scientists at the University of Maryland has developed a promising new approach for interpreting machine learning algorithms.