Artificial intelligence usually has a black box character. But only transparency can create trust. Special software is available to explain the respective solution path. A study by Fraunhofer IPA has now compared and evaluated different methods that make machine learning processes explainable.
Artificial intelligence, which was still science fiction a few decades ago, has now become part of everyday life. In manufacturing, it detects anomalies in the production process, in banks it decides on loans and at Netflix it finds the right movie for every customer. Behind it all are highly complex algorithms that operate in secret. The more challenging the problem, the more complex the AI model - and therefore the more inscrutable it becomes.
However, users want to understand how a decision is made, especially in critical applications: Why was the workpiece rejected as faulty? What is causing the wear on my machine? This is the only way to make improvements, which increasingly also affect safety. In addition, the European General Data Protection Regulation requires decisions to be traceable.
Software comparison for xAI
An entire field of research has emerged to solve this problem: "Explainable Artificial Intelligence", or xAI for short. There are now numerous digital aids on the market that make complex AI solutions explainable. For example, they mark the pixels in an image that have led to faulty parts being rejected. Experts from the Fraunhofer Institute for Manufacturing Engineering and Automation IPA in Stuttgart have now compared nine common explanation methods - such as LIME, SHAP or Layer-Wise Relevance Propagation - and evaluated them with the help of exemplary applications. Three criteria were particularly important:
- Stability: The program should always provide the same explanation for the same task. It must not be the case that sensor A and then sensor B are held responsible for an anomaly in the production machine. This would destroy trust in the algorithm and make it more difficult to derive options for action.
- Consistency: At the same time, only slightly different input data should also receive similar explanations.
- Fidelity: It is also particularly important that explanations actually reflect the behavior of the AI model. It must not happen that the explanation for the refusal of a bank loan states that the customer is too old, although the decisive factor was actually that the income was too low.
The decisive factor is the use case
Conclusion of the study: all of the explanatory methods examined proved to be useful. "But there is no one perfect method," says Nina Schaaf, who is responsible for the study at Fraunhofer IPA. For example, there are major differences in the runtime required by a method. The selection of the best software is also largely dependent on the task at hand. For example, Layer-Wise Relevance Propagation and Integrated Gradients are particularly suitable for image data. "And finally, the target group of an explanation is always important: an AI developer wants and should receive an explanation differently to the production manager, as both draw different conclusions from the explanations," summarizes Schaaf.
