Interpretable AI: The Zing-Thing to Bolster the Adaptability of AI

(This is a follow-on article to my previous article on the Future of AI in Industries.)

A twenty years old story

Almost 20 years ago I was working in the ASIC design industry. I was in a team making digital IP cores for chips to be used in future automobiles. My friend was working with another team working for smartphone customers. My group had to meet the ATP of releasing cores which should be 99.99 % bug-free. My friend’s team had to meet the 75%-bug-free threshold. Most industries require a high level of reliability. Reliability also implies traceability. This is the reason why most quality management systems focus on well traceable processes.

Why Interpretable AI (I-AI)?

In the past decade, the new wave of deep learning-based AI (dAI) has achieved tremendous feats. One of the exciting achievements of dAI has been to automatically learn the features best suited for the task. Given enough data, dAI could even beat humans at certain tasks. After these tremendous developments, we are entering the era of ubiquitous implementation of AI algorithms in industries. However, a major issue with the current generation of dAI is their black-box nature. This discounts traceability which is a major requirement from many industries. Interpretable AI architecture will enable traceability.

The other issue that AI can not remain oblivious to is the ethical dimension. In April 2021, the EU published its proposed AI regulations. These are some of the most advanced and considered regulations we have seen so far. One of the major areas of focus in these regulations is the need for AI solution architectures where there should be no surprises. This need means AI solutions that should be interpretable. For example, some AI implementations have shown unfortunate biases (e.g. racial and gender biases) and an I-AI architecture will reduce the probability of such biases.

Lastly, when one works on architectures to use AI algorithms in industrial settings, one needs to process a Herculean amount of data streaming from a large number of sensors. These data, often, are heterogeneous. The data, mostly, comes with no guarantee about its quality and reliability. Hence, it is mostly impossible to reuse an existing AI architecture. This is another reason why interpretable AIs will tremendously accelerate the adoption in industries.

Current fortes of AI

Before we proceed towards planning towards I-AI, it will be nice to do a quick facts-checking about our current position in AI innovation. The dAI algorithms have strongly established themselves as correlation extractors. This enables them to extract not -only apparent features but also latent features important for an AI task. This enables them to, at times, out-perform humans. Similarly, they are also successful in extracting features for data that are not easily understood by humans (e.g. MRI images). The biases AI algorithms learn are due to the way learning happens. They are imported from the dataset. For example, if we are trying to train a dAI algorithm to detect butterflies and the dataset mostly has images of butterflies in their natural environment (i.e. gardens), the AI algorithm may learn that a green background means the existence of butterflies!

Another exciting development in the domain of AI is the endeavours where dAI are trying to achieve generative properties. Traditionally dAI algorithms have been discriminative. However, with the advent of GAN algorithms, this distinction is blurring.

Lastly, the developments in the domain of encoder-decoder have been empowering us with new algorithms to develop more interpretable AI architectures. One of the most successful of these architectures is the transformer networks which have enabled the third time-scale (through modelling attention) and have revolutionised NLP.

Three Dimensions of I-AI

To achieve interpretability in AI algorithms, we need to push the boundaries in three dimensions, viz. what are the relations that the network learned, how is it learning, and what are the objects or phenomena it has learnt.

• Interpreting the Learnt: One of the major challenges for modern AI engineers is to understand what it is that a deep learning algorithm has learnt. One of the reasons for the power of dAI is the compositional manner in which it learns a task. Each layer learns a combination of features learnt in the layers deeper than itself. Interpreting the features or concepts learnt by each neuron is an emerging piece of research.

For industrial applications, a preferred modus operandi is to start with data analysis. This can be followed by slowly increasing the number of neurons and layers in the network. Especially for CNNs, it is an interesting exercise to investigate what is the filter that is learnt at each layer. For example, in our work on the design of identification of impulsive radio frequency interfaces (RFIs), analysis of the data gave rise to a dictionary model of RFIs. Then, we designed a CNN+LSTM based architecture to identify the RFIs. In that, we could identify CNN-filters that looked similar to the dictionary of RFIs we have seen in the previous work.

• Interpreting the Learning: The second range of initiatives needed to interpret AI architectures to understand the learning process. This brings us to the domain of symbolic AI. It was the predecessor to the current wave of AI algorithms (which are mostly sub-symbolic AI algorithms). Symbolic-AI architectures are mostly expert-designed which limits their ability to learn from data. Some of the potentially adoptable symbolic AI algorithms are brain-inspired.

As we have discussed, dAI algorithms are extremely powerful in extracting correlation patterns and useful features from data. The fusion of symbolic and sub-symbolic architectures will enable us to interpret the way learning happens. In one of the initiatives, we have proposed a gating-mechanism-based architecture to fuse these two paradigms. In another recent work, we have also proposed a brain-inspired cognitive architecture.

• Interpreting the Learned: The third thing we need to interpret is the phenomenological relevance of learning to the physical world. Through our measurements, we acquire a representation of reality. Like Plato’s cavemen, data/signal processing engineers try their best to know what is the physical relevance of the signal or data that we have. In this endeavour, we try to train the network specific to each event or phenomenon of interest. This is quite similar to the manner in which smaller animals act in their niche ecosystem. Hence, this dimension of I-AI can learn a lot from neuroethology. For example, in a recent work, I proposed an AI architecture inspired by the neuroethology of weakly electric fish. One of the fascinating points to be noted about neuroethology is the fact that small animals co-evolve with a lot of other organisms in their ecosystem. Hence, their neurons transpose the data into different layers with each layer corresponding to certain events or phenomena of interest. Thereby, they achieve detection and recognition in one single step.

The Future

AI will play a major role in the tremendous amount of changes happening around us. interesting opportunities are coming up which will push for larger adoption of AI. For example, the EU is pushing hard to have better circular economy action plans and plastic strategies.

Human judgements are known to be biased and unpredictable, as elegantly put in the new book “Noise” by Kahneman! One of the major selling points for AI is that, potentially, it can be designed to be less biased than humans. Hence, AI is expected to be a major accoutrement in achieving sustainable development goals.

Interpretability is the zing thing that AI architects need to have in order to take AI application to this next stage where it will aid in creating a greener and more prosperous world.