Before making AI's minds visible...
Recently, as an NDM researcher, I'm about to promote my research on XAI (Explainable Artificial Intelligence). I'm especially interested in how NDM can be contribute to figuring out the direction of research & development of XAI. Also, I think it's necessary to consider how human can adjust societal changes XAI brings about.
Besides the 'frame shift' issue (that AI cannot efficiently choose the targeted point unlike human), AI has its 'black box' issue. Black box means that AI cannot show its thought process to the human user. To overcome this issue, XAI has been developed. XAI picks up some important words and phrases to explain its judgement & decision-making process.
Usually, almost all NDM researchers spend much time to conduct cognitive interview with experts in the field (of course, to compare with the expert, we have interviews with novices and beginners). The interview method is known as Cognitive Task Analysis (CTA). We analyze the remarks during the interview and sometimes observe their performance in the real setting. Then, the researcher tries to figure out the meanings of the expert's perception and decision-making process.
Before we think of how we can make AI's thought process visible, we must compare human (the expert's) thought process with AI's. As I pointed out in my Japanese book about AI, human can adopt both intuitive and analytical thinking, AI totally depend on algorithmic (analytical) and quantitative thinking. Such difference between human and machine often causes incongruity and even serious trouble.
Another point is that we still need to know our minds. Modern psychologists are studying the function of mind, but amazingly they don't answer the most fundamental question: 'What is mind?' Our minds are invisible and intangible, but mutable. For instance, if someone asks me, 'Do you have your mind?', I would reply to the person, 'Yes, I do'. However, we usually have much difficulty proving our minds scientifically -location, weight, height and width, etc.
Again, before making AI's mind visible, we must repeatedly ask ourselves the basic question, 'What is mind?' and keep pursuing the answer....