These days, artificial intelligence has a huge impact on not only academic fields but also people’s daily life. Artificial intelligence algorithms were inspired by the human brain neural network and attracted attention among scientists. However, operations needed for AI rely on von Neumann architecture based systems and they require a huge amount of energy and time. Henceforth, neuromorphic engineering that mimics neurons and synapses to function efficiently with low power became a trending topic. In an interview with The KAIST Herald, PhD students Ji-Man Yu and Gyeongdo Ham from the Department of Electrical Engineering introduced their paper titled “A Multiple-State Ion Synaptic Transistor Applicable to Abnormal Car Detection with Transfer Learning'’ published on March 1.

Could you explain your scientific paper to the readers?

Yu: Synaptic neuromorphic engineering can be mainly divided into two groups: mimicry of biological synapses and artificial neural network. Therefore, we decided to build a neural device that can best mimic transfer learning of the artificial neural network. The weights of the synaptic connections were adjusted with their conductance values to control how much current is flowing between nodes in the neural network. In short, we designed an analog-adjustable transistor-based memory device that can yield multi-state outputs [weight values]. We applied these transistor characteristics to the deep neural network to detect abnormal car behavior in [traffic-related] video-based data.

Ham: When you drive a car, you immediately feel something is wrong [if the car in front of you starts moving dangerously]. We wondered if we could implement this “learning” to the semiconductor device. So, [using] the [traffic-related] video dataset that we gathered on YouTube, we [conducted semi-supervised learning in] four categories: drowsy driving, driving under the influence of alcohol, car skidding on ice, and car skidding in the rain. We  found out that the accuracy of the software using GPU was between 88%  and 89%, but this requires too much energy and time. We compared [this result] with the [results from] simulation of Abnormal Car Detection (ACD) with a multi-state neuromorphic transistor using transfer learning, which runs in an analog way. The result suggested that even with low energy consumption of 10fJ pulse-1, the device was still able to achieve 85% accuracy.  

What would be the scientific merit of this study?

Yu: So far, neuromorphic engineering-based deep learning methods can only train and infer very low-level data such as simple images or voices. I believe that a deep learning mechanism should be able to interpret dynamic video-based data to be used in real life, because it will be ultimately applied to build robots or self-driving cars that make decisions through [video data]. In this respect, I think our paper was the first paper that dealt with learning and inferring video-based data using neuromorphic devices. 

What kind of research do you want to do in the future?

Ham: We are doing a follow-up study, and our final goal is “life-long learning” through online learning. [For example,] a robot needs to be consistently updated manually for better performance, [which is bothersome]. However, this process can become automated with “life-long learning.” I plan to implement a new neural network technique based on Vision Transformer as well as convolutional neural network.

Yu: If you keep using devices, they get degraded, because the process of raising and lowering the conductance puts lots of stress on them. Therefore, I have been researching how to strengthen the [durability] by implementing a “self-curing” function to the device.

Do you have any advice or tips for others?

Yu: I am a symbol of failure in our lab. It is normal for me to fail experiments and to have my papers rejected. It was just my daily routine. Whenever that happened, it hit me hard. A long time ago, I read a book and one of the quotes was, “The definition of success is to go all the way.” I was encouraged by this. Even if you slip into a slump or fail lots of times, there will still be an end to the PhD program, so I hope you all cheer up and do well.