Artificial intelligence is increasingly gaining the attention of both researchers and the public, more so in recent times due to technology like the ChatGPT which has proven its unlimited potential to be implemented in diverse fields. The KAIST Herald interviewed Donghyeon Han, who conducted this research at Professor Hoi-jun Yoo’s laboratory in the School of Electrical Engineering and is currently undergoing his post-doctoral course at Massachusetts Institute of Technology. He published his research paper on MetaVRain, the world's first semiconductor integrated into 3D modeling rendering with AI, on February 7. Han gave a presentation at the International Solid-State Circuit Design Conference (ISSCC), which was held from February 20, regarding his paper “MetaVRain: A 133mW Real-time Hyper-realistic 3D NeRF Processor with 1D-2D Hybrid Neural Engines for Metaverse on Mobile Devices”.

Can you explain your scientific paper to our readers?

Originally, the 3D games we play are modeled through a method called “ray tracing”, which calculates trajectories of the light rays created by the light source one by one, thus requiring massive calculation and memory. Our research specifically focuses on semiconductor design to perform 3D rendering using a deep neural network to eliminate inefficient calculation processes, allowing the rendering at a lower power but through a shorter time. 

What was the initiative for this research?

I am quite interested in VR and AR games, but I always thought that the graphics and rendering were of poor quality and unrealistic when I played them. The news nowadays says that the metaverse will be the new mainstream, but I thought the quality of the current technology is quite distant from the dream future described in the media. I initiated this research out of curiosity that I might be able to improve 3D modeling technology by integrating deep neural networks. I was also inspired by another research that tried to integrate artificial intelligence with 3D rendering at the European Conference on Computer Vision (ECCV) held in 2020.

Were there any difficulties that your team faced while conducting the research? How did you overcome them?

The most difficult part was persuading both my team and our professor to initiate this research as it was an unprecedented research topic in our laboratory. They were reluctant because they thought the current ray tracing method was dominating the market and did not see the need to invest in using semiconductors for this field. However, I was confident in my motivation to initiate this research and I tried to persuade them by providing more objective evidence like relevant research papers and statistics regarding the potential of semiconductors for 3D modeling.

What is the significance of MetaVRain in the scientific field and our lives?

I do not have an illusion that this research will completely change the whole 3D modeling industry in an instant because the semiconductor research carried out by universities is much smaller in terms of scale and available infrastructure compared to those by established companies. While we believe that there are significant limitations for MetaVRain to be commercially used immediately, we actually got a lot of interest from very renowned companies at the ISSCC conference about commercialization. We expect that our research on MetaVRain may be the starting point for its commercialization in the future. I hope that MetaVRain can lower the barriers of 3D entertainment for a wider audience, where people can actively engage to create their own contents of 3D modeling rather than relying solely on graphic engineers. 

Is there any follow-up research you want to conduct in the future?

My initial research area was developing “the algorithm for semiconductors co-design” to be used for deep neural networks, and MetaVRain was just one possible result of my research that can be useful to people. I would like to research how semiconductors can be utilized in the future where artificial intelligence like ChatGPT becomes an essential part of our lives, specifically more on semiconductors demanding low power but achieving high performance so that AI can be used effectively in small platforms for the public.