In computer graphics, the three-dimensional reconstruction of objects from two-dimensional images requires high processing power and, therefore, results in high costs. More functional alternatives using artificial intelligence (AI) resources will facilitate applications in games, virtual reality, the Internet of Things, among others. Supervised by professor Luciano Pereira Soares, student André Corrêa Santos, from Insper's Computer Engineering program, investigated these possibilities in the Institutional Program of Scholarships for Technological Development and Innovation (PIBITI). The report received an honorable mention in the PIBITI category during the 2nd Scientific and Technological Initiation Symposium of the school, held in 2024.

 

Soares notes that the 3D reconstruction of objects using AI is a hot topic among researchers. "Everything is very new in this area, so we had to read the most recent papers and implement," says the professor. During the initiation, Santos thoroughly studied two approaches: Neural Radiance Fields (NeRFs) and the Gaussian Splatting rendering technique. Broadly speaking, the Gaussian technique is a less expensive path than the neural network. The student created different shaders (color shading computer programs). "The visuals obtained are very good. André dedicated himself a lot to the project and managed to produce effects in the 3D reconstructions that turned out quite interesting," Soares says.

 

Now in the ninth semester of the program, Santos comments: "The problem we were trying to solve is related to the cost in computational terms. So, if there isn't a very efficient solution specific to the problem, you can't get a minimally stable or functional result. This Gaussian technique is an innovation that emerged in the last two years for representing these three-dimensional objects from photosets, which saves a lot of computer performance. From 2020 to 2023, we mainly used neural networks. In this research, I mainly did low-level programming on the video card, the so-called GPU."

 

The honorable mention was an unexpected but rewarding recognition. "I was very happy because it was a risky project, somewhat sophisticated in technical terms for me," the student says. "Until the last moments, I didn't know if I could achieve enough performance to execute the project. PIBITI requires technological innovation. It's not enough to do a literature review. My technological innovation was something I added to relight the scenes generated by Gaussians. However, until the last submissions, I couldn't achieve the performance I was looking for. Toward the end, some ideas stitched the thing together and made it work. So, I was quite happy, but it was a roller coaster of trial and error."

 

Santos had never worked with computer graphics before. The topic suggestion came from Luciano Soares as a complement to the research the professor conducts in partnership with colleagues from the University of Illinois at Urbana-Champaign in the United States. "I don't know if I'll work with computer graphics right after college, but I realized that this field offers a valuable foundation for any area of computing," says the student, who spent two months in the summer research program at the University of Illinois last year.

 

As he envisions graduation at the end of 2025, Santos continues to collaborate remotely with the virtual reality projects of the Parasol laboratory, where he was allocated in the summer program. Participation in PIBITI is another experience to value in the résumé. "PIBITI helps the student delve into a topic, which is not always possible in the day-to-day of the courses, when there are other assignments or grades to improve. In scientific and technological initiation, you have much more guidance, with a professor who meets weekly to guide you and see what is going well or wrong. You push yourself to the limit," Santos says.