Four students from the Computer Engineering program at Insper programmed a solution for automatically detecting tired drivers on the Nvidia Jetson Nano platform using a USB camera. In the Final Engineering Project (PFE), Guilherme Dantas Rameh, Luiza Valezim Augusto Pinto, Raphael Lahiry de Barros, and Rodrigo Nigri Griner started from an artificial intelligence (AI) neural network that identifies emotions on the human face. The challenge was to use only Nvidia’s proprietary tools, a partner of Insper’s PFEs, to test the AI technologies that are easily accessible to developers — such as the Jetson Nano line.

 

The PFE advisor, Professor Fábio de Miranda, says that one of the project’s intentions was to generate demonstrations of the capability of these embedded processors for the automotive industry. Automobile manufacturers already use various resources and equipment—via camera or otherwise—to detect the driver’s reactions and issue alerts for situations of fatigue and inattention while driving. The advancement sought by Nvidia is in simplifying the structure of the usual detection systems, involving fewer sensors than those currently installed in vehicles.

 

Miranda explains that the group adapted a system from Nvidia, the EmotionNet, which recognizes points on the human face to determine emotions such as happiness, surprise, disgust, tension, or neutrality, among others. In short, they conducted a process of transfer learning, which involves using the initial layers of a deep learning network and changing its end. The AI is then trained to detect fatigue in the human face instead of defining emotions. The result is not yet a ready solution, but the undergraduate students in Engineering took the first steps to show that the system is possible if there is industry interest.

 

Student Raphael Lahiry says that the group decided to adopt an agile methodology with sprints that were not rigid and could be changed depending on the moment and direction of the project, according to the company’s requirements. “The initial schedule underwent several changes during the project’s progress, which allowed us to have considerable flexibility to explore and develop the work with autonomy and freedom,” states Lahiry.

 

The PFE began with a theoretical study on what it scientifically means to be tired and how to make this identification using technology. “Although we already knew that we would move towards fatigue detection from a video input collected by a camera, we read several papers to understand the state of the art for this type of work,” says Lahiry.

 

He explains: “The first half of the project was much more exploratory and focused on studying the problem. In the second half, once we understood the resources we had available and where we wanted to go, we actually started developing the solution that was delivered at the end of the final project. One of the main points of evolution in the project was when we managed to achieve face detection, which was one of the milestones to reach the final solution, using exclusively Nvidia tools.”

 

The students concluded that the neural network focused on fatigue needs to be improved, as EmotionNet requires a complete view of the driver’s face to identify or rule out fatigue. However, people move their heads while driving, and not all 68 points of the face needed for detection are recorded by the camera feeding the AI. Thus, the neural network tries to infer the 68 landmarks only on the focused part of the face, risking false positive or negative alerts.

 

“The requirement to build the project solution using specific resources that we had never practiced before was quite challenging and became a point of professional growth for the group,” states Lahiry. “I believe that the practice of following requirements in an engineering project and having the agility and autonomy to learn to work with new tools to build a solution is an essential skill for an engineer, and this was well addressed during this final project.”

 

Colleague Rodrigo Nigri agrees with the idea of personal and professional growth. “In college, we had numerous opportunities to carry out projects that resembled reality, but the PFE was my first major contact with a company,” says Nigri. “This provided a unique experience that brought me closer to the professional environment, allowing me to apply theoretical knowledge in practice and develop critical skills, such as teamwork, communication, and problem-solving.”

 

For Lahiry, the first fruits of the experience came during the selection process for an intern position at the fintech QI Tech. He states that the theme of detecting tired drivers through a camera has characteristics in common with the anti-fraud technology developed at the company. One of the tasks of this technology is the user’s life proof with mobile device cameras. “In my view, this gave me a huge advantage in securing the position,” says Lahiry.