Three students from Insper have qualified for the semifinals of the CSBC AWS DeepRacer, a national league of autonomous races based on artificial intelligence and machine learning. Promoted by the Congress of the Brazilian Computer Society (CSBC) and Amazon Web Services (AWS), the competition continues from July 20 to 24, in Maceió, Alagoas. Felipe Drummond (Mechatronics Engineering), João Alonso Casella (Economics) and Tomás Alessi (Computer Science) are students of the elective course Reinforcement Learning, taught by professor Fabrício Barth.

 

Only seven participants from the Southeast Region passed the virtual stage, held in May — three others study at the State University of Campinas (Unicamp) and one at the University of São Paulo (USP). The event gathers Brazilian students from higher education, who have already tested their machine learning skills through a cloud-based 3D racing simulator. In the next stage, the semifinalists will put their algorithms to the test in a 1:18 scale autonomous race car, powered by reinforcement learning. Each region qualifies two finalists for the last in-person race.

 

Traditionally, the SBC annual congress, held since 1978, promotes events involving students, researchers, and developers, like the CSBC AWS DeepRacer. “The competition's proposal is to train a miniature car to run on tracks autonomously and without human intervention, as fast as possible,” explains Barth. The principle is the same as driverless vehicles produced by car manufacturers and technology companies, like Zoox, owned by Amazon itself.

 

The students use algorithms trained by reinforcement learning, a method in machine learning that offers rewards for each successful or unsuccessful attempt of the vehicle. This algorithm is responsible for moving the race car, first in the virtual environment and then on the track. Throughout the semester, there are several assignments in the Reinforcement Learning course. Barth then proposed that the results obtained in the classification phase of the competition could be one of the assignments for students wishing to participate in DeepRacer.

 

Qualified for the final and competing individually, Alessi, Casella, and Drummond need to optimize their virtual stage algorithm model. The three already know they need to reduce the average lap time by at least two seconds because this was the difference compared to the first place in the classification phase. “To reduce time, they are studying much more in-depth the configuration of the algorithms, discovering how to determine what important information the robot has to collect,” says Barth. “If it weren't for the competition, perhaps some would deliver any model, but now everyone has the desire to present the best in a national event.”

 

From a holistic perspective on education, the possibility for students to make public what they have built is important for the positive reinforcement of the learning process, according to Barth. The competition helps close the learning cycle. Driven by curiosity and practice, students feel challenged to understand how other colleagues achieved a more efficient algorithm. At this stage, competitors do not have access to each other's models.

 

Studying Economics at Insper and passionate about Econometrics, Casella identified DeepRacer as a gateway to the world of technology and artificial intelligence. “Throughout my journey in data science, I realized that my fascination with prediction went beyond the traditional tools in an economist's toolbox,” he states. The perception was right. Casella achieved Insper's best time and third place in the Southeast without having the classic "dev" profile — as software developers are known.

 

Casella interprets the experience as a sign that the undergraduate program in Economics can and should see more synergies between these two worlds. “Every day, new predictive models are created, and I believe reinforcement learning will play a fundamental role in stochastic event simulations and decision-making under uncertainty,” says the student, citing concepts from Economics. “I hope my story encourages other economists to broaden their scopes, question the limits of their toolbox, and explore new paths in data, algorithms, and reinforcement learning.”

 

Alessi recalls that in the algorithm, he tested several function models to understand what was happening with the car in the virtual environment. “Basically, we had to create a reward function, which is how the car will learn to navigate the track in the best possible way,” says the Computer Science student. “You could regulate the car's maximum and minimum speed and also the maximum turn angle it could make. Once, the car was going off track a lot, and I increased the penalty for wheels off the track drastically, so in the next test, the car was afraid to go near the edges and ended up too slow. I needed to find a middle ground.”

 

What started as a playful exercise amid learning became a surprise with the announcement of the names qualified for the in-person stage. “I didn't even think I could compete in Maceió, but now I'm very excited,” Alessi says. “I see it as an opportunity to deepen my knowledge on the subject. The Reinforcement Learning course is not focused on competition preparation, but in it, I learned how a reward function operates. I also needed more specific knowledge, which I sought in DeepRacer workshops and a bit of outside research.”

 

For Drummond, participating in DeepRacer was an intense and extremely enriching experience. “From the beginning, I saw the challenge as an opportunity to explore the practical application of advanced reinforcement learning techniques, combining my background in Mechatronics Engineering with concepts of artificial intelligence,” says the student. “Developing effective strategies required not only deep technical knowledge but also lots of experimentation and continuous learning.”

 

Drummond considers qualifying for the semifinals as incredible recognition of the effort he put in throughout the semester. “This effort allowed me to compete with the best in the event and brought me even more motivation to continue studying and improving my skills in machine learning,” Drummond says. Until the very last week of July, with the professor's support, the three Insper students plan to dedicate many hours of study and work to find a solution for the mere two seconds that separate them from the final.