We used Deep Neural Network to teach a drone to recognize and follow forest trails in search for lost people. More info in the FAQ below.
Paper:
A. Giusti et al., A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots, IEEE Robotics and Automation Letters, 2016.
PDF: http://rpg.ifi.uzh.ch/docs/RAL16_Giusti.pdf
Project webpage and datasets: http://www.leet.it/home/giusti/website/doku.php?id=wiki:forest
FAQs:
What is the paper about?
We present the first work using a Deep Neural Networks (DNNs) image classifier running onboard our vision-controlled drone to recognize and autonomously follow forest trails. Unlike previous works, which relied on image salience or low-level features, our DNN-based image classifier operates directly on pixel-level image intensities and outputs the direction of the trail with respect to the heading direction of the drone. If a trail is visible, the software steers the drone in the corresponding direction.
How did we train the classifier?
In order to gather enough data to train our DNN classifier, we hiked several hours along different trails in the Swiss Alps and took more than 20 thousand images of trails using cameras attached to a helmet (Fig. 4 in the paper). This effort paid off: when tested on a new, previously-unseen trail, the DNN was able to find the correct direction in 85% of cases; in comparison, humans faced with the same task guessed correctly 82% of the time.
Real time and onboard?
Yes. The classifier ran in real time and onboard the smartphone processor (Odroid quadcore computer) on our custom-made vision-controlled quadrotor. Both visual odometry (based on SVO) and control were also running onboard.
Why do we want drones to follow forest trails?
To save lives. Every year hundreds of thousand people get lost in the wild worldwide. In Switzerland alone, around 1000 emergency calls per year come from hikers, most of whom are injured or have lost their way. Drones are an efficient complement to human rescuers and can be deployed in large numbers, are inexpensive and prompt, and thus minimize the response time and the risk of injury for those who are lost and those who work in rescue teams.
Is the training and testing data available for research?
Yes, from the project webpage.