The purpose of this project was to build a system that allows user to control the Arlo mobile robot using static hand gestures, with use of camera. The project consists of three parts, the first was to develop script that supposed to recognize the gestures controlling the robot. For this purpose author used convolutional neural network, which had to be properly trained. The next step was to create a robot program that had to properly react to application commands. The last step was to handle communication between programs, Bluetooth 4.0 Low Energy standard was used for this purpose. Unfortunately, during the final control test Bluetooth module has been damaged, in this case the author decided to use simple serial communication instead. The main assumptions of the project were to achieve real-time processing system and achieve high accuracy of hand gesture recognition.
Arlo: https://www.parallax.com/product/arlo-robotic-platform-system
Presentation: https://www.youtube.com/watch?v=j6qOpACT1z0

This section should list any major frameworks that you built your project using. Leave any add-ons/plugins for the acknowledgements section. Here are a few examples.

• Pytorch
• PyQt
• Parallax
• Bleak
• PySerial
• OpenCv

1. Clone the repo
2. Install python 3
3. Install python dependencies, such as: Pytorch, Bleak, PyQt
4. Run script "mainAplication.py"

Gui interface:

1. Manual command list,
2. Send manual chosen command,
3. Command window, contains information about, gestures captured, execution command status etc.,
4. Frames per second,
5. On/Off neural network processing,
6. On/Off automatic gesture detection. If gesture is detected for more than 2s script automatic sends command to Arlo. Can be also activated via Ctrl.
7. Connect/Disconnect,
8. Emergency stop, cancels all commands actual beeing in execution. Can be also activates via Shift
9. Commands and probability of ocurance in actual dataframe
10. Dataframe preview in 640x480 resolution, grey area is ignored in gesture detection process.
    Controlling commands:
    
11. Default
12. Turn left
13. Turn right
14. Turn around
15. Forward
16. Backward Setting commands:
    
17. Speed up
18. Slow down
19. Bigger turn angle
20. Smaller turn angle
21. Bigger step
22. Smaller step
23. Emergency Stop

Distributed under the MIT License. See LICENSE for more information.

Sylwester Dawida Poland AGH 2020