Control Arduino Uno Robot with Hand Gestures using Machine Learning

Control Arduino Uno Robot with Hand Gestures using Machine Learning

Table of Contents

  1. Introduction
  2. Preparing the Robot
  3. Training the Gesture Recognition Model
  4. Setting Up PictoBlox
  5. Uploading Firmware to Arduino
  6. Connecting Arduino via Bluetooth
  7. Adding the Machine Learning Extension
  8. Loading the Trained Model
  9. Setting Up the Stage
  10. Creating Custom Function Blocks
  11. Creating the Main Script
  12. Conclusion

🖐️ Introduction

Welcome back to our Channel! In this video Tutorial, we will show you how to control an Arduino Uno Robot with HAND gestures using PictoBlox's Machine Learning extension. By training a gesture recognition model and uploading it to PictoBlox, you'll be able to make your robot move forward, turn left, turn right, and even stop, simply by using hand gestures. So let's get started!

🛠️ Preparing the Robot

Before we begin, it's important to make sure you have an assembled 2-wheel or 4-wheel drive Arduino robot. If you don't have one yet, you can find the instructions to build it by clicking on the link provided in the video description. Additionally, you'll need a laptop or computer with a camera, the latest version of PictoBlox, and a good internet connection.

🎓 Training the Gesture Recognition Model

The first step is to train the gesture recognition model using Google's Teachable Machine. To do this, go to https://teachablemachine.withgoogle.com/ and click the "Get Started" button. On the new page, click on the "Image Project" tile. We will begin by adding the first class, which is the gesture for moving the robot forward. Using the webcam, Record images of your open Palm. Repeat this process for the gestures of turning left, turning right, and stopping. Once your training data is complete, you can train the model by clicking the "Train Model" button. Take a moment to test the trained model before proceeding.

📥 Setting Up PictoBlox

Now that we have our trained model, let's open PictoBlox and add the Machine Learning extension. To do this, sign in or login to PictoBlox and ensure that you have a stable internet connection. Select the Arduino board and upload the firmware to it. Connect your Arduino Uno to the computer using a USB cable, choose the appropriate serial port, and click on the "Upload Firmware" button. Once the firmware is successfully uploaded, disconnect the Arduino from the computer. Next, connect the Arduino via Bluetooth by clicking on the "Connect" button and selecting the appropriate Bluetooth port from the list. Now, let's add the Machine Learning extension. Click on the "Add extension" button and choose the Machine Learning Extension.

📲 Loading the Trained Model

With the Machine Learning extension added, we can now load our trained model into PictoBlox. Click on the "Load Model" button and a dialog box will appear. Paste the sharable link you obtained from Google's Teachable Machine into the space provided and click on "Load Model". Wait for a moment while the model gets loaded. Once the loading is complete, you'll see the Machine Learning blocks in the palette.

🌟 Setting Up the Stage

Let's start building our program by setting up the stage. Drag and drop the "when flag clicked" hat block from the Events palette to execute the program when the green flag is clicked. To display the video feed from the camera, add the "turn on video on stage with 0 transparency" block from the Machine Learning palette.

🧱 Creating Custom Function Blocks

We will now create custom function blocks to make programming easier. Go to the "My Blocks" palette and click on the "Make a Block" button. For the first custom function, name it "go straight". Inside the function block, place the "run motor 1 in direction forward with speed 100%" block from the actuators palette. Duplicate this block and place it below, but this time, choose motor 2. Repeat this process to create custom blocks for turning left, turning right, and braking.

🔨 Creating the Main Script

Now it's time to create the main script. Place an "if-else" block from the Controls palette. From the Machine Learning palette, add the "is identified class from web camera is forward" block and choose "stage" from the dropdown. This block will allow us to recognize the gesture for moving the robot forward. In the "if" arm, place the "go straight" custom function block. Duplicate the "if" block and place it in the "else" arm. Replace the "go straight" block with the custom function blocks for turning left, turning right, and stopping.

✅ Conclusion

Congratulations! You've successfully learned how to control an Arduino Uno Robot with hand gestures using PictoBlox's Machine Learning extension. Now you have the power to make the robot move, turn, and stop with simple hand gestures. We hope you enjoyed this tutorial and don't forget to share your creations and queries with us in our PictoBlox community on Facebook. Be sure to subscribe to STEMpedia and follow us on Facebook, Instagram, and Twitter for more exciting projects. Stay safe and happy coding!

Highlights

  • Learn how to control an Arduino Uno Robot with hand gestures
  • Utilize PictoBlox's Machine Learning extension for gesture recognition
  • Train a gesture recognition model using Google's Teachable Machine
  • Set up PictoBlox and load the trained model
  • Create custom function blocks for easier programming
  • Build the main script to recognize gestures and perform actions by controlling the robot

FAQ

  1. Can I use any Arduino robot for this project?

    • Yes, you can use an assembled 2-wheel or 4-wheel drive Arduino robot for this project.
  2. Do I need a camera to control the robot with hand gestures?

    • Yes, you'll need a laptop or computer with a camera to capture the hand gestures.
  3. Is it necessary to train the gesture recognition model?

    • Yes, training the model is a crucial step to teach the system to recognize specific hand gestures.
  4. Can I control the robot with more than four gestures?

    • Yes, you can train and add additional gestures to control the robot in different ways.
  5. Are there any specific system requirements for using PictoBlox and the Machine Learning extension?

    • You need to have a stable internet connection and the latest version of PictoBlox to utilize the Machine Learning extension.

Find AI tools in Toolify

Join TOOLIFY to find the ai tools

Get started

Sign Up
App rating
4.9
AI Tools
20k+
Trusted Users
5000+
No complicated
No difficulty
Free forever
Browse More Content