Arduino controlled Robot using hand gestures using Python

Arduino controlled Robot using hand gestures using Python

• 1.
  | © Copyright2023, InfluxData 1 Building a gesture controlled bot with Arduino & Python Suyash Joshi sjoshi@influxdata.com PyData London Aug 2023
• 2.
  | © Copyright2023, InfluxData 2 WhoAmI ● Senior developer advocate at InfluxData. ● 10+ years of software engineering ar Oracle & RingCentral in the SF Bay Area. ● MA in Game Design from Goldsmiths, Univ. of London, ● BS in Computer Science from University of Michigan ● Meetup Organizer: Creative Code & AI ● STEM Evangelist & Magician 🎩 ● @suyashcjoshi on social media
• 3.
  | © Copyright2023, InfluxData 3 Bill of Material 1. Arduino Uno R4, 1 DC motor with shield, 5 Servo Motors, 9V Battery and connector* 2. MeArm Robot (Laser Cut, assembled by hand) 3. Walking Robot (Laser cut, assembled by hand) with 1 DC Motor and Battery soldered on 4. Miscellaneous: Screwdriver, Soldering Iron, Solder, jumper wires etc
• 4.
  | © Copyright2023, InfluxData 4 Why Arduino based Robotics ? - Your entry to robotics! - Simulates real world use cases : Crane operation, Robot Dog - Real robots are expensive! - Open source code, DIY plans, Affordable robot kits are easily available Boston Dynamic Spot Robot Industrial Crane
• 5.
  | © Copyright2023, InfluxData 5 Software 1. Open Source Arduino & Python program using multiple libraries 2. GestureRecognizer.py: Using MediaPipe and pre-trained model to detect hand gestures and showing using OpenCV 3. MotorController.py: Connects to Arduino and motor using PySerial and sends positional data to servo motor(s) 4. InfluxDBWriter.py : Connectina and writing values at time intervals very quickly to the DB
• 6.
  | © Copyright2023, InfluxData 6 write() query()* pyserial MediaPipe for Gesture Recognition InfluxDB Arduino Uno R4 (WiFi, Serial) hand gesture rotation*/name is sent to the servo motor
• 7.
  | © Copyright2023, InfluxData 7 Live Code Demo & Code Walkthrough! 7
• 8.
  | © Copyright2023, InfluxData 8 Why Hand Motion and Gesture? - Natural User Interface, alternate would game controller, keypads - Easy to detect with Machine Learning - MediaPipe - Can be trained for custom gestures, movements etc and deployed on web, mobile, IoT devices for free! - Trendy - AR, VR etc
• 9.
  | © Copyright2023, InfluxData 9 Why Arduino ? - Affordable specially ESP-32 boards - Accessible : Open source, lot’s of peripherals like motors, sensors etc - Huge community online sharing Code, guides answering questions etc - More fun! - Can talk to Rapsberry Pi, Web, computer, Mobile etc
• 10.
  | © Copyright2023, InfluxData 10 Why a Time Series Database ? - Optimized for ingestion of large quantities of data rapidly - InfluxDB is a leading Open Source & Cloud time series database available to get started for free! (no credit card required) - For Logging & Monitoring (Observability) - For saving state of the motor
• 11.
  | © Copyright2023, InfluxData 11 User Interface : Hand landmarks and gesture detection to control robot
• 12.
  | © Copyright2023, InfluxData 12 Natural Human Interface Robot Arm Human hand AI
• 13.
  | © Copyright2023, InfluxData 13 | © Copyright 2023, InfluxData 13 References 1. Google MediaPipe: https://github.com/google-ai-edge/mediapipe 2. Robot Arm: https://www.meetup.com/creativecode/ 3. InfluxDB: https://www.influxdata.com/ 4. Arduino Uno: https://store.arduino.cc/products/uno-r4-wifi 5. WebSerial: https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API 6. PySerial: https://github.com/pyserial/pyserial 7. OpenCV: https://opencv.org/ 8. Code on GitHub: https://github.com/suyashcjoshi/HandGestureBot
• 14.
  | © Copyright2023, InfluxData 14 | © Copyright 2023, InfluxData 14 Thank you! Suyash Joshi @suyashcjoshi sjoshi@influxdata.com