IRJET-Gesture Recognition Robot using Digital Image Processing

IRJET-Gesture Recognition Robot using Digital Image Processing

• 1.
  International Research Journalof Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 846 Gesture Recognition Robot Using Digital Image Processing Supriya Zinjad Department of Electronics & Telecommunication, Bharati Vidyapeeth’s College of Engineering for Women, Pune, India ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - Robots are increasingly being integrated into working functions for replacing humans and performing various tasks. The main aim of this project is to create an interaction between human hand and a Robotic arm. TheArm is being controlled by human hand gestures with the help of Image Processing techniques. This idea introduces the technology used to control the Robotic Arm for different activities. Non-contact type of mechanism is being used for different hand gestures which are to be recognized by MATLAB Software using RGB color strips. For each position of hand there is a color to detect its position. These colors will be detected by MATLAB and position will be given to each color depending upon the axis defined for the video screen. This is real time application in that whatever action is performed by human hand the exact action will be performedbyroboticarm using Arduino Key Words: Arduino, RGB, MATLAB,RoboticArm,L293D, Gesture, Contact, Non-Contact. 1. INTRODUCTION This paper introduces the technology which uses tracking the human arm and interfacing it to control the Robotic Arm for performing different operations. Recently many Robotic arms are used in industries which are trained to perform specific operations like diffusing bomb, handling hazardous chemicals, picking and placing objects etc. This project is implemented using MATLAB through whichwecancalculate the distance and angle between X and Y axis. For each position of our hand there are colour strips to detect its position that is from shoulder to elbow, elbow to wrist. These colours are detected using MATLABandgivesposition to the Robotic Arm depending upon the axis defined for the video screen. These axis locations are captured and interfaced with the controller. The controller will give the command to the robot then particular robotic servo motor will move which is placed in the axis of robotic arm. 2. METHODOLOGY In this area two types of techniques are used: 1. Non-Contact type 2. Contact type Fig 1- Block Diagram 2.1. Non- Contact type: In Non- Contact type technique our hand is not in contact with any type of devices. 2.2. Contact type: In Contact type technique Arm circuitry are in contact with the human hand. Such devices use sensors which will send the signal to controller to receive the data. 3. OBJECTIVES The main aim of this project is to create human-machine interface. In this project non-contact type of tracking system is used so that we do not have any type of equipment on our hand. We are sticking RGB color band which will be tracked by using MATLAB. Hand Gestures will be performedandwill be sent via Bluetooth HC05 which is a wirelesstransmission media to the robotic arm. All the actions of the robot can be controlled by the user and operations like bomb diffusion, hazardous chemical handling, pick and place of objects. 4. WORKING 4.1. Gesture Recognition Input may be taken as a video signal of the human hand and analysis is done to track the different color strips on our hand at different locations. Video signal may be taken by a
• 2.
  International Research Journalof Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 847 laptop camera or a camera build on robotic arm itself, which will make it portable and efficient for handling. 4.1.1. Tracking colors Tracking of colors contain RGB color strips, assigned to the user’s hand. These colors are assigned from shoulder to elbow, from elbow to wrist and ahead of wrist. To convert the image from normal image to gray image the 3D image will get converted to 2D image. 4.1.2. Calculating angles Position of angle from the axis is calculated using the trigonometric equations. These angles will tell rotation and direction of user’shand. The angle signal istransmittedfrom MATLAB to the micro-controller to move the robotic arm and process the signal. This processed signal is then transmitted to the robotic arm to perform various actions. By using following equations, the angle between X andYaxis can be calculated. 4.2. Machine Interface A boundary which shares two separate components of a computer system and exchange information is called as an interface. This exchange can be between software,computer hardware, peripheral devices, humans and combinations of these. 4.2.1. Video camera A video camera of a laptop is used which streamsitsimagein real time. When the image is captured it is saved to the receiving laptop via Bluetooth HC05 if the transmission is wireless or via USB cable if its wired transmission. 4.2.2. MATLAB Programming MATLAB is a numerical computing environment developed by MathsWork. MATLAB is used because it allows us to perform Matrix manipulation, plotting of functions of data and implementation of algorithmsand creation of user interface is easily possible. 4.2.3. Arduino language Programming An Arduino board consists of an Atmel 8-bit AVR microcontroller. It has complementary components that facilitate programming. Arduino connects to the PC usingan USB port which acts like a Serial connection. Arduino programsare written in C or C++. The Arduino IDE is usedto upload programs. 4.2.4. Features of Arduino board A 16mhz clock which makes it the speediest micro- controller.32 KB of flash memory is used for storing user’s code. There are 13 digital pins and 6 analog pins which allows user to connect external hardware to Arduino. These pins are key for extending the computing capability of the Arduino. Plug the devices and sensors into the sockets that correspond to each of these pins. An ICSP connector is used for bypassing the USB port and interfacing the Arduino directly asa serial device. An on-board LED isattachedtothe digital pin number 13 for fast an easy debugging of code. 4.2.5. Bluetooth Module HC05 HC-05 is an easy to use Serial Port Protocol(SPP) module. It is designed for transparent wireless serial connectionsetup. Serial Port Bluetooth module is a fully qualified Bluetooth V2.0+Enhanced Data Rate of 3Mbps modulation with complete 2.4GHz radio trans receiver and baseband. 4.3. Actuation of Robotic Arm A Robotic arm is a mechanical arm which is programmable. This type of arm can be used to perform any type of application such as welding, gripping etc. The robotic arm contains different servo motors used for axial rotations of hand. In this project every gesture performed by the user corresponds to a function which is written and stored in database which controls the robotic arm. Whenever the gesture matches with a gesture from the database, the instruction set corresponds to that gesture which identifies and passes it on to the robotic arm for execution. In this way the robotic arm can be controlled by hand gesturesusinglive camera.
• 3.
  International Research Journalof Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 01 | Jan-2018 www.irjet.net p-ISSN: 2395-0072 © 2018, IRJET | Impact Factor value: 6.171 | ISO 9001:2008 Certified Journal | Page 848 4.4. L293D Connections The pin diagram shown is the connection pins of L293D IC. Pin1 and Pin9 are” Enable” pins. They should be connected to +5V for the drivers to function. If they are pulled low (GND), then the outputs will be turned off, stopping the motors. Pin4, Pin5, Pin12 and Pin13 are ground pins which should ideally be connected to microcontroller’s ground. Pin2, Pin7, Pin10 and Pin15 are logic input pins. These are control pins which are connected to micro-controller pins. Pin2 and Pin7 control the first motor (left); Pin10 and Pin15 control the second motor(right). Pin3, Pin6, Pin11, and Pin14 are output pins. Tie Pin3 and Pin6 to the first motor, Pin11 and Pin14 to second motor. Pin16 powers the IC and it should be connected to regulated +5Volts. Pin8 powers the two motors and should be connected to positive lead of a secondary battery. As per the data sheet, supply voltage can be as high as 36 Volts. 5. APPLICATIONS 5.1. INDUSTRIAL APPLICATIONS: This kind of robotic arm can be used in any industrial application to pick and place very heavy materials which a human cannot do by their hand. 5.2. BOMB DIFFUSION: Humans are scared to go to hazardous situation, but robotic arm can be moved far over the distance to do the action depending upon themovement of human hand and to diffuse the bomb. 5.3. ELECTRICAL CONNECTIONS:Veryhighvoltageelectrical connections can be done by the robotic arm very accurately. 5.4. MEDICAL APPLICATIONS: Some medical applications which are hazardous to human being like a virus called as ’EBOLA’ which a human can’t touch. 6. CONCLUSION In this project the implementation of human hand gesture recognition using robotic arm is shown. The arm is controlled by the user’s hand and it will imitate all the actions of the user. This technique can be used even if the user is dumb or deaf asit only imitates the action of the user. 7. REFERENCES [1] Rafiqul ,“HAND GESTURE RECOGNITION: A LITERATURE REVIEW”, International Journal of Artificial Intelligence & Applications(IJAIA) [2] “MEMS Based Gesture Controlled Robot Using Wireless Communication.”, International Journal of Engineering Trends and Technology (IJETT) – Aug 2014 [3] Giulio Marin, Fabio Dominio, Pietro Zanuttigh,” HAND GESTURE RECOGNITION WITH LEAP MOTION AND KINECT DEVICES”, Department of Information Engineering, University of Padova.