Virtual reality simulation of a smart eating table for handicapped people

This thesis deals with simulated human-robot interaction and how to use a robot in different everyday life activities. It is in line with the significant developments in the sector of artificial intelligent and robotic fields that have enabled the emergence of a new smart robot generation. The idea of this project came from the large number of disabled people, especially handicapped people. This number has been substantially increased by the terrorist war in my country. The challenge of this task is how to build a new smart robotic system that can help those people in everyday life activities, especially in eating and drinking. Due to many reasons that will be clarified in the first chapter, we decided to use simulation software to implement such a system. The proposed simulated system comprises three sub-systems: the robotic system (a JACO manipulator arm), the semantic camera system – which can detect the different environment objects according to their types and names, which finally gives us the coordinates for each one – and the sound recognition system, which will recognize the words from the person using it. It has been programmed to refer to the process type (eating or drinking), as well as the food types. The whole environment has been built using a simulation program (MORSE) that completely supports Python and is based on Blender (a 3D drawing and animation program). The simulated environment comprises a human model, a food table, two dishes with different colours to represent the food types, a kitchen room with some plants, heaters, sink, and tables. The robot arm has been implemented with the same design as the real arm. It includes the same joints, links, and functions. The robotic arm simulated in this project is a JACO robotic arm. This arm has been designed especially for disabled people who have minor problems with eating and drinking. We improved this arm by adding two systems, one for semantic camera system and one for sound. This will help people who cannot use their hands. An external microphone is connected to the PC and starts its function when the program starts, whereby the system is controlled by using a robot operation system (ROS) that supports C++. The system then starts with the human saying the required word, which represents the food type or a drinkable liquid (e.g. water). Each of the available food types has a specific word. When the person says this word, it will be recognized by the sound recognition system. Subsequently, the semantic camera system will detect the coordinates of the required food or drink, which will be stored and then transferred to move the robotic arm. The robotic arm will drive to the food dish or the water cup and pick up it, before moving to the mouth and finally returning the rest food or cup to the first position and continuing in this execution loop. This project will be very helpful to people who have physical problems with eating or drinking, especially for disabled and handicapped people.