Setting Up an Anonymous Gesture Database as Well as Enhancing It with a Verbal Script Simulator for Rehabilitation Applications

Physical therapy patients are rehabilitated by performing exercises at home that do not consider proper movement and can be detrimental to the healing process. Maintaining patient anonymity is an important aspect of collecting patient data. Using our method, we are able to collect information about limb movements in a completely anonymous manner by taking a picture of the patient in the clinic and immediately converting the picture into an anatomical skeleton. A human gesture database accompanied by a verbal script simulator and anonymous tagging was created with the intention of tagging, measuring, and inferring human gestures using neural networks. We have developed a system that utilizes neural network autoencoder architecture to classify the quality and accuracy of patients’ movements in videos. Since there is a lack of videos of tagged physiotherapy exercises, we simulate patients’ movements to enhance the database. The purpose of this paper is to describe a simulator that mimics the output of OpenPose software so that synthetic human skeletal movements can be computed without utilizing OpenPose. As inputs, these vectors are fed to the autoencoder which, after compressing them into low dimension vectors, classifies them according to their movement using the Dynamic Time Warping (DTW) distance algorithm. Validation of the research was performed on a dataset of 7 different physiotherapy exercises, and 91.8% accuracy was achieved.