Commit 48e5c2d3 authored by burcharr's avatar burcharr 💬
Browse files

automatic writing commit ...

parent ad56a5bc
# Introduction
In this thesis we aim to develop several neural network based machine learning methods that can be used to detect hand washing and compulsive hand washing in inertial sensor data of wrist worn devices. We evaluate different approaches for multiple scenarios of hand washing classification. We examine the real world applicability of the developed approach with multiple users.
In this thesis we aim to develop several neural network based machine learning methods that can be used to detect hand washing and compulsive hand washing on inertial sensor data of wrist worn devices. We evaluate different approaches for multiple scenarios of hand washing classification. We examine the real world applicability of the developed approach with multiple users.
## Motivation
### Hand washing detection
Hand washing is an important part of every humans personal hygiene. We wash our hands many times every day. Washing ones hands can remove dirt or grease and importantly helps to prevent infection with pathogens @noauthor_when_2020. There are many occasions, in which it is desired that we wash our hands, among which are @noauthor_when_2020:
Hand washing is an important part of every human's personal hygiene. We wash our hands multiple times each day. Washing ones hands can remove dirt or grease and importantly helps to prevent infection with pathogens @noauthor_when_2020. There are many occasions, in which it is desired that we wash our hands, among which are @noauthor_when_2020:
- After using the toilet
- Before and after preparing or eating food
......@@ -17,28 +17,27 @@ Added to that, hand washing using soap or disinfectants is also part of the work
In order to monitor the effectiveness and frequency of hand washing, we could use a sensor based computer system to detect the activity of hand washing and its duration. Further advanced systems could also be used to predict the quality of the hand washing. These systems could then be used to reduce the risk of contaminations or infections by ameliorating the hygiene of their users.
### Obsessive-Compulsive Disorders
While it is usually really helpful and a basic part of hygiene, hand washing can also be overdone, i.e. be too frequent or be done too thoroughly. One example of persons for which overly excessive hand washing is a problem, is the small percentage of humans suffering Obsessive-Compulsive Disorders (OCD). OCD affects about $1-3\,\%$ of humans during their life @valleni-basile_frequency_1994, @fawcett_women_2020. OCD appears in the form of obsessions, that lead to compulsive behavior. There are multiple subgroups of obsessions and compulsions, including contamination concerns, symmetry and precision concerns, saving concerns and more @stein_obsessive-compulsive_2002. These concerns lead to respective compulsive behavior: Symmetry and precision concerns lead to arranging and ordering, saving concerns lead to hoarding and contamination concerns can lead to excessive washing, bathing and showering. This work's will focus on detecting hand washing and also trying to tell it apart from compulsive hand washing of OCD patients.
While it is usually really helpful and a basic part of hygiene, hand washing can also be overdone, i.e. be too frequent or be done too thoroughly. One example of persons for which overly excessive hand washing is a problem, is the small percentage of humans suffering from Obsessive-Compulsive Disorders (OCD). OCD affects about $1-3\,\%$ of humans during their life @valleni-basile_frequency_1994, @fawcett_women_2020. OCD appears in the form of obsessions, that lead to compulsive behavior. There are multiple subgroups of obsessions and compulsions, including contamination concerns, symmetry and precision concerns, saving concerns and more @stein_obsessive-compulsive_2002. These concerns lead to respective compulsive behavior: Symmetry and precision concerns lead to arranging and ordering, saving concerns lead to hoarding and contamination concerns can lead to excessive washing, bathing and showering. This work will focus on detecting hand washing and also try to tell apart hand washing from compulsive hand washing of OCD patients.
The separation of compulsive hand washing from ordinary hand washing is an even harder problem than just hand washing detection itself. It is unclear, whether it is possible to predict the type of hand washing with high probability, as there is no previous work in this area.
The separation of compulsive hand washing from ordinary hand washing is an even harder problem than just hand washing detection itself. It is unclear, whether it is possible to predict the type of hand washing with high probability, as there is no previous work in this area. It is reasonable to assume, that their are strong similarities between the kinds of hand washing, as well as subtle differences, e.g. in intensity and length.
One method of treatment for clinical cases of OCD is exposure and response prevention (ERP) therapy @meyer_modification_1966 @whittal_treatment_2005. Using this method, patients that suffer from OCD are exposed to situations in which their obsessions are stimulated and they are helped at preventing compulsive reactions to the stimulation. The patients can then "get used" to the situation in a sense, and thus the reaction will be weakened over time.
One method of treatment for clinical cases of OCD is exposure and response prevention (ERP) therapy @meyer_modification_1966 @whittal_treatment_2005. Using this method, patients that suffer from OCD are exposed to situations in which their obsessions are stimulated and they are helped at preventing compulsive reactions to the stimulation. The patients can then "get used" to the situation in a sense, and thus the reaction to the stimulation will be weakened over time. This means that their quality of life is improved, as the severity of their OCD declines.
A successful, i.e. reliable and accurate system for obsessive hand washing detection could be used to intervene, whenever the compulsive hand washing is detected. It could therefore help psychologists and their patients in the treatment of the symptoms. It could help the user to stop the compulsive behavior by issuing a warning. Such a warning could be a vibration of the device, or a sound that is played upon the detection of compulsive behavior. However, the hypothesis of usefulness is yet to be tested, as no such systems exists as of now. Therefore we want to develop a system that can not only detect hand washing, but also discriminate between usual hand washing and obsessive-compulsive hand washing.
A successful, i.e. reliable and accurate system for obsessive hand washing detection could be used to intervene, whenever the compulsive hand washing is detected. It could therefore help psychologists and their patients in the treatment of the symptoms. It could help the user to stop the compulsive behavior by issuing a warning. Such a warning could be a vibration of the device, or a sound that is played upon the detection of compulsive behavior. However, the hypothesis of usefulness is yet to be tested, as no such systems exists as of now. Therefore we want to develop a system that can not only detect hand washing with low latency and in real time, but also discriminate between usual hand washing and obsessive-compulsive hand washing at the same time. The system could then, as described, be used in ERP therapy sessions, but also in every day life, to prevent compulsive hand washing.
### Wrist worn sensors
Different types of sensors can be used to detect activities such as hand washing. It is possible to detect hand washing from RGB camera data to some extent. However, in order for this to work, we would need to place a camera at every place and room a subject could want to wash their hands at. This is unfeasible for most applications of hand washing detection, and could be very expensive. Added to that it might be problematic to place cameras inside wash or bath rooms for privacy reasons. Thus, a better alternative could be body worn, camera-less devices.
Inertial measurement units (IMUs) can measure different types of time series movement data, e.g. the acceleration or angular velocity of the device they are embedded in. IMUs are embedded in most modern smart phones and smart watches, which makes them easily available. For hand washing detection, especially the movement of the hands and wrists can contain information that can help us classify hand washing. Therefore, we can use a smart watch and its embedded IMU to try to predict whether a user is washing their hands or not. Added to that, if the user is washing their hands, we could try to predict if they are washing them in an obsessive-compulsive way or not. Another advantage of using a smart watch would be, that they usually have in-built vibration motors or even speakers. These means could be used to intervene, whenever compulsive hand washing is detected, as described above.
Inertial measurement units (IMUs) can measure different types of time series movement data, e.g. the acceleration or angular velocity of the device they are embedded in. IMUs are embedded in most modern smart phones and smart watches, which makes them easily available. For hand washing detection, especially the movement of the hands and wrists can contain information that can help us classify hand washing. Therefore, we can use a smart watch and its embedded IMU to try to predict whether a user is washing their hands or not. Added to that, if the user is washing their hands, we could try to predict if they are washing them in an obsessive-compulsive way or not. Another advantage of using a smart watch would be, that they usually have in-built vibration motors or even speakers. These means could be used to intervene, whenever compulsive hand washing is detected, as described above. Therefore, wrist worn sensors, especially those embedded into the very versatile smart watch systems, are used in this work. The wrist worn devices can also be used to execute machine learning models in real time, using publicly available libraries, e.g. on smart watches running Wear OS.
## Goals
In this work, we want to develop a method for the real time detection of hand washing and compulsive hand washing. We also want to test the method and report meaningful statistics of its success. Further, we want to test parts of the developed method in a real world scenario. We then want to draw conclusions on the applicability of the developed systems in the real world.
### Detection of hand washing in real time from inertial motion sensors
### Detection of hand washing in real time utilizing inertial measurement sensors
We want to show that neural network based classification methods can be applied to the recognition of hand washing. We want to base our method on sensor data from inertial measurement sensors in smart watches or other wrist worn IMU-equipped devices. We want to detect the hand washing in real time and directly on the mobile, i.e. on a wrist wearable device, such as a smart watch. Doing so, we would be able to give instant real time feedback to the user of the device.
### Separation of hand washing and compulsive hand washing
Added to the detection of hand washing, the detection of obsessive-compulsive hand washing is part of our goals. We want to be able to separate compulsive hand washing from non compulsive hand washing, based on the inertial motion data. Especially for the scenario of possible interventions used for the treatment of OCD, this separation is crucial, as patients do also wash their hands in non compulsive ways.
Added to the detection of hand washing, the detection of obsessive-compulsive hand washing is part of our goals. We want to be able to separate compulsive hand washing from non compulsive hand washing, based on the inertial motion data. Especially for the scenario of possible interventions used for the treatment of OCD, this separation is crucial, as OCD patients do also wash their hands in non compulsive ways and we do not want to intervene for these kinds of hand washing procedures.
### Real world evaluation
We want to evaluate the most promising of the developed models in a real world evaluation, in order to obtain a realistic estimate of its applicability in the task of hand washing detection. We want to report results of an evaluation with multiple subjects to obtain a meaningful performance estimation. From this estimation we want to draw conclusions on the applicability of the developed system in real world therapy scenarios.
\ No newline at end of file
We want to evaluate the most promising of the developed models in a real world evaluation, in order to obtain a realistic estimate of its applicability in the task of hand washing detection. We want to report results of an evaluation with multiple subjects to obtain a meaningful performance estimation. From this estimation we want to draw conclusions on the applicability of the developed system in real world therapy scenarios. Added to that, we want to derive future improvements, that could be applied to the system.
\ No newline at end of file
......@@ -35,5 +35,5 @@ declaration: Hiermit erkläre ich, dass ich diese Arbeit selbstständig verfasst
#abstract
abstract-de: Die automatische Erkennung von Händewaschen und zwanghaftem Händewaschen hat mehrere Anwendungsbereiche in Arbeits- und medizinischen Umgebungen. Die Erkennung von Händewaschen kann in zur Überprüfung der Einhaltung von Hygieneregeln eingesetzt werden, da das Händewaschen eine der wichtigsten Komponenten der persönlichen Hygiene ist. Allerdings kann das Händewaschen auch übertrieben werden, was bedeutet, dass es für die Haut und die allgemeine Gesundheit schädlich sein kann. Manche Patienten mit Zwangsstörungen waschen sich zwanghaft und zu häufig die Hände auf diese schädliche Weise. Die automatische Erkennung von zwanghaftem Händewaschen kann bei der Behandlung dieser Patienten helfen. Ziel dieser Arbeit ist es, auf neuronalen Netzen basierende Methoden zu entwickeln, die in der Lage sind, Händewaschen und zwanghaftes Händewaschen in Echtzeit auf einem am Handgelenk getragenen Gerät zu erkennen, wobei die Daten der Bewegungssensoren des am Handgelenk getragenen Geräts verwendet werden. Wir erreichen eine hohe Genauigkeit für beide Aufgaben und evaluieren Teile der Arbeit mit Probanden in einem realen Experiment, um die starke theoretische Leistung zu bestätigen.
abstract-en: The automatic detection of hand washing and compulsive hand washing has multiple areas of application in work and medical environments. Hand washing detection can be used in compliance and hygiene scenarios, as hand washing is one of the main components of personal hygiene. However, hand washing can also be overdone, which means it can be hurtful to the skin and general health. Patients with obsessive-compulsive disorder sometimes compulsively wash their hands in such a harmful way. In order to help with their treatment, the automatic detection of compulsive hand washing can possibly be applied. This thesis aims to develop neural network based methods which are able to detect hand washing as well as compulsive hand washing in real time on a wrist worn device using intertial motion sensor data of said wrist worn device. We achieve high accuracy for both tasks and evaluate parts of the work on subjects in a real world experiment, in order to confirm the strong theoretical performance achieved.
abstract-en: The automatic detection of hand washing and compulsive hand washing has multiple areas of application in work and medical environments. Hand washing detection can be used in compliance and hygiene scenarios, as hand washing is one of the main components of personal hygiene. However, hand washing can also be overdone, which means it can be unhealthy for the skin and general health. Patients with obsessive-compulsive disorder sometimes compulsively wash their hands in such a harmful way. In order to help with their treatment, the automatic detection of compulsive hand washing can possibly be applied. This thesis aims to develop neural network based methods which are able to detect hand washing as well as compulsive hand washing in real time on a wrist worn device using intertial motion sensor data of said wrist worn device. We achieve high accuracy for both tasks and evaluate parts of the work on subjects in a real world experiment, in order to confirm the strong theoretical performance achieved.
---
......@@ -302,7 +302,7 @@ The sensitivity is the rate of positive samples that get correctly recognized, t
For the multiclass problem of distinguishing obsessive hand washing from normal hand washing from other activities, the binary metrics are not applicable. Here, we report normalized confusion matrices, and their mean diagonal values as one performance measure. The confusion matrix shows, which amount of samples belonging to a certain class (true label, rows of the matrix) are predicted to belong to which other class (predicted label, columns of the matrix). The normalized version of the confusion matrix replaces the total values by ratios in proportion to the amount of true labels for each class. This means that for each true label row in the matrix, the values sum to 1.
The mean diagonal value of this matrix can be seen as a mean class accuracy score, as the diagonal values of the normalized confusion matrix are the accuracy values for each possible class.
Added to that, we report an adapted F1 score. The adapted multiclass F1 score is calculated by taking the mean over all classes $\mathbf{C}$, of the F1 scores if we treat the class $\mathbf{C}_i, i \in [0,1,2]$ as the positive class, and the remaining classes as the negative class:
Added to that, we report an adapted F1 score, identical to the one used by Zeng et al. @zeng_understanding_2018. The adapted multiclass F1 score is calculated by taking the mean over all classes $\mathbf{C}$, of the F1 scores if we treat the class $\mathbf{C}_i, i \in [0,1,2]$ as the positive class, and the remaining classes as the negative class:
\begin{align}
F_1\ score\ multi = \frac{1}{3}\cdot \sum_{i=0}^2 F_1\ score(\mathbf{C}_i)
\end{align}
......
This diff is collapsed.
......@@ -119,7 +119,7 @@ The activities leading to false positives include:
- Brushing teeth
- Cleaning
The full list of reported activities can be found in the appendix.
The full list of reported activities for which false positives occurred can be found in the appendix.
Some subjects also reported difficulties with the smart watch application (not part of this work), which lead to the model not being run at all sometimes, which might also have influenced the results. It could be possible, that for some hand washing procedures, the smart watch application was not executed, which would lead the user to note down a false negative, also decreasing the sensitivity in the results.
......
No preview for this file type
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment