Commit 29943cab authored by burcharr's avatar burcharr 💬
Browse files

automatic writing commit ...

parent 7f299467
......@@ -50,12 +50,21 @@ To conclude the results of problem 3, the overall performance of this more diffi
### Practical applicability
The data from the real world evaluation with our test subjects shows, that most real world hand washing procedures are detected by our smart watch system. Overall, the system's sensitivity was ... in the evaluation of a "normal day", which is ... compared to the theoretical results. However, this was to be expected, since real hand washing knows many forms and patterns, that are unlikely to all be captured during the explicit recording of training data. Our theoretical results could therefore not be reached in the real life scenario. Because of the smoothing that was applied to the data, at least some consecutive windows must be classified into the positive class, which means that a real hand washing procedure needs to be longer than or around $10\,s$. In practice, it can happen that washing ones hands does take a shorter amount of time, which the system will then not detect properly. All in all, the system was able to correctly detect most hand washing procedures, and is therefore somewhat effective at this task.
The data from the real world evaluation with our test subjects shows, that most real world hand washing procedures are detected by our smart watch system. Overall, the system's sensitivity was ... in the evaluation of a "normal day", which is ... compared to the theoretical results. However, this was to be expected, since real hand washing knows many forms and patterns, that are unlikely to all be captured during the explicit recording of training data. Our theoretical results could therefore not be reached in the real life scenario. Because of the smoothing that was applied to the data, at least some consecutive windows must be classified into the positive class, which means that a real hand washing procedure needs to be longer than or around $10\,s$. In practice, it can happen that washing ones hands does take a shorter amount of time, which the system will then not detect properly. All in all, the system was able to correctly detect most hand washing procedures, and is therefore somewhat effective at this task.
The data also showed, that higher intensity or a longer duration of the hand washing have a positive influence on the detection probability by the model on the smart watch. This seems logical for the longer duration due to the smoothing, but also for the intensity, it can be assumed, that the system can reach higher certainties with high intensity compared to low intensity washing. TODO (check if this holds for final results)
Added to that, the system did detect an average of xy TODO false positives per subject per hour. These false positives could lead to annoyances and ultimately to the users loosing trust in the detection capabilities of the system
## Comparison of goals to results
TODO
#### Detection of hand washing in real time from inertial motion sensors
The goal of detecting hand washing and separating it from other activities in real time was reached by employing the trained DeepConvLSTM-A network which achieved good performance in our evaluation. The detection is not perfect yet, especially the separation from other activities seems to still have some weaknesses, especially when washing activities other than hand washing are included. However, the system was able to detect and correctly identify hand washing in most of the cases, which is why we consider our goal reached.
#### Separation of hand washing and compulsive hand washing
The separation of hand washing from compulsive hand washing worked extremely well for the theoretical evaluation, which is the only evaluation we were able to test it with. A sensitivity of $99,7\,\%$ was reached with smoothing, while maintaining a specificity of $83,9\,\%$. This means that almost all compulsive hand washing in our test data was detected by the system, although the false positive rate is still a bit higher than we want it to be. Nevertheless, the performance of the model trained for this problem was really strong and fully matched our expectations. We think that a performance on this level in the real world could possibly really be applied in the treatment of patients with OCD, which is why we consider this goal as reached, too.
#### Real world evaluation
The real world evaluation provided us with valuable feedback, showing us strengths and weaknesses of the hand washing detection model. Especially for the previously mentioned washing activities, the evaluation showed us the need of their inclusion as negative training examples. Apart from the false positives, the real world evaluation confirmed our hopes of the system actually being able to detect hand washing with high precision. Although the performance in the real world test was lower than the one of the theoretical evaluation, the real world evaluation still yielded a strong performance (TODO: check if true). The estimation of performance for the intense and long washing task (scenario 2) was much closer to the performance reached on our pre-recorded test set, which showed that the system is able to detect stronger hand washing even better. Overall, the real world evaluation was really successful, returning us valuable information about the weak points and strengths of the system so far.
## Future work
......
......@@ -34,6 +34,6 @@ reviewer2: "TBD"
declaration: Hiermit erkläre ich, dass ich diese Arbeit selbstständig verfasst habe, keine anderen als die angegebenen Quellen/Hilfsmittel verwendet habe und alle Stellen, die wörtlich oder sinngemäß aus veröffentlichten Schriften entnommen wurden, als solche kenntlich gemacht habe. Darüber hinaus erkläre ich, dass diese Arbeit nicht, auch nicht auszugsweise, bereits für eine andere Prüfung angefertigt wurde.
#abstract
abstract-de: In dieser Arbeit... todo
abstract-en: In this thesis, ... todo
abstract-de: Die automatische Erkennung von Händewaschen und zwanghaftem Händewaschen hat mehrere Anwendungsbereiche in Arbeits- und medizinischen Umgebungen. Die Erkennung von Händewaschen kann in zur Überprüfung der Einhaltung von Hygieneregeln eingesetzt werden, da das Händewaschen eine der wichtigsten Komponenten der persönlichen Hygiene ist. Allerdings kann das Händewaschen auch übertrieben werden, was bedeutet, dass es für die Haut und die allgemeine Gesundheit schädlich sein kann. Manche Patienten mit Zwangsstörungen waschen sich zwanghaft und zu häufig die Hände auf diese schädliche Weise. Die automatische Erkennung von zwanghaftem Händewaschen kann bei der Behandlung dieser Patienten helfen. Ziel dieser Arbeit ist es, auf neuronalen Netzen basierende Methoden zu entwickeln, die in der Lage sind, Händewaschen und zwanghaftes Händewaschen in Echtzeit auf einem am Handgelenk getragenen Gerät zu erkennen, wobei die Daten der Bewegungssensoren des am Handgelenk getragenen Geräts verwendet werden. Wir erreichen eine hohe Genauigkeit für beide Aufgaben und evaluieren Teile der Arbeit mit Probanden in einem realen Experiment, um die starke theoretische Leistung zu bestätigen.
abstract-en: The automatic detection of hand washing and compulsive hand washing has multiple areas of application in work and medical environments. Hand washing detection can be used in compliance and hygiene scenarios, as hand washing is one of the main components of personal hygiene. However, hand washing can also be overdone, which means it can be hurtful to the skin and general health. Patients with obsessive-compulsive disorder sometimes compulsively wash their hands in such a harmful way. In order to help with their treatment, the automatic detection of compulsive hand washing can possibly be applied. This thesis aims to develop neural network based methods which are able to detect hand washing as well as compulsive hand washing in real time on a wrist worn device using intertial motion sensor data of said wrist worn device. We achieve high accuracy for both tasks and evaluate parts of the work on subjects in a real world experiment, in order to confirm the strong theoretical performance achieved.
---
......@@ -106,6 +106,10 @@ The duration and intensity of the hand washing process also played a role, as ca
The highest accuracy was achieved for ....
The lowest accuracy was achieved for ....
The correlation of duration of the hand washing with the detection rate is ...
Likewise, the correlation of the intensity of washing with the detection rate is ... TODO
For the reported false positives, the subjects experiences varied. The subjects reported xy (+-) false hand washing detections on this day. Assuming a 12h recording period, that means there are xy / 12 false detections per hour.
The activities leading to false positives include:
......
......@@ -15,7 +15,7 @@
& \textbf{LSTM-A} & 0.470 & 0.919 & 0.844 & 0.622 \\
& \textbf{Majority Classifier} & \textbf{1.000} & 0.000 & 0.000 & 0.000 \\
& \textbf{RFC} & 0.938 & 0.422 & 0.581 & 0.582 \\
& \textbf{Random Classifier} & 0.344 & 0.660 & 0.667 & 0.452 \\
& \textbf{Random Classifier} & 0.329 & 0.662 & 0.666 & 0.439 \\
& \textbf{SVM} & 0.939 & 0.396 & 0.555 & 0.557 \\
\cline{1-6}
\multirow{10}{*}{\textbf{True }} & \textbf{CNN} & 0.510 & 0.730 & 0.742 & 0.600 \\
......@@ -26,7 +26,7 @@
& \textbf{LSTM-A} & 0.355 & 0.762 & 0.734 & 0.484 \\
& \textbf{Majority Classifier} & \textbf{1.000} & 0.000 & 0.000 & 0.000 \\
& \textbf{RFC} & 0.946 & 0.205 & 0.332 & 0.336 \\
& \textbf{Random Classifier} & 0.344 & 0.660 & 0.667 & 0.452 \\
& \textbf{Random Classifier} & 0.329 & 0.662 & 0.666 & 0.439 \\
& \textbf{SVM} & 0.951 & 0.266 & 0.412 & 0.415 \\
\bottomrule
\end{tabular}
......
No preview for this file type
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment