Commit aebc2c27 authored by burcharr's avatar burcharr 💬
Browse files

automatic writing commit ...

parent 0213765f
......@@ -14,7 +14,7 @@ In all tables of this chapter, the best values for a specific metric will be hig
The values for the metrics specificity and sensitivity will be reported in the tables, but not discussed separately, because they are included in the more meaningful metrics F1 score and S score. The results generally show that achieving a high value in only one metric out of specificity and sensitivity, at cost of reaching low values in the other one, brings about worse performance in the F1 score and S score.
### Distinguishing hand washing from all other activities
For the first task of classifying hand washing in contrast to non hand washing activities, we report the results with and without the application of label smoothing. The results without label smoothing are shown in table \ref{tbl:washing} and @fig:p1_metrics.
For the first task of classifying hand washing in contrast to non hand washing activities, we report the results with and without the application of label smoothing. The results without label smoothing and without normalization are shown in table \ref{tbl:washing} and @fig:p1_metrics.
\input{tables/washing.tex}
......@@ -28,11 +28,11 @@ As we can see, without label smoothing, the neural networks outperformed the con
![F1 score and S score for problem 1, with smoothing](img/washing_rm.pdf){#fig:p1_metrics_rm width=98%}
With label smoothing, we can reach an increased performance with all of the model classes, including the traditional machine learning methods RFC and SVM. The results with a 20 prediction wide average filter smoothing can be seen in table \ref{tbl:washing_rm} and @fig:p1_metrics_rm. The top performing neural network architectures do not change with the smoothing. However, the performance measures increase. DeepConvLSTM has the best F1 score ($0.892$), followed by LSTM-A ($0.891$), DeepConvLSTM-A ($0.890$) and CNN ($0.888$). These results are higher by about $0.03$ to $0.05$ compared to utilizing the raw predictions, without smoothing. In the S score metric, DeepConvLSTM-A performs best ($0.819$), followed by DeepConvLSTM-A ($0.814$) and CNN ($0.808$). For the S score, the advantage of the label smoothing is bigger in general, between $0.05$ to $0.06$ for all model classes except the LSTM, which only improves by $0.015$. RFC and SVM to not improve with the label smoothing, their scores decrease by about $0.04$ for both of the metrics.
With label smoothing, we can reach an increased performance with all of the model classes, including the traditional machine learning methods RFC and SVM. The results with a 20 prediction wide average filter smoothing can be seen in table \ref{tbl:washing_rm} and @fig:p1_metrics_rm. The top performing neural network architectures do not change with the smoothing. However, the performance measures increase. DeepConvLSTM has the best F1 score ($0.892$), followed by LSTM-A ($0.891$), DeepConvLSTM-A ($0.890$) and CNN ($0.888$). These results are higher by about $0.03$ to $0.05$ compared to utilizing the raw predictions, without smoothing. In the S score metric, DeepConvLSTM-A performs best ($0.819$), followed by DeepConvLSTM-A ($0.814$) and CNN ($0.808$). For the S score, the advantage of the label smoothing is bigger in general, between $0.05$ to $0.06$ for all model classes except the LSTM, which only improves by $0.015$. RFC and SVM do not improve with the label smoothing, their scores decrease by about $0.04$ for both of the metrics.
The models running on normalized data also profit from the label smoothing, however they still cannot reach the performance of the non normalized models.
For the special case of the models initially trained on problem 3 which were then binarized and run on problem 1, we only report some results in this section. The full results can be found in the appendix, in table \ref{tbl:washing_binarized} and fig. \ref{fig:washing_binarized}. Surprisingly, the models trained on problem 3 reach similar F1 scores on the test data of problem 1 as the models trained on problem 1. DeepConvLSTM achieves an F1 score of $0.857$, DeepConvLSTM-A achieves $0.847$. The F1 score of DeepConvLSTM is even higher than the highest F1 score of the models trained for problem 1 by $0.004$. However, for the S score metric, the models trained for problem 3 can only reach up to $0.704$ (CNN) or $0.671$ (DeepConvLSTM-A), which is lower by $0.052$ than the best performing model trained for problem 1.
For the special case of the models initially trained on problem 3 which were then binarized and run on problem 1, we only report some results in this section. The full results can be found in the appendix, in table A\ref{tbl:washing_binarized} and fig. A\ref{fig:washing_binarized}. Surprisingly, the models trained on problem 3 reach similar F1 scores on the test data of problem 1 as the models trained on problem 1. DeepConvLSTM achieves an F1 score of $0.857$, DeepConvLSTM-A achieves $0.847$. The F1 score of DeepConvLSTM is even higher than the highest F1 score of the models trained for problem 1 by $0.004$. However, for the S score metric, the models trained for problem 3 can only reach up to $0.704$ (CNN) or $0.671$ (DeepConvLSTM-A), which is lower by $0.052$ than the best performing model trained for problem 1.
\FloatBarrier
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment