Commit 26624d2c authored by Alexander Henkel's avatar Alexander Henkel
Browse files

more pseudo labels

parent 4fec348f
...@@ -42,7 +42,7 @@ In the next step I want to observe if smoothing could have a negative effect if ...@@ -42,7 +42,7 @@ In the next step I want to observe if smoothing could have a negative effect if
\section{Evaluation of different Pseudo label generations} \section{Evaluation of different Pseudo label generations}
In this section, I describe the evaluation of different pseudo labeling approaches using the filters introduced in \secref{sec:approachFilterConfigurations}. For each filter configuration, the base model is used to predict the labels of the training sets and create pseudo labels. After that the filter is applied to the pseudo labels. To determine the quality of the pseudo labels, they are evaluated against the ground truth values using soft versions of the metrics $Sensitivity^{soft}$, $Specificity^{soft}$, $F_1^{soft}$, $S^{soft}$. The general model is then trained by the refined pseudo labels. All resulted models are evaluated by their test sets and the mean over all is computed. \figref{fig:pseudoModelsEvaluation} shows a bar plot over the metrics for all filter configuration. I concentrate to the values of S score. The configurations \texttt{all}, \texttt{high\_conf}, \texttt{scope}, \texttt{all\_corrected\_null}, \texttt{scope\_corrected\_null}, \texttt{all\_corrected\_null\_hwgt}, \texttt{scope\_corrected\_null\_hwgt} lead to a lower performance than the base model. Insights gives \figref{fig:pseudoModelsTrainingData} (b). All of the generated training data contain false positive labels, i.e. \textit{null} samples which are labeled as hand washing, or there are just a few true negative labels. For all \texttt{all\_null\_*} configurations nearly all not hand washing samples are correctly labeled as \textit{null}. More over, the training data consists of slightly more true positive labels than the others. The resulted models reach almost the same performance as the supervised trained model. In this section, I describe the evaluation of different pseudo labeling approaches using the filters introduced in \secref{sec:approachFilterConfigurations}. For each filter configuration, the base model is used to predict the labels of the training sets and create pseudo labels. After that the filter is applied to the pseudo labels. To determine the quality of the pseudo labels, they are evaluated against the ground truth values using soft versions of the metrics $Sensitivity^{soft}$, $Specificity^{soft}$, $F_1^{soft}$, $S^{soft}$. The general model is then trained by the refined pseudo labels. All resulted models are evaluated by their test sets and the mean over all is computed. \figref{fig:pseudoModelsEvaluation} shows a bar plot over the metrics for all filter configuration. I concentrate to the values of S score. The configurations \texttt{all}, \texttt{high\_conf}, \texttt{scope}, \texttt{all\_corrected\_null}, \texttt{scope\_corrected\_null}, \texttt{all\_corrected\_null\_hwgt}, \texttt{scope\_corrected\_null\_hwgt} lead to a lower performance than the base model. Insights gives \figref{fig:pseudoModelsTrainingData} (b). All of the generated training data contain false positive labels, i.e. \textit{null} samples which are labeled as hand washing, or there are just a few true negative labels. For all \texttt{all\_null\_*} configurations nearly all not hand washing samples are correctly labeled as \textit{null}. More over, the training data consists of slightly more true positive labels than the others. The resulted models reach almost the same performance as the supervised trained model. In the case of \textitt{all\_cnn\_*} configurations, the training data achieve similar evaluation values as for \texttt{all\_null\_*} configurations but the scaled values are slightly lower. This also leads to a bit lower performance. Additionally the dataset of \textitt{all\_cnn\_convlstm3} also contains
\input{figures/experiments/supervised_pseudo_models} \input{figures/experiments/supervised_pseudo_models}
\input{figures/experiments/supervised_pseudo_models_training_data} \input{figures/experiments/supervised_pseudo_models_training_data}
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment