\chapter{Results and evaluation}\label{five}
I chose and implemented three different approaches of visualization for 3D facial data. For scenarios where two models are processed and compared, I chose superposition principle with several adjustments and cues to improve the shape and distance perceptibility -- intersection contours highlight, surface opacity modulation, shadow-casting glyphs indicating surface curvature, and simulation of fog based on the distance between surfaces.

Second method I designed is based on cross-sectional slices. Its purpose is to visualize local variability and shape. This technique transfers the local data from 3D to 2D view, which reduces visual complexity and allows the observer to focus on important details. 

The third and final tool I created employs heat maps for visualization of numerical data exported from the FIDENTIS Analyst application. The purpose of this tool is to reduce the need for further data post-process\-ing in other applications and provide yet another outlook on data.

Examples of this techniques may be found in attachment \ref{quest} as a~part of evaluation questionnaire. 

I created a standalone application in which all the described visualization techniques are implemented, compatible with data used in FIDENTIS Analyst application and ready to be integrated there as well.

To evaluate my visualization techniques I conducted a user study among anthropologists working on FIDENTIS project. The techniques were evaluated by four scientists working in a field of facial morphometry and analysis. The users were first introduced to the visualization principles and the test application, in which they could try the visualizations. They were then asked to fill out a questionnaire in which they evaluated the selected techniques. The questionnaire can be found in full extent in attachment \ref{quest}.

\section{Surface superimposition}
The questionnaire consisted of four parts, the first one concerning evaluation of visualizations for pair comparison based on surface superimposition method. In this part eight selected combinations of techniques were presented to scientists (see Figure \ref{ev_8}), including the visualization technique currently employed in FIDENTIS Analyst application.

\begin{figure}[h]
\centering
\scalebox{0.35}{\includegraphics{pictures/ev_8.png}}
\caption{Visualizations presented for evaluation.} \label{ev_8}
\end{figure} 

For each of these techniques the users were asked to rank the following questions:
\begin{enumerate}
\item{How well does this visualization convey shapes of individual models?}
\item{How well does this visualization convey differences between models?}
 \end{enumerate}
The scale from \textit{very well} through \textit{well, neutral, poorly} to \textit{very poorly} was provided for evaluation. The averaged results of their evaluation can be seen in Figure \ref{res1}. The detailed answers are included in attachment \ref{quest_res}.

\begin{figure}[h]
\centering
\scalebox{0.7}{\includegraphics{pictures/ev1p.pdf}}
\caption{Averaged evaluation of surface superimposition visualization techniques.\newline} \label{res1}
\end{figure} 

It can be seen from these results, that while the visualization currently employed in the application -- visualization (a) -- ranks high on the second question (imparting differences), it ranks low on conveying shapes in comparison with other methods. In fact, out of the presented methods, this one ranked highest for the second question, but lowest for the first one.

Concerning the newly proposed methods, evaluated as the best suited for conveying the shapes of models were: visualization with shadow-casting glyphs (b), color overlay (c), color mapping on inner surface (e) and combination of shadow-casting glyphs and color mapping on inner surface (h). It is notable that for conveying the shapes all new methods were ranked higher than the one currently used.

For conveying the differences between models, the best ranked of the new methods was visualization with shadow-casting glyphs (b), closely followed by transparency mapping on outer surface (d) and combination of shadow-casting glyphs and color mapping on inner surface (h).

The visualizations (b) -- shadow-casting glyphs and (h) -- combination of shadow-casting glyphs and color mapping on inner surface rank high in both questions, balancing both -- need for illustration of the shape and imparting the differences between models. 

During the evaluation some remarks were made concerning the visualizations. The scientists appreciated that when using glyphs, the shape of outer surfaces is indicated even when the surface is almost transparent. However, it was suggested that the light position should not be fixed in respect to orientation of the models in order to achieve moving shadows when the models are rotated. This idea was prompted by occurrence of shadow distortion on surfaces that were almost parallel to light direction.

Regarding the fog simulation, users found it less straightforward and understandable than the glyphs. The main concern was the view dependent intensity of fog. Due to the fact that the intensity is based on distance between surfaces in viewing direction, with model rotation the intensity may change at certain locations. It was suggested that the technique should be made view independent.

It was further noted that the intersection contours are very beneficial for intersection detection and suggested that option of highlighting them should be added also to the currently employed color map visualization.

The scientists further appreciated the interactivity of the application, the option of combining and adjusting the techniques. For example, using these interactive adjustments they designed a visualization of only the volume between two surfaces by setting both surfaces transparent and adding Color overlay fog simulation -- Figure \ref{vol}. I did not considered such an option prior to the evaluation, but they found the visualization contributory to their work.

\begin{figure}[h]
\centering
\scalebox{0.075}{\includegraphics{pictures/vol.png}}
\caption{Visualization of volume between surfaces.} \label{vol}
\end{figure} 
\section{Cross sections}
The second part of questionnaire was dedicated to evaluation of local variability visualization based on cross sections. The scientists were asked to evaluate this technique by three questions:

\begin{enumerate}
\item{How well does this visualization convey variability in a set of models at specific (sampling) points? (again on a scale from \textit{very well} to \textit{very poorly})}
\item{Does this visualization convey local variability better or worse than color map on surface? (\textit{better, I don't know, worse})}
\item{Is this visualization contributory to variability perception and analysis? (\textit{yes, I don't know, no})}
 \end{enumerate}

In the first question all respondents answered \textit{very well} or \textit{well}, proclaiming the visualization fairly demonstrative for local variability. 

In the second question, the scientists deemed this visualization better that the one used in FIDENTIS Analyst application in terms of conveying local variability, appreciating possibility to focus on one place and scale the visual results.

All respondents marked the technique as contributory to variability perception and analysis in third question.

The scientists liked the option of showing cross sections of entire dataset versus vectors indicating variability. There was only one remark -- it was suggested to add the option of displaying absolute variability values (as opposed to currently used relative, which take into account orientation of vectors).

\section{Plots}
In third part of the questionnaire I asked for the evaluation of heat plots I created for numerical and auxiliary results. For both parts I prepared two questions:
\begin{enumerate}
\item{How well does this visualization convey variability in a set of models? (on a scale from \textit{very well} to \textit{very poorly} )}
\item{Is this visualization contributory to analysis of facial dataset? (\textit{yes, I don't know, no})}
 \end{enumerate}

For visualization of numerical results, the respondents claimed the visualization convey variability in a set of models mostly \textit{very well}. All respondents marked this visualization as contributory to the analysis, especially appreciating the color-coded histogram.

As for visualization of auxiliary results, the scientists found it less demonstrative of variability, marking it \textit{well} or \textit{neutral} in first question and only two respondents found it contributory to the analysis of dataset. It was remarked that when performing facial analysis the scientists are most interested in facial location with greatest variability. Therefore it was suggested to improve this visualization by displaying the location of selected vertices on facial model.

\section{Summary}
In the last part of the questionnaire I asked the respondents to choose the best suited visualization method for specific tasks to see if and where my methods were usable. Based on the discussions with scientists prior and during the development of my work, I have listed the tasks they typically perform with FIDENTIS Analyst application:

\begin{itemize}
\item{Verifying the alignment of models.}
\item{Analyzing the shape of models.}
\item{Comparing two models.}
\item{Analyzing local variability of models.}
\item{Analyzing a set of models (comparison, variability, etc.).}
 \end{itemize}

For each of these tasks the scientist were asked to choose up to three techniques they found suitable for it.

For \textit{verifying the alignment of models} the two selected visualizations were surface superimposition method with shadow-casting glyphs and cross-sectional slices with option of showing slices of all models in dataset. It was also suggested that for verification of alignment of two models the superimposition method with fully opaque models and only intersection contours (no glyphs) would be suitable, since glyphs increase image complexity, which was considered unnecessary for this task.

In order to \textit{analyze the shape of models} the scientist selected as most suitable again the cross-sectional slices and shadow-casting glyphs visualization.

For \textit{comparing two models}, the color map on model was found as most suitable by two respondents. Their other choices consisted of combinations of fog simulations and glyphs, but not showing any significant preference for one particular method.

For the task of \textit{analyzing local variability} the preferred methods were color map on model, cross-sectional slices and heat plot for numerical results.

Finally, in case of \textit{analyzing a set of models} all scientists uniquely settled on heat plot for numerical results and some of them also found cross-sectional slices beneficial for the task.

In conclusion, the most useful visualizations were found to be color map on model (currently employed in application), shadow-casting glyphs without fog, cross-sectional slices and heat plot for numerical results. The fog simulations were often found confusing, with scientists stating that although visually nice-looking, the interpretation "takes some getting used to". As for visualization of auxiliary results, the lack of information about location of displayed values made it less preferable to other techniques.
