\chapter{Selected visualization techniques} \label{kap3}
After conducting a research of related work that I have introduced in previous chapter, I presented the researched techniques to the anthropologists and together we selected several methods that could serve as a basis for my work. These methods were selected to provide a solution to the three main drawbacks of the currently employed visualizations in FIDENTIS Analyst application -- the lack of shape information, the lack of local information and the limited view of data.

\section{Surface superimposition}
The first set of the techniques which were selected addresses the issue of preserving the shape information when comparing two models. The two main demands for this visualization were that it should \textit{preserve the shape of both models} and \textit{indicate the differences between models}. The easiest solution to shape preservation would be to use juxtaposition -- to display the models next to each other. However, it is difficult to judge the correlation between models just from seeing them rendered side by side. On the other, hand placing them on top of each other causes occlusions and increases the visual complexity. Despite this, I have decided to use superimposition principle with several visual enhancements based on the work of Busking et al. \cite{busking2011image} -- see Figure \ref{transp} for example of proposed visualizations. 

\begin{figure}[h]
\centering
\scalebox{0.065}{\includegraphics{pictures/transparency.png}}
\caption{Overview of visualization techniques. (a) Models rendered with 50\% opacity. (b) Opaque inner surface, transparent outer surface with shadow casting glyphs and intersection contours. (c) Simulation of fog between surfaces. (d) Combination of (b) and (c).} \label{transp}
\end{figure}

\subsection{Transparency}\label{transp_m}
The problem of occlusions can be solved by using transparency. Unfortunately, simply decreasing the opacity of superimposed models tends to make the final image incomprehensible, especially when the models are intersecting and their surfaces are very close to each other. One of the popular techniques for improving the understandability of transparent surfaces is modulation of transparency values based on the placement of the surface in respect to other surfaces. A variation of this technique suitable for cases like Pair comparison -- when we are dealing with two nested models -- is splitting the surfaces into inner parts -- parts of one model enclosed within the other model -- and outer parts and making the inner parts completely opaque.

Since I am working with facial models which are usually not closed, it is impossible to define something as lying inside or outside of such model. The terms \textit{inner} and \textit{outer} may thus be a bit confusing. To clarify the terminology, I will call the surface that is closest to the camera the \textit{outer surface} and the surface further from camera the \textit{inner surface}. This classification of surfaces takes place in image space, which allows easy handling of special cases such as the one highlighted in Figure \ref{transp_schema} -- the solution for such cases will be discussed in chapter \ref{four}.

The technique of turning the inner surfaces opaque reduces the complexity of final image, as this way the observer, for the most part, sees only two layers of surfaces nearest to the camera -- only one of which is transparent. Therefore the user should have no problem with determining the adherence of surfaces to the models. Well-chosen contrasting colors of the models and interactive adjustment of opacity values should also help with image interpretation. 
\begin{figure}[h]
\centering
\scalebox{0.45}{\includegraphics{pictures/in_out.pdf}}
\caption{Surface transparency and fog simulation scheme. The dotted parts of models (a) and (b) are considered \textit{outer} and are rendered transparently, while the \textit{inner} parts (solid line) are rendered opaque. The intensity of fog (pink color) depends on the distance between surfaces along the viewing direction. The area highlighted in red shows special case when the second surface along viewing ray belongs to the same model as the first one, and thus is classified as the \textit{outer} as well.} \label{transp_schema}
\end{figure}

\subsection{Fog simulation}\label{fog_s}
The modulation of transparency helps the user with classification of order of the surfaces. Unfortunately, it is not very helpful for judging the distance between these surfaces. As a visual clue for this task I chose two techniques. First of them is fog simulation. The aim of this technique is to simulate a partially transparent volume -- fog -- of color different to the colors of the models and filling the space between the two surfaces. In reality, the presence of such a volume would clue the observer in on the distance thanks to the accumulation of opacity. Supposing the outer surface is nearly completely transparent, in places where surfaces are close to each other the inner surface would be still visible quite well. However, with growing distance between surfaces the opacity of the fog would accumulate, so in places with large enough surface distances, the inner surface would be completely occluded by the fog -- see Figure \ref{transp_schema}. It should be noted here, that the term distance in this case denotes the distance along viewing ray -- the result is therefore view dependent.

The downside of the real case scenario is that the outer layer needs to be nearly completely transparent, otherwise the mixture of three colors (two models and fog) would be confusing, or, if the outer surface was close to opaque, the fog would be hardly visible, which would defeat the entire purpose of this technique. To deal with this issue, I have devised three different fog simulation methods based on the real case scenario (and thus also view dependent). Figure \ref{fog} shows the comparison of these methods.
\begin{figure}[t!]
\centering
\scalebox{0.085}{\includegraphics{pictures/fog.png}}
\caption{Fog simulation techniques. (a) Models rendered with full opacity. (b) Color overlay -- notice the illusion that the blue surface lies behind the red one. (c) Transparency mapping on outer surface. (d) Color mapping on inner surface.} \label{fog}
\end{figure} 

\begin{itemize}
\item \textbf{Color overlay.} The first method modifies the color of outer surface based on the distance between surfaces -- the distance serves as ratio between original color of the surface and the color of the fog. This yields similar result as if the final image was overlaid with new layer in color of fog with opacity based on the surface distances. It solves the issue of the transparency necessity for outer surface, on the other hand it creates misleading illusions about surface adherence to models.
\item \textbf{Transparency mapping on outer surface.} With this method the entire outer surface is colored to the color of the fog. The distance is then mapped on the opacity values of outer surface -- the bigger the distance, the higher the opacity. This method yields nice visual results, but the downside of the method is that the interpretability of surface adherence to models is reduced by coloring the entire outer layer to one color.
\item \textbf{Color mapping on inner surface.} The last method modifies the color of the surface much the same as the first method -- mixing the color of model and the fog based on distance -- only this time the color is mapped on the inner layers. It simulates the real case scenario, which may be the most natural for users, but it also has the same problem as the real case scenario -- the outer layer needs to be transparent.
\end{itemize}

\subsection{Shadow-casting curvature glyphs}
It has been found in several studies \cite{erens1993perception, mamassian1998perception} that shadows aid the human perception of depth and shape. Therefore, as the second method dedicated to improving the interpretability of distances between surfaces I have chosen shadow-casting glyphs, a method base on works \cite{interrante1997conveying, weigle2005visualizing}. These glyphs are mapped on the outer parts of the surfaces and cast shadows on the inner surfaces. The light source position is fixed in respect to models, so when the users rotate the scene, they can explore the shadows from various angles. The placement of the glyphs on individual models is chosen is such way that they would not intersect, but would be evenly distributed across the surface.

The color and shape of glyphs were chosen to provide additional information. The color of the glyphs matches the color of the surface so they are only visible when the outer surface is not fully opaque -- this is done intentionally to help with classification of surface adherence to models. As for the shape, directional strokes are often employed, with length, orientation and density used to convey properties of surface such as curvature. However, these are more suitable for single surface shape analysis. The primary aim of my work is to illustrate the distances between surfaces, therefore I have opted for the shape presented in \cite{weigle2005visualizing}, a \textit{plus} sign of constant size elongated in one direction. The elongated part of the glyph indicates the direction of maximal principal curvature at the center of the glyph. The perpendicular direction indicated by the shorter part of the glyph is then direction of minimal principal curvature.

\subsection{Intersection contours}
The last enhancement technique I have chosen is contouring of intersections. The motivation for this is that, with transparency and glyphs added to the surface, sometimes the intersection are not very prominent in the image, or are not visible at all -- see Figure \ref{int} for example. On the other hand bump edges and occlusions may be misinterpreted as intersections in places where there are none. Explicitly marking the contours of surface intersections is therefore highly beneficial, especially for comparative studies of models.

\begin{figure}[h]
\centering
\scalebox{0.14}{\includegraphics{pictures/int.png}}
\caption{Example of case when the small intersection is hidden due to glyph placement. (a)~Case without intersection contours. (b)~Case with intersection contours.} \label{int}
\end{figure}

\section{Cross sections}
I have described a set of methods for visualization of two intersecting facial surfaces. However comparing two models covers but a small part of the tasks anthropologists perform with the FIDENTIS Analyst application. One to many and Batch comparison modes are designed for processing large datasets where displaying all models at once is impossible. Nevertheless, the information about the shape variation, especially local shape variation displayed on local scale, is desirable. But how to deal with the complexity of the data? 

\begin{figure}[h]
\smallskip
\centering
\scalebox{0.26}{\includegraphics{pictures/slices.png}}
\smallskip
\caption{Cross sections. (a) Reference picture of average face with the plane specifying cross-sectional slice. (b) Red -- intersection of plane with the average face. Black -- intersections with all faces in dataset. (c) Distance span in the direction of normal to the primary intersection curve at sampling points. (d) Vectors indicating average distance in the normal direction from average face to all faces in dataset. (e) Same as (d) with enhanced vector sizes.} \label{slices}
\end{figure} 

Projection, or rather reduction of the 3D data into 2D space is a~popular approach, as 2D images can be in some cases easier to interpret than 3D. The cross section method I proposed for this intent is inspired by the technique typically used for visualization of volumetric data. A slicing plane is taken and intersection of 3D data with this plane is computed and then displayed in 2D. In my case I assumed that all the models in dataset are aligned and that one facial model will be selected as primary -- this may be the primary face in One to many comparison mode or average face in Batch processing. This model serves as a base for visualization and computation. It is displayed in 3D space along with slicing plane -- see Figure \ref{slices} (a). The users can move and rotate the slicing plane in the space of primary model to get the desired intersection. The intersections of the plane with every model in dataset are then computed, the intersection with primary model (primary intersection) is sampled and the variability at each sample point is determined.

There are three main options of what user can display, each option representing the variability of shapes in a different way.
\begin{itemize}
\item \textbf{Intersections with all faces.} Figure \ref{slices} (b). When the intersections with all faces from dataset are displayed, it is possible to observe how well the models are aligned, especially with interactive manipulation of slicing plane, when user can move across the model and see the local changes. Unfortunately, with increasing number of models the interpretability of final image with all the intersections displayed decreases.
\item \textbf{Distance span.} Figure \ref{slices} (c). Distance span indicates the interval of distances from given sampling point to the intersection curves of each model from dataset. The distance is computed in the direction of normal to the primary intersection. It is displayed as line segment with endpoints indicating maximal distances in positive and negative direction of normal at given point.
 \item \textbf{Average distance.} Figure \ref{slices} (d,e). Average distance form given sampling point to the intersection curves of each model from dataset, again in the normal direction. This indicates the variability in dataset at given slice. The shorter the vectors are the lower is the variability. Naturally, the user can scale the vectors to better analyze the differences.
\end{itemize}

\section{Plots}
The last visualization method I have decided to use is dedicated to displaying numerical results computed during the comparison of larger datasets -- One to many or Batch comparison. A typical ways of displaying large sets of numerical data are tables and various plots and charts. Somewhere in between them lies a technique called heat plot or heat map. The principle of this method lies in displaying the data in a table-like manner, but instead of numerical values, the data are encoded as colors of the table cells. By filtering and reordering of the data correlations may be discovered more easily as with studying the numerical values themselves, as it is usually more intuitive for users to compare colors than numbers.

I have created two versions of heat maps, one for numerical results of Batch processing and one for auxiliary results of either Batch or One to many comparison.

\subsection{Numerical results}
The numerical results in Batch processing consist of table of $n \times n$ values representing the measurement between each pair of models in the dataset consisting of $n$ faces. These measurements can represent maximal or minimal distance between the two faces, variance, geometric mean, etc. 

To display the values I have created a heat plot of $n \times n$ cells, each cell representing one measurement -- Figure \ref{num}. The exact numerical value can by displayed by hovering over the cell. The users can filter the lowest or/and highest values on interactive scale. There is also an option of reordering the values in the plot according to rows and columns. These options help with easy identification of extrema in the data and observation of relationships of data in dataset.

As an additional feature a histogram of values may be displayed. The color scale corresponding to one used in heat plot is used for drawing the histogram to help with matching the intervals of the scale with data in the plot. The histogram illustrates the distribution and variability of the values in the set.

\begin{figure}[h]
\centering
\scalebox{0.3}{\includegraphics{pictures/num.png}}
\scalebox{0.44}{\includegraphics{pictures/num_hist.png}}
\caption{Heat plot for visualization of numerical results and the accompanying histogram.} \label{num}
\end{figure} 

\subsection{Auxiliary results}
Auxiliary results are detailed results computed for each model in the processed dataset. For each vertex of given model they contain the distances to the closest points on each model in dataset. So the auxiliary results for model $M$ from the dataset consisting of $n$ models would contain \textit{number of vertices of M}$\times n$ values.

Since the models typically contain thousands of vertices it is rather impractical to display separate cell for each value. Therefore I adjusted the heat plot to display each value as a vertical line segment of 1px width -- Figure \ref{aux}. One row of the heat plot then depicts distances from vertices of model of which the auxiliary results are displayed (primary model) to the nearest vertices of one model in dataset. A vertical slice at position $x$ then represents the distances form $xth$ vertex of the primary model to the nearest vertex of each model from dataset. In simpler terms, one row corresponds to one model, and one column correspond to one vertex of primary model. 

\begin{figure}[h]
\centering
\scalebox{0.33}{\includegraphics{pictures/auxl.png}}
\caption{Heat plot for visualization of auxiliary results of average face computed from dataset of 25 models. Each row depicts distances of vertices of average face from one model. The pink highlight shows zooming window and the detailed view of individual values.} \label{aux}
\end{figure} 

\newpage
When scaling the cell width to 1px is not enough to fit the plot into dedicated space, the neighboring entries in vertex dimension are averaged and displayed as one value. Since this is the typical case, the zooming window that can be moved along the selected row has been added. The individual values of the area covered by the window are than displayed under the heat plot. Again, the values can be filtered by interactive scale and the histogram of values can be displayed here as well, either for entire set of results or just for the selected row. 

The intent of this visualization is to provide means for analysis of variation in the dataset and for comparison of models in dataset in closer detail. As entries in each row at given horizontal position correspond to same vertex or vertices of primary face, it is possible to observe how models in the dataset differ from primary model at given point.


