23/09/2019
1
PA198
Augmented Reality Interfaces
Lecture 4
Designing Augmented Reality Interfaces
Fotis Liarokapis
liarokap@fi.muni.cz
24th September 2019
User Interface Design
What is User Interface Design (UID)?
• The process of designing effective and user
friendly interfaces for software systems and
applications
– Applies in all computer science
– Other disciplines as well
• i.e. Hardware design
10 Commandments of UID
https://www.designmantic.com/blog/infographics/the-10-commandments-of-ui-design/
The User Interface
• Users usually judge a software application based
on its interface rather than its functionality
– A poorly designed interface can cause a user to
make catastrophic errors
– Poor user interface design is the main reason why so
many software systems are never used
Many Types of Interfaces
• Command
• Speech
• Data-entry
• Form fill-in
• Query
• Graphical
• Web
• Pen
• Augmented reality
• Gesture
• Brain-computer interfaces
23/09/2019
2
Graphical User Interfaces (GUIs)
• GUIs allows users to interact with electronic
devices and software apps through graphical
icons and visual indicators such as secondary
notation
– Opposed to text-based interfaces, typed command
labels or text navigation
GUI Characteristics
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
GUI Advantages
• They are easy to learn and use
– Without experience can use the system quickly
– Can switch quickly from one task to another
– Can interact with several different applications
– Information remains visible in its own window when
attention is switched
• Fast, full-screen interaction is possible with
immediate access to anywhere on the screen
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User-Centred Design
• In user-centred design the needs of the user are
paramount and where the user is involved in the
design process
• User interface design always involves the
development of prototype interfaces
User Interface Design Process
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User Interface Design Process .
http://ui-designer.net/interface_design.htm
23/09/2019
3
User Interface Design Principles
• User interface design must consider the needs,
experience and capabilities of the system users
– Principles underlie interface designs although not all
are applicable to all designs
• Designers should:
– Be aware of people’s physical and mental limitations
(e.g. limited short-term memory)
– Recognise that people make mistakes
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User Interface Design Principles .
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Design Principles
• User familiarity
– The interface should be based on user-oriented terms and
concepts rather than computer concepts
• i.e. An office system should use concepts such as letters, documents,
folders etc. rather than directories, file identifiers, etc.
• Consistency
– The system should display an appropriate level of consistency
– Commands and menus should have the same format,
command punctuation should be similar, etc.
• Minimal surprise
– If a command operates in a known way, the user should be
able to predict the operation of comparable commands
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Design Principles .
• Recoverability
– The system should provide some resilience to user errors
and allow the user to recover from errors
• This might include an undo facility, confirmation of destructive
actions, 'soft' deletes, etc
• User guidance
– Some user guidance such as help systems, on-line
manuals, etc. should be supplied
• User diversity
– Interaction facilities for different types of user should be
supported
• i.e. Accessibility
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User-System Interaction
• Two problems must be addressed in interactive
systems design
– How should information from the user be provided to
the computer system?
– How should information from the computer system
be presented to the user?
• User interaction and information presentation
may be integrated through a coherent
framework such as a user interface metaphor
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Interaction Styles
23/09/2019
4
Common Interaction Styles
• Direct manipulation
• Menu selection
• Form fill-in
• Command language
• Natural language
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Pros and Cons
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Interaction
style
Main advantages Main disadvantages Application
examples
Direct
manipulation
Fast and intuitive
interaction
Easy to learn
May be hard to implement.
Only suitable where there is a
visual metaphor for tasks and
objects.
Video games
CAD systems
Menu
selection
Avoids user error
Little typing required
Slowfor experienced users.
Can become complex if many
menu options.
Most generalpurpose
systems
Form fill-in Simple data entry
Easy to learn
Checkable
Takes up a lot of screen space.
Causes problems where user
options do not match the form
fields.
Stock control,
Personal loan
processing
Command
language
Powerful and flexible Hard to learn.
Poor error management.
Operating systems,
Command and
control systems
Natural
language
Accessible to casual
users
Easily extended
Requires more typing.
Natural language understanding
systems are unreliable.
Information
retrieval systems
Direct Manipulation Advantages
• Users feel in control of the computer and are
less likely to be intimidated by it
• User learning time is relatively short
• Users get immediate feedback on their
actions so mistakes can be quickly detected
and corrected
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Direct Manipulation Disadvantages
• The derivation of an appropriate information
space model can be very difficult
• Given that users have a large information
space, what facilities for navigating around
that space should be provided?
• Direct manipulation interfaces can be complex
to program and make heavy demands on the
computer system
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Menu Systems
• Users make a selection from a list of
possibilities presented to them by the system
• The selection may be made by pointing and
clicking with a mouse, using cursor keys or by
typing the name of the selection
• May make use of simple-to-use terminals such
as touch screens
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Menu Systems Advantages
• Users need not remember command names as
they are always presented with a list of valid
commands
• Typing effort is minimal
• User errors are trapped by the interface
• Context-dependent help can be provided
– The user’s context is indicated by the current menu
selection
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
23/09/2019
5
Menu Systems Disadvantages
• Actions which involve logical conjunction (and)
or disjunction (or) are awkward to represent
• Menu systems are best suited to presenting a
small number of choices
– If there are many choices, some menu structuring
facility must be used
• Experienced users find menus slower than
command language
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Command Interfaces
• User types commands to give instructions to the
system
– i.e. UNIX
• May be implemented using cheap terminals
• Easy to process using compiler techniques
• Commands of arbitrary complexity can be
created by command combination
• Concise interfaces requiring minimal typing can
be created
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Command Interfaces Disadvantages
• Users have to learn and remember a command
language
– Unsuitable for occasional users
• Users make errors in command
– An error detection and recovery system is required
• System interaction is through a keyboard so
typing ability is required
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Command Languages
• Often preferred by experienced users because
they allow for faster interaction with the system
• Not suitable for casual or inexperienced users
• May be provided as an alternative to menu
commands
– i.e. Keyboard shortcuts
• In some cases, a command language interface
and a menu-based interface are supported at
the same time
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Natural Language Interfaces
• The user types a command in a natural language
• Generally, the vocabulary is limited and these
systems are confined to specific application
domains
– i.e. Timetable enquiries
• NL processing technology is now good enough to
make these interfaces effective for casual users
– But experienced users find that they require too
much typing
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Biofeedback
• Biofeedback is the process of gaining greater
awareness of many physiological functions
primarily using instruments that provide
information on the activity of those same
systems, with a goal of being able to manipulate
them at will
• Common processes that can be controlled:
brainwaves, muscle tone, skin conductance,
heart rate and pain perception
https://en.wikipedia.org/wiki/Biofeedback
23/09/2019
6
How Biofeedback Works
http://www.stress-relief-tools.com/how-biofeedback-works.html
emWave Personal Stress Reliever
• Developed by HeartMath
System
• Measures heart-rate
coherence, a particular
heart-rate pattern that is
beneficial for reducing
stress and promoting
health
http://store.heartmath.com/
emWave2 Video
https://www.youtube.com/watch?v=jYDvwbK7hzA
Noumic Biofeedback Device
• Focused for stress
management, and
meditation skills
• Measures:
– Skin Conductivity
Information
– Hand-muscle Tension
Information
http://www.noumic.com/biofeedback.html
GSR 2 Relaxation Monitor
• Home biofeedback device
• Monitors stress levels by translating tiny
tension-related changes in skin pores into a
rising or falling tone
• Not available in Europe
http://thoughttechnology.com/index.php/gsr-2.html
Iom (Wild Divine)
• The Iom features three finger rings that are worn to gather
the following bio-signs:
– Heart rate BPM
– Pulse strength and waveform
– Heart rate variability
– Galvanic skin response
• Included with the package are 15 games and 30 training
exercises, along with a slick graphing tool to directly
monitor and record your bio-signs during training sessions
– The data from this tool provides you with raw evidence of your
progress
https://wilddivine.com/
23/09/2019
7
Iom From Wild Divine (Video)
https://www.youtube.com/watch?v=5nZuWu4np6g
Brain Computer Interfaces
• Direct way of communication
• Reasonable result for patients
– Loads of months of training
• Experimental for healthy samples
– Still software and hardware technology is not
there
Stages of a BCI System
• Signal acquisition
– Signal is captured by a neuro-imaging device (i.e. EEG)
– A BCI system may be acquiring several kinds of signals at the
same time (but must be synchronised and time-locked to the
interaction with the device)
• Signal pre-processing or signal enhancement
– Signal is prepared to further processing, including artefact
removal (e.g. muscle movement and noise reduction) are
typicallyperformed at this stage
• Feature extraction
– Discriminativefeatures are identified and mapped onto a
vector; first order parameters (i.e. amplitude of signal or
latency), and second-order parameters (i.e. time-frequency
parameters extracted from a Fourier transform)
Desney S Tan and Anton Nijholt. Brain-Computer Interfaces: applying our minds to human-computer interaction. Springer, 2010.
Stages of a BCI System .
• Classification
– Involves the classification of the parameters previously
extracted, with the aim of ascribing meaning to them;
various techniques from machine learning can be applied,
but this imposes an overhead in time and processing
power that is not suitable to all BCI applications, which
demands real-time interaction
• Control interface
– Results of classification are translated into commands and
send to a connected machine such as a wheelchair or a
computer, which provide the user with feedback and
close the interactive loop between the user and the
device
Desney S Tan and Anton Nijholt. Brain-Computer Interfaces: applying our minds to human-computer interaction. Springer, 2010.
Multiple User Interfaces
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Information Presentation
• Information presentation is concerned with
presenting system information to system users
• The information may be presented directly (e.g.
text in a word processor) or may be transformed
in some way for presentation (e.g. in some
graphical form)
• The Model-View-Controller approach is a way of
supporting multiple presentations of data
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
23/09/2019
8
Information Presentation .
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Types of Information
• Static information
– Initialised at the beginning of a session
• Does not change during the session
– May be either numeric or textual
• Dynamic information
– Changes during a session and the changes must
be communicated to the system user
– May be either numeric or textual
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Information Display Factors
• Is the user interested in precise information or
data relationships?
• How quickly do information values change?
• Must the change be indicated immediately?
• Must the user take some action in response to a
change?
• Is there a direct manipulation interface?
• Is the information textual or numeric? Are
relative values important?
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Models of Human
Computer Behaviour
Mental Models in HCI
• Introduced by Donald
Norman in his book
"The Design of Everyday
Things"
– He used mental models
to describe how a
system is designed and
implemented on the
basis of the designer's
mental model
https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction/mental-models
Adapted from Norman (1988) p. 16
Models of Human-Computer
Behaviour
• Two categories
– Low level
– High Level
23/09/2019
9
Video
• Computationally Modeling Human Emotion
• https://cacm.acm.org/magazines/2014/12/18
0787-computationally-modeling-human-
emotion/abstract
Low Level Models
• Some low-level theories can be used to
predict human performance
– Fitt’s law
• Time to select an item with a pointing device
– Remember from Lecture 2
– Keystroke level model
• Sums up times for keystroking, pointing, homing,
drawing, thinking and waiting
High Level Models
• Developing Theories in HCI
– Must explain and predict human behavior in the
human-computer system
– Must work in a wide variety of task situations
– Must work within broad spectrum of system
designs and implementations
High Level Models .
• General models that explain human behavior
with machines
– Syntactic/semantic model (Shneiderman)
– Stages of interaction (Norman)
– All of psychology!
Design Rules - Schneiderman
1. Strive for consistency
2. Enable frequent users to use shortcuts
3. Offer informative feedback
4. Design dialogs to yield closure
5. Offer error prevention and simple error handling
6. Permit easy reversal of actions
7. Support internal locus of control
8. Reduce short-term memory load
Dix A., Finlay J.E., Abowd G.D., Beale R. Human-Computer Interaction, 3rd Edition, Pearson Education Ltd, 2004.
Design Rules - Norman
1. Use both knowledge in the world and
knowledge in the head
2. Simplify the structure of tasks
3. Make things visible: bridge the gulfs of Execution
and Evaluation
4. Get the mappings right
5. Exploit the power of constraints, both natural
and artificial
6. Design for error
7. When all else fails, standardize
Dix A., Finlay J.E., Abowd G.D., Beale R. Human-Computer Interaction, 3rd Edition, Pearson Education Ltd, 2004.
23/09/2019
10
Video
• The Seven Stages of Action
• https://www.youtube.com/watch?v=n4fCHYb
RcKw AR Interfaces
Introduction to AR Interfaces
• Browsing Interfaces
– Simple (conceptually!), unobtrusive
• 3D AR Interfaces
– Expressive, creative, require attention
• Tangible Interfaces
– Embedded into conventional environments
• Tangible AR
– Combines TUI input + AR display
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Browsing Interfaces
• 2D/3D virtual objects are
registered in 3D
– “VR in Real World”
• Interaction
– 2D/3D virtual viewpoint
control
• Applications
– Visualization, training
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
3D AR Interfaces
• Virtual objects displayed in 3D
physical space and
manipulated
– HMDs and 6DOF head-tracking
– 6DOF hand trackers for input
• Interaction
– Viewpoint control
– Traditional 3D user interface
interaction
• Manipulation, selection, etc
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Kiyokawa, et al. 2000
Tangible Interfaces
• Dangling String
– Jeremijenko 1995
– Ambient Ethernet monitor
– Relies on peripheral cues
• Ambient Fixtures
– Dahley, Wisneski, Ishii 1998
– Use natural material qualities for
information display
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
23/09/2019
11
Augmented Surfaces and Tangible
Interfaces
• Basic principles
– Virtual objects are
projected on a surface
– Physical objects are used
as controls for virtual
objects
– Support for collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR
• AR overcomes limitation of
TUIs
– Enhance display possibilities
– Merge task/display space
– Provide public and private views
• TUI + AR = Tangible AR
– Apply TUI methods to AR
interface design
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Space Multiplexed vs. Time
Multiplexed
• Space-multiplexed
– Many devices each with one function
• Quicker to use, more intuitive, clutter
• Real Toolbox
• Time-multiplexed
– One device with many functions
• Space efficient
• Mouse
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR: Tiles (Space Multiplexed)
• Tiles semantics
– Data tiles
– Operation tiles
• Operation on tiles
– Proximity
– Spatial arrangements
– Space-multiplexed
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR: Time-Multiplexed
Interaction
• Use of natural physical object
manipulations to control
virtual objects
– Catalog book:
• Turn over the page
– Paddle operation:
• Push, shake, incline, hit, scoop
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Interface Design Path
• Prototype Demonstration
• Adoption of Interaction Techniques from other
interface metaphors
• Development of new interface metaphors
appropriate to the medium
• Development of formal theoretical models for
predicting and modeling user actions
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
23/09/2019
12
Interface Metaphors
• Designed to be similar to a physical entity but also
has own properties
– e.g. desktop metaphor, search engine
• Exploit user’s familiar knowledge, helping them to
understand ‘the unfamiliar’
– Conjures up the essence of the unfamiliar activity,
enabling users to leverage of this to understand more
aspects of the unfamiliar functionality
• People find it easier to learn and talk about what
they are doing at the computer interface in terms
familiar to them
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Example: The Spreadsheet
• Analogous to ledger
sheet
• Interactive and
computational
• Easy to understand
• Greatly extending what
accountants and others
could do
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Why was it so good?
• It was simple, clear, and obvious to the users
how to use the application and what it could do
• “it is just a tool to allow others to work out their
ideas and reduce the tedium of repeating the
same calculations.”
• Capitalized on user’s familiarity with ledger
sheets
• Got the computer to perform a range of
different calculations in response to user input
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Another Classic
• 8010 Star office system targeted at workers not
interested in computing per se
– Spent several person-years at beginning working out
the conceptual model
• Simplified the electronic world, making it seem
more familiar, less alien, and easier to learn
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
The Star Interface
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Benefits of Interface Metaphors
• Makes learning new systems easier
• Helps users understand the underlying
conceptual model
• Can be innovative and enable the realm of
computers and their applications to be made
more accessible to a greater diversity of users
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
23/09/2019
13
Problems with interface metaphors
(Nielson, 1990)
• Break conventional and cultural rules
– i.e. Recycle bin placed on desktop
• Can constrain designers in the way they
conceptualize a problem
• Conflict with design principles
• Forces users to only understand the system in terms
of the metaphor
• Designers can inadvertently use bad existing designs
and transfer the bad parts over
• Limits designers’ imagination with new conceptual
models
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
AR Design Principles
• Interface Components
– Physical components
– Display elements
• Visual/audio
• Interaction metaphors
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR Design Principles
• Tangible AR Interfaces use TUI principles
– Physical controllers for moving virtual content
– Support for spatial 3D interaction techniques
– Time and space multiplexed interaction
– Support for multi-handed interaction
– Match object affordances to task requirements
– Support parallel activity with multiple objects
– Allow collaboration between multiple users
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
The MagicBook
• Design Goals:
– Allows user to move smoothly between reality
and virtual reality
– Support collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Features
• Seamless transition between Reality and
Virtuality
– Reliance on real decreases as virtual increases
• Supports egocentric and exocentric views
– User can pick appropriate view
• Computer becomes invisible
– Consistent interface metaphors
– Virtual content seems real
• Supports collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Collaboration
• Collaboration on multiple
levels:
– Physical Object
– AR Object
– Immersive Virtual Space
• Egocentric + exocentric
collaboration
– Multiple multi-scale users
• Independent Views
– Privacy, role division, scalability
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
23/09/2019
14
MagicBook Technology
• Reality
– No technology
• Augmented Reality
– Camera – tracking
– Switch – fly in
• Virtual Reality
– Compass – tracking
– Press pad – move
– Switch – fly out
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Video
https://www.youtube.com/watch?v=tNMljw0F-aw
Generic AR Interface
Layers of the AR Interface
AR Applications
Interface framework
Interaction algorithms
Audio algorithms based on OpenAL API
Visual algorithms based on OpenGL API
Vision SDK Video Libraries
Windows OS
Libraries and Drivers
(Graphics and Sound Card, Video Camera, etc)
ARToolKit tracking Libraries
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Interactions
Standard I/O
Computer
Physical Manipulation Touch Screen
Interface Menu Interaction
SpaceMouseUser
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Interface Visualisation Capabilities
• Visualisation
– 3D models
– 3D text
– 3D sound
– Videos
– Pictures
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
23/09/2019
15
GUI Environment
• The GUI consists of a
– Menu
– Toolbar
– Status bar
– Dialog boxes
• Allows participants to have the same access to
the augmented virtual information as they
have had using standard interaction
techniques
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Some Algorithms
Multiple Augmentation Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Tracking Multiple Markers
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Textual Augmentation Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
3D Sound Generation
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
23/09/2019
16
SpaceMouse Interaction
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Magic Book Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Designing Markers
Square Marker Cards
• The easiest solution
• Will work out well for
assignment
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Other Shapes Markers
• There are limitations with the number of
square markers that can be designed
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Hand Markers
• Does not work very well with ARToolKit
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
23/09/2019
17
Case Studies
Learning Requirements
• The potential benefits of AR applied to HE
include:
– Visualisation of the theoretical parts in 3D
– Practical exploration of the theory
– Effective collaboration and discussion amongst the
participants
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Learning Requirements .
• An ideal AR system must include at least the
following requirements
– Be simple and robust
– Provide the user with clear and concise information
– Enable the teacher to input information in a simple
and effective manner
– Enable easy interaction between users
– Make complex procedures transparent to the user
– Be cost effective
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Kolb’s Learning Cycle
Enhanced Learning Cycle using AR
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Design of Teaching Material
• Off-line process and consists of:
– A set of distinctive marker cards - this is the link
between the real and the digital information;
– Digital information - is the digital information
including pictures, 3D models, textual
descriptions, video animations and auditory
information;
– Educational tutorials - consist of a number of
predefined learning scenarios which combine
theory and practice at the same time
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
23/09/2019
18
Tutorials Generation
• Theoretical tutorials
– Most important parts of the theory are described through
visual and auditory means of augmentation having
limited user interaction
• Practical tutorials
– Based on the theory, students have to use the specific set
of marker cards to describe a simple but complete
process using collaborative interaction techniques
• Assessment tutorials
– 3D graphical representations of theoretical and practical
issues are assessed in a semi-automatic way based on all
the proposed types of human-computer interactions
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
First Prototype: MARIE
• Initial prototype for AR in education
• Focused on teaching electronics
Liarokapis,F., Petridis, P., Lister, P.F., White, M. MultimediaAugmented Reality Interface for E-Learning(MARIE), World Transactionson Engineeringand
Technology Education,UICEE,1(2): 173-176, 2002. (ISSN: 1446-2257)
Second Prototype: Wed3D & AR
Pilot user experiencing an AR learning scenario,
University of SussexAugmentation of a camshaft
Web3D visualisation of a camshaft
Liarokapis,F., Mourkoussis,N.,White,M.,Darcy,J., Sifniotis,M.,Petridis,P., Basu,A.,Lister, P.F.Web3Dand Augmented Realityto supportEngineeringEducation,World
Transactions on Engineeringand TechnologyEducation,UICEE,3(1):11-14,2004.(ISSN:1446-2257)
Design of Teaching Material
• Create appropriate markers
• Create material
• Link together
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
University of Sussex
• Application in Informatics
• Goal:
– Understand the computer
Liarokapis,F., Anderson,E.UsingAugmentedRealityas a Medium to Assist Teachingin Higher Education,Proc.ofthe 31st Annual Conference ofthe European Associationfor Computer Graphics (Eurographics2010),
Education Program,Norrkoping,Sweden,4-7May,9-16,2010.(ISSN:1017-4656)
City University
• Application in Geography and GIS
• Goal:
– Understand GIS in London
Liarokapis,F., Anderson,E.UsingAugmentedRealityas a Medium to Assist Teachingin Higher Education,Proc.ofthe 31st Annual Conference ofthe European Associationfor Computer Graphics (Eurographics2010),
Education Program,Norrkoping,Sweden,4-7May,9-16,2010.(ISSN:1017-4656)
23/09/2019
19
Video City University Coventry University
• Application in
Computer Graphics
• Goal
– Understand basic
principles in CG
Liarokapis,F.Augmented RealityInterfaces for AssistingComputer Games UniversityStudents,Bulletinofthe TechnicalCommittee on LearningTechnology,IEEEComputingSociety,14(4):7-10,October, 2012.(ISSN:
2241-3766)
Evaluation
• Very useful tool
• Need some time to adapt
• Teacher can combine
different learning approaches
– i.e. Activity Lead Learning
Masaryk University
• Aim:
– Demonstrate the learning effectiveness of
augmented reality and project development
approaches for higher education teaching
Liarokapis,F.UsingActivityLed Learningfor TeachingComputer Graphics Principles ThroughAugmented Reality,Proc.ofthe 38th Annual Conference ofthe EuropeanAssociationfor Computer(Eurographics 2017),
Education Program,Lyon,France, 24-28April,43-50,2017.(DOI:10.2312/eged.20171025)
Approach
Liarokapis,F.UsingActivityLed Learningfor TeachingComputer Graphics Principles ThroughAugmented Reality,Proc.ofthe 38th Annual Conference ofthe EuropeanAssociationfor Computer(Eurographics 2017),
Education Program,Lyon,France, 24-28April,43-50,2017.(DOI:10.2312/eged.20171025)
Student Work Examples
Liarokapis,F.UsingActivityLed Learningfor TeachingComputer Graphics Principles ThroughAugmented Reality,Proc.ofthe 38th Annual Conference ofthe EuropeanAssociationfor Computer(Eurographics 2017),
Education Program,Lyon,France, 24-28April,43-50,2017.(DOI:10.2312/eged.20171025)
23/09/2019
20
AR Ray Casting Video AR Image Effects Video
Graphics Fundamentals Video Results - Observations
• Satisfaction
– All students have managed to finish their assignment
at a satisfying level
• Different result from previous teaching approaches,
whereas there were cases that students could not
complete their task
• Engagement
– Student engagement was very high
• They seem to want to collaborate by themselves to show
progress and get internal feedback
• Also some students explored the possibilities of integrating
their solution with a motion tracking system
Results - Observations .
• Design
– Students demonstrated different design skills
– About one third of them did a more advanced GUI,
whereas the rest of them did something very minimal
• Graphics
– The majority of the students used a game engine to
implement the interface
– This allowed them to spend more time on the actual
design rather than on the implementation
– Only a few tried to use lower level APIs such as
OpenGL
Results - Observations ..
• Collaboration
– Most of the students focused on basic collaboration techniques
• Apart from the last presented project
• Tutorials
– Those who followed ’practical tutorials’ used Unity with ARToolKit
plug-in as their main development platform
– Students who focused on ’theoretical tutorials’ used C++ and
ARToolKit (C version), whereas for students using ’assessment
tutorials’ employed Vuforia
• Learning:
– Arts based students focused more on task based learning, trying to
illustrate a particular part of the theory
• They did not bother so much on the graphics quality
– Informatics students were focused more on implementing complete
scenarios or even a complete game
23/09/2019
21
Some Videos
BaekAR Video
https://www.youtube.com/watch?v=ZyZMdymlwms
IKEA Video
https://www.youtube.com/watch?v=ohDaowN1KYY
3D User Interface Design Video
https://www.youtube.com/watch?v=B4GVWk3L9hE
Multi-Players Learning Video
https://www.youtube.com/watch?v=jqE-EmIhGw4
Assignment Tips
23/09/2019
22
Assignment
• Make use of an AR API to create an educational
game
– i.e. ARToolKit
• Implementation in C/C++/C sharp
• Emphasis will be given on the interaction and
visualisation techniques
– Not on tracking!
• Deadline end of the term
Details
• The game should be focused on indoor
environments
– Not mobile!
• The topic is focused on designing a game/tool to
assist students to learn computer graphics
• Visualisation
– All types of multimedia information can be
superimposed
• Tracking
– Single or multiple markers
Visualisation
• ARToolKit main platform
• Graphics in ARToolkit C++ version is not well
supported
• Can wrap it with other applications
• Develop tools for handling different media
– Text, sound, 3D, images, video
Content
• Best to find it online
• Loads of resources
• Might need to make small adjustments
• Don’t model things from scratch, no time!
Markers Tracking
• Don’t need to create new tracking methods
– Just select the best one from the examples
presented in the lab
• But think of the presentation
– Single
– Multiple
– Combination
23/09/2019
23
Interaction
• Need to determine requirements and user
needs
• Take other constraints into account
– i.e. Time, hardware
• Also will depend on suitability of technology
for activity being supported
Report Structure
• Title page
• Contents
• Abstract (or summary) (1/2 page)
• Introduction (1 page)
• Background theory (3-5 pages)
• Methodology and results (5-10 pages)
• Conclusions (1 page)
• References
• Appendices
Questions