11/2/2015
1
PA198
Augmented Reality Interfaces
Lecture 5
Designing Augmented Reality Interfaces
Fotis Liarokapis
liarokap@fi.muni.cz
2nd November 2015
User Interface Design
What is User Interface Design?
• The process of designing effective and user
friendly interfaces for software systems and
applications
– Applies in all computer science
– Other disciplines as well
• i.e. Hardware design
The User Interface
• Users usually judge a software application based
on its interface rather than its functionality
– A poorly designed interface can cause a user to
make catastrophic errors
– Poor user interface design is the main reason why so
many software systems are never used
Many Types of Interfaces
• Command
• Speech
• Data-entry
• Form fill-in
• Query
• Graphical
• Web
• Pen
• Augmented reality
• Gesture
Graphical User Interfaces (GUIs)
• GUIs allows users to interact with electronic
devices through graphical icons and visual
indicators such as secondary notation
– Opposed to text-based interfaces, typed command
labels or text navigation
• GUIs can be found in hand-held devices and
many applications exist
– i.e. Augmented reality
11/2/2015
2
GUI Characteristics
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
GUI Advantages
• They are easy to learn and use
– Without experience can use the system quickly
– Can switch quickly from one task to another
– Can interact with several different applications
– Information remains visible in its own window when
attention is switched
• Fast, full-screen interaction is possible with
immediate access to anywhere on the screen
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User-Centred Design
• In user-centred design the needs of the user are
paramount and where the user is involved in the
design process
• User interface design always involves the
development of prototype interfaces
User Interface Design Process
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User Interface Design Principles
• User interface design must consider the needs,
experience and capabilities of the system users
– Principles underlie interface designs although not all
are applicable to all designs
• Designers should:
– Be aware of people’s physical and mental limitations
(e.g. limited short-term memory)
– Recognise that people make mistakes
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User Interface Design Principles .
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
11/2/2015
3
Design Principles
• User familiarity
– The interface should be based on user-oriented terms and
concepts rather than computer concepts
• i.e. An office system should use concepts such as letters, documents,
folders etc. rather than directories, file identifiers, etc.
• Consistency
– The system should display an appropriate level of consistency
– Commands and menus should have the same format,
command punctuation should be similar, etc.
• Minimal surprise
– If a command operates in a known way, the user should be
able to predict the operation of comparable commands
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Design Principles .
• Recoverability
– The system should provide some resilience to user errors
and allow the user to recover from errors
• This might include an undo facility, confirmation of destructive
actions, 'soft' deletes, etc
• User guidance
– Some user guidance such as help systems, on-line
manuals, etc. should be supplied
• User diversity
– Interaction facilities for different types of user should be
supported
• i.e. Accessibility
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
User-System Interaction
• Two problems must be addressed in interactive
systems design
– How should information from the user be provided to
the computer system?
– How should information from the computer system
be presented to the user?
• User interaction and information presentation
may be integrated through a coherent
framework such as a user interface metaphor
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Interaction Styles
Interaction Styles
• Direct manipulation
• Menu selection
• Form fill-in
• Command language
• Natural language
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Pros and Cons
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Interaction
style
Main advantages Main disadva ntages Application
examples
Direct
manipulation
Fast and intuitive
interaction
Easy to learn
May be hard to implement.
Only suitable where there is a
visual metaphor for tasks and
objects.
Video games
CAD systems
Menu
selection
Avoids user error
Little typing required
Slow for experienced users.
Can become complex if many
menu options.
Most generalpurpose
systems
Form fill-in Simple data entry
Easy to learn
Checkable
Takes up a lot of screen space.
Causes problems where user
options do not match the form
fields.
Stock control,
Personal loan
processing
Command
language
Powerful and flexible Hard to learn.
Poor error manage ment.
Operating systems,
Command and
control systems
Natural
language
Accessible to casual
users
Easily extended
Requires more typing.
Natural languageund erstanding
systems are unreliable.
Information
retrieval systems
11/2/2015
4
Direct Manipulation Advantages
• Users feel in control of the computer and are
less likely to be intimidated by it
• User learning time is relatively short
• Users get immediate feedback on their
actions so mistakes can be quickly detected
and corrected
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Direct Manipulation Disadvantages
• The derivation of an appropriate information
space model can be very difficult
• Given that users have a large information
space, what facilities for navigating around
that space should be provided?
• Direct manipulation interfaces can be complex
to program and make heavy demands on the
computer system
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Menu Systems
• Users make a selection from a list of
possibilities presented to them by the system
• The selection may be made by pointing and
clicking with a mouse, using cursor keys or by
typing the name of the selection
• May make use of simple-to-use terminals such
as touch screens
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Menu Systems Advantages
• Users need not remember command names as
they are always presented with a list of valid
commands
• Typing effort is minimal
• User errors are trapped by the interface
• Context-dependent help can be provided
– The user’s context is indicated by the current menu
selection
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Menu Systems Disadvantages
• Actions which involve logical conjunction (and)
or disjunction (or) are awkward to represent
• Menu systems are best suited to presenting a
small number of choices
– If there are many choices, some menu structuring
facility must be used
• Experienced users find menus slower than
command language
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Command Interfaces
• User types commands to give instructions to the
system
– i.e. UNIX
• May be implemented using cheap terminals
• Easy to process using compiler techniques
• Commands of arbitrary complexity can be
created by command combination
• Concise interfaces requiring minimal typing can
be created
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
11/2/2015
5
Command Interfaces Disadvantages
• Users have to learn and remember a command
language
– Unsuitable for occasional users
• Users make errors in command
– An error detection and recovery system is required
• System interaction is through a keyboard so
typing ability is required
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Command Languages
• Often preferred by experienced users because
they allow for faster interaction with the system
• Not suitable for casual or inexperienced users
• May be provided as an alternative to menu
commands
– i.e. Keyboard shortcuts
• In some cases, a command language interface
and a menu-based interface are supported at
the same time
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Natural Language Interfaces
• The user types a command in a natural language
• Generally, the vocabulary is limited and these
systems are confined to specific application
domains
– i.e. Timetable enquiries
• NL processing technology is now good enough to
make these interfaces effective for casual users
– But experienced users find that they require too
much typing
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Multiple User Interfaces
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Information Presentation
• Information presentation is concerned with
presenting system information to system users
• The information may be presented directly (e.g.
text in a word processor) or may be transformed
in some way for presentation (e.g. in some
graphical form)
• The Model-View-Controller approach is a way of
supporting multiple presentations of data
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Information Presentation .
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
11/2/2015
6
Types of Information
• Static information
– Initialised at the beginning of a session
• Does not change during the session
– May be either numeric or textual
• Dynamic information
– Changes during a session and the changes must
be communicated to the system user
– May be either numeric or textual
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
Information Display Factors
• Is the user interested in precise information or
data relationships?
• How quickly do information values change?
• Must the change be indicated immediately?
• Must the user take some action in response to a
change?
• Is there a direct manipulation interface?
• Is the information textual or numeric? Are
relative values important?
https://www.ics.uci.edu/~taylor/ics52_fq01/UISlides.pdf
AR Interfaces
Introduction to AR Interfaces
• Browsing Interfaces
– Simple (conceptually!), unobtrusive
• 3D AR Interfaces
– Expressive, creative, require attention
• Tangible Interfaces
– Embedded into conventional environments
• Tangible AR
– Combines TUI input + AR display
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Browsing Interfaces
• 2D/3D virtual objects are
registered in 3D
– “VR in Real World”
• Interaction
– 2D/3D virtual viewpoint
control
• Applications
– Visualization, training
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
3D AR Interfaces
• Virtual objects displayed in 3D
physical space and
manipulated
– HMDs and 6DOF head-tracking
– 6DOF hand trackers for input
• Interaction
– Viewpoint control
– Traditional 3D user interface
interaction
• Manipulation, selection, etc
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Kiyokawa, et al. 2000
11/2/2015
7
Tangible Interfaces
• Dangling String
– Jeremijenko 1995
– Ambient Ethernet monitor
– Relies on peripheral cues
• Ambient Fixtures
– Dahley, Wisneski, Ishii 1998
– Use natural material qualities for
information display
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Augmented Surfaces and Tangible
Interfaces
• Basic principles
– Virtual objects are
projected on a surface
– Physical objects are used
as controls for virtual
objects
– Support for collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR
• AR overcomes limitation of
TUIs
– Enhance display possibilities
– Merge task/display space
– Provide public and private views
• TUI + AR = Tangible AR
– Apply TUI methods to AR
interface design
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Space Multiplexed vs. Time
Multiplexed
• Space-multiplexed
– Many devices each with one function
• Quicker to use, more intuitive, clutter
• Real Toolbox
• Time-multiplexed
– One device with many functions
• Space efficient
• Mouse
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR: Tiles (Space Multiplexed)
• Tiles semantics
– Data tiles
– Operation tiles
• Operation on tiles
– Proximity
– Spatial arrangements
– Space-multiplexed
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR: Time-Multiplexed
Interaction
• Use of natural physical object
manipulations to control
virtual objects
– Catalog book:
• Turn over the page
– Paddle operation:
• Push, shake, incline, hit, scoop
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
11/2/2015
8
Interface Design Path
• Prototype Demonstration
• Adoption of Interaction Techniques from other
interface metaphors
• Development of new interface metaphors
appropriate to the medium
• Development of formal theoretical models for
predicting and modeling user actions
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Interface Metaphors
• Designed to be similar to a physical entity but
also has own properties
– e.g. desktop metaphor, search engine
• Exploit user’s familiar knowledge, helping them
to understand ‘the unfamiliar’
• Conjures up the essence of the unfamiliar activity,
enabling users to leverage of this to understand
more aspects of the unfamiliar functionality
• People find it easier to learn and talk about what
they are doing at the computer interface in terms
familiar to them
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Example: The Spreadsheet
• Analogous to ledger
sheet
• Interactive and
computational
• Easy to understand
• Greatly extending what
accountants and others
could do
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Why was it so good?
• It was simple, clear, and obvious to the users
how to use the application and what it could do
• “it is just a tool to allow others to work out their
ideas and reduce the tedium of repeating the
same calculations.”
• Capitalized on user’s familiarity with ledger
sheets
• Got the computer to perform a range of
different calculations in response to user input
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Another Classic
• 8010 Star office system targeted at workers
not interested in computing per se
• Spent several person-years at beginning
working out the conceptual model
• Simplified the electronic world, making it
seem more familiar, less alien, and easier to
learn
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
The Star Interface
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
11/2/2015
9
Benefits of Interface Metaphors
• Makes learning new systems easier
• Helps users understand the underlying
conceptual model
• Can be innovative and enable the realm of
computers and their applications to be made
more accessible to a greater diversity of users
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Problems with interface metaphors
(Nielson, 1990)
• Break conventional and cultural rules
– i.e. Recycle bin placed on desktop
• Can constrain designers in the way they
conceptualize a problem
• Conflict with design principles
• Forces users to only understand the system in terms
of the metaphor
• Designers can inadvertently use bad existing designs
and transfer the bad parts over
• Limits designers’ imagination with new conceptual
models
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
AR Design Principles
• Interface Components
– Physical components
– Display elements
• Visual/audio
• Interaction metaphors
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
Tangible AR Design Principles
• Tangible AR Interfaces use TUI principles
– Physical controllers for moving virtual content
– Support for spatial 3D interaction techniques
– Time and space multiplexed interaction
– Support for multi-handed interaction
– Match object affordances to task requirements
– Support parallel activity with multiple objects
– Allow collaboration between multiple users
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
The MagicBook
• Design Goals:
– Allows user to move smoothly between reality
and virtual reality
– Support collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Features
• Seamless transition between Reality and
Virtuality
– Reliance on real decreases as virtual increases
• Supports egocentric and exocentric views
– User can pick appropriate view
• Computer becomes invisible
– Consistent interface metaphors
– Virtual content seems real
• Supports collaboration
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
11/2/2015
10
MagicBook Collaboration
• Collaboration on multiple
levels:
– Physical Object
– AR Object
– Immersive Virtual Space
• Egocentric + exocentric
collaboration
– Multiple multi-scale users
• Independent Views
– Privacy, role division, scalability
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Technology
• Reality
– No technology
• Augmented Reality
– Camera – tracking
– Switch – fly in
• Virtual Reality
– Compass – tracking
– Press pad – move
– Switch – fly out
Billinghurst, M. COSC 426: Augmented Reality, Sep 5th, 2012.
MagicBook Video
https://www.youtube.com/watch?v=tNMljw0F-aw
Generic AR Interface
Layers of the AR Interface
AR Applications
Interface framework
Interaction algorithms
Audio algorithms based on OpenAL API
Visual algorithms based on OpenGL API
Vision SDK Video Libraries
Windows OS
Libraries and Drivers
(Graphics and Sound Card, Video Camera, etc)
ARToolKit tracking Libraries
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Interactions
Standard I/O
Computer
Physical Manipulation Touch Screen
Interface Menu Interaction
SpaceMouseUser
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
11
Interface Visualisation Capabilities
• Visualisation
– 3D models
– 3D text
– 3D sound
– Videos
– Pictures
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
GUI Environment
• The GUI consists of a
– Menu
– Toolbar
– Status bar
– Dialog boxes
• Allows participants to have the same access to
the augmented virtual information as they
have had using standard interaction
techniques
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Some Algorithms
Multiple Augmentation Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Tracking Multiple Markers
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Textual Augmentation Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
12
3D Sound Generation
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
SpaceMouse Interaction
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Magic Book Algorithm
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Designing Markers
Square Marker Cards
• The easiest solution
• Will work out well for
assignment
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Other Shapes Markers
• There are limitations with the number of
square markers that can be designed
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
13
Hand Markers
• Does not work very well with ARToolKit
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Tracking & Calibration
Measuring ARToolKit’s Tracking Error
• In wide area applications, the positioning
accuracy of ARToolKit is not very robust
• In distances between 1m and 2.5m the error
in the x and y values increases proportionally
with the distance from the marker
• Calculate error in distances ranging between
20 cm and 80 cm under normal lighting
conditions
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Measuring ARToolKit’s Tracking Error .
• The optimal area, which
contains the least error, is the
one that is perpendicular to
the marker card
• A rigid path is set so that the
camera can not loose its
direction while moving
backwards
• Numerous measurements of
the location of the web
camera in a local co-ordinate
system
Web
Camera
Marker
20 cm
60 cm
Wall
Ground
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Measuring ARToolKit’s Tracking Error ..
• Error is proportional to the distance
Calculation of ARToolKit's Error (Method I)
0
100
200
300
400
500
600
700
800
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57
step
mm
Actual Values Measured Values
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Measuring ARToolKit’s Tracking Error …
• Camera facing the marker at variable angle
(yaw) having the other two (pitch, roll) stable
Calculation of ARToolKit's Error (Method II)
0
100
200
300
400
500
600
700
800
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57
step
mm
Actual Values Measured Values
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
14
Measuring ARToolKit’s Tracking Error
….
• Differences in the error produced from the
experiments compared with the actual values
Overview of Measured Error
0
100
200
300
400
500
600
700
800
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57
step
mm
Actual Values Method I Method II
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Calculating Camera
Parameters
ARToolKit’s Calibration Method
• ARToolKit provides two software tools called
calib_dist and calib_param that can be used to
calculate these camera properties
– calib_dist is used to measure the lens distortion
and the image centre point
– calib_param is used to compute the focal length of
the camera
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
ARToolKit’s Calibration Method .
• In calib_dist program, an image of a pattern
6x4 dots spaced equally apart is used
Camera calibration for lens distortion Camera calibration for focal length
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
KLT Tracking Method
• A feature-based vision method is known as
the KLT (Kanade-Lucas-Tomasi) algorithm
based on a model or affine image changes
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Camera Calibration Toolbox for Matlab
• Offers an automatic mechanism for counting the
number of squares in each grid
• All calibration images are searched and focal and
distortion factor are automatically estimated
Estimated focal (pixels) Estimated distortion factor (kc)
Image 1 201.2503 -0.2
Image 2 224.1415 -0.2
Image 3 231.7859 -0.2
Image 4 114.2254 -0.055
Image 5 265.8596 -0.28
Image 6 211.5925 -0.2
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
15
Calibration Without Lens Distortion
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Calculation of Extrinsic Parameters
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Error Calculation of Extrinsic Camera
Parameters
Initial Error Corrected Error
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Case Studies
Learning Requirements
• The potential benefits of AR applied to HE
include:
– Visualisation of the theoretical parts in 3D
– Practical exploration of the theory
– Effective collaboration and discussion amongst the
participants
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Learning Requirements .
• An ideal AR system must include at least the
following requirements
– Be simple and robust
– Provide the user with clear and concise information
– Enable the teacher to input information in a simple
and effective manner
– Enable easy interaction between users
– Make complex procedures transparent to the user
– Be cost effective
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
11/2/2015
16
Kolb’s Learning Cycle Enhanced Learning Cycle using AR
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Design of Teaching Material
• Off-line process and consists of:
– A set of distinctive marker cards - this is the link
between the real and the digital information;
– Digital information - is the digital information
including pictures, 3D models, textual
descriptions, video animations and auditory
information;
– Educational tutorials - consist of a number of
predefined learning scenarios which combine
theory and practice at the same time
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
Tutorials Generation
• Theoretical tutorials
– Most important parts of the theory are described through
visual and auditory means of augmentation having
limited user interaction
• Practical tutorials
– Based on the theory, students have to use the specific set
of marker cards to describe a simple but complete
process using collaborative interaction techniques
• Assessment tutorials
– 3D graphical representations of theoretical and practical
issues are assessed in a semi-automatic way based on all
the proposed types of human-computer interactions
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
First Prototype: MARIE
• Initial prototype for AR in
education
• Focused on teaching
electronics
Liarokapis,F., Petridis, P., Lister, P.F., White, M. MultimediaAugmented Reality Interface for E-Learning(MARIE), World Transactionson Engineeringand
Technology Education,UICEE,1(2): 173-176, 2002. (ISSN: 1446-2257)
Second Prototype: Wed3D & AR
Pilot user experiencing an AR learning scenario,
University of SussexAugmentation of a camshaft
Web3D visualisation of a camshaft
Liarokapis,F., Mourkoussis,N.,White,M.,Darcy,J.,Sifniotis,M.,Petridis,P.,Basu,A.,Lister, P.F.Web3Dand Augmented Realityto supportEngineeringEducation,WorldTransactions on Engineeringand Technology
Education,UICEE,3(1):11-14,2004.(ISSN:1446-2257)
11/2/2015
17
Design of Teaching Material
• Create appropriate markers
• Create material
• Link together
Liarokapis, F., Augmented Reality Interfaces - Architectures for Visualising and Interacting with Virtual Information, Sussex theses S 5931, Department of
Informatics, School of Science and Technology, University of Sussex, Falmer, UK, 2005
University of Sussex
• Application in Informatics
• Goal:
– Understand the computer
Liarokapis,F., Anderson,E.UsingAugmentedRealityas a Medium to Assist Teachingin Higher Education,Proc.ofthe 31st Annual Conference ofthe European Associationfor Computer Graphics (Eurographics2010),
Education Program,Norrkoping,Sweden,4-7May,9-16,2010.(ISSN:1017-4656)
City University
• Application in Geography and GIS
• Goal:
– Understand GIS in London
Liarokapis,F., Anderson,E.UsingAugmentedRealityas a Medium to Assist Teachingin Higher Education,Proc.ofthe 31st Annual Conference ofthe European Associationfor Computer Graphics (Eurographics2010),
Education Program,Norrkoping,Sweden,4-7May,9-16,2010.(ISSN:1017-4656)
Video City University
Coventry University
• Application in
Computer Graphics
• Goal
– Understand basic
principles in CG
Liarokapis,F.Augmented RealityInterfaces for AssistingComputer Games UniversityStudents,Bulletinofthe TechnicalCommittee on LearningTechnology,IEEEComputingSociety,14(4):7-10,October, 2012.(ISSN:
2241-3766)
Evaluation
• Very useful tool
• Need some time to adapt
• Teacher can combine
different learning approaches
– i.e. Activity Lead Learning
11/2/2015
18
Some Videos
BaekAR Video
https://www.youtube.com/watch?v=ZyZMdymlwms
IKEA Video
https://www.youtube.com/watch?v=ohDaowN1KYY
3D User Interface Design Video
https://www.youtube.com/watch?v=B4GVWk3L9hE
Multi-Players Learning Video
https://www.youtube.com/watch?v=jqE-EmIhGw4
Assignment Tips
11/2/2015
19
Assignment
• Make use of an AR API to create an educational
game
– i.e. ARToolKit
• Implementation in C/C++/C sharp
• Emphasis will be given on the interaction and
visualisation techniques
– Not on tracking!
• Deadline end of the term
Details
• The game should be focused on indoor
environments
– Not mobile!
• The topic is focused on designing a game/tool to
assist students to learn computer graphics
• Visualisation
– All types of multimedia information can be
superimposed
• Tracking
– Single or multiple markers
Visualisation
• ARToolKit main platform
• Graphics in ARToolkit C++ version is not well
supported
• Can wrap it with other applications
• Develop tools for handling different media
– Text, sound, 3D, images, video
Content
• Best to find it online
• Loads of resources
• Might need to make small adjustments
• Don’t model things from scratch, no time!
Markers Tracking
• Don’t need to create new tracking methods
– Just select the best one from the examples
presented in the lab
• But think of the presentation
– Single
– Multiple
– Combination
11/2/2015
20
Interaction
• Need to determine requirements and user
needs
• Take other constraints into account
– i.e. Time, hardware
• Also will depend on suitability of technology
for activity being supported
Report Structure
• Title page
• Contents
• Abstract (or summary) (1/2 page)
• Introduction (1 page)
• Background theory (3-5 pages)
• Methodology and results (5-10 pages)
• Conclusions (1 page)
• References
• Appendices
Questions