Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Jan Sedmidubsky
Jakub Valcik
Pavel Zezula
Motion Retrieval
for Security Applications
Faculty of Informatics
Masaryk University
Brno, Czech Republic
fi-logo cartwheel.gif
1/11

Jan Sedmidubsky
•Motion capturing devices
–Record and digitize human movement into motion capture data in real time
–Examples: Microsoft Kinect, ASUS Xtion
•Motion capture data
–3D joint coordinates estimated for each video frame
–e.g., Microsoft Kinect v2: skeleton model with 25 joints
September 23, 2014
Motion Retrieval for Security Applications
Introduction
2/11

Jan Sedmidubsky
m1.png healthCare2.png
•Analysis of recorded motion data in various areas:
–Health care – success of rehabilitative treatments
–Sports – performance aspect comparison
–Security – person identification, event detection
–Computer animation – realistic motion synthesis
–
–
•Aspects:
–Motion features
–Similarity comparison
–Indexing & searching
September 23, 2014
Motion Retrieval for Security Applications
Introduction
Similar?
Similarity model – depends on
the application purpose
3/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Applications We Focus on
•Content-based (sub-)motion retrieval:
–Search for (sub-)motions in a database that are visually similar to a query motion example
•
–
•Person identification:
–Search for database motions that should belong to the same person as the query motion, in order to
reveal the name of query person
Query motion
Result – 3 similar motions
Query motion of
unknown person
Result – estimated name of query person
Jack
Jack
Emma
Jack (80%)   Emma (20%)
4/11
Index:
content-based retrieval
Index:
person identification

Jan Sedmidubsky
•Content-based retrieval index:
–Motion features – joint-angle rotations
•Each frame = 28-D vector of angles of joints
•Individual frames compared by the L1 metric
–Motion similarity comparison
•Average distance between selected key frames
–Indexing & searching
•A specialized key-frame retrieval algorithm
•Sedmidubsky, J., Valcik, J., and Zezula P. A Key-Pose Similarity Algorithm for Motion Data
Retrieval. In 12th International Conference on Advanced Concepts for Intelligent Vision Systems
(ACIVS 2013). Springer, 2013.
September 23, 2014
Motion Retrieval for Security Applications
Content-based Retrieval Index
Index:
content-based retrieval
5/11

Jan Sedmidubsky
•Person identification index:
–Motion features
•Extracted separately for individual walking cycles
–Each frame = 21-D vector of relative velocities of joints ~ each walking cycle represented by 21
time series
–Each time series smoothed by Fourier transform. (10 harm.)
•Walking cycle = matrix 21x10 of transformed values
–Motion similarity comparison
•Matrices of walking cycles compared by a weighted L1 metric
•Combination of skeleton proportions and walking cycles (matrices)
–Indexing & searching – parallel sequential scan
•KNN classification for estimating person name from the most similar retrieved walking cycles
September 23, 2014
Motion Retrieval for Security Applications
Person Identification Index
Index:
person identification
frames
6/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Architecture of Retrieval System
HDM05 + CMU
motion database
Index:
content-based retrieval
GUI: online web application
•content-based + identification
Index:
person identification
GUI: desktop Kinect application
•identification
Kinect
database
Query motion
Result – 3 similar motions
Query motion of
unknown person
Result – estimated name of query person
Jack
Jack
Emma
Jack (80%)   Emma (20%)
7/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Online Web Application
•Online demo: http://disa.fi.muni.cz/motion-match/
•HDM05 + CMU motion databases:
–2,515 motions of 5.4M frames ~ 12 hours of video
8/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Desktop Kinect Application
•Database has to be created
9/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Future Work
•Future research directions:
–Developing new similarity models to achieve better retrieval effectiveness
–Developing more efficient retrieval algorithms to speed up the search process
–Fusing face and motion recognition methods to improve the quality of person identification
–Creating a larger database of different kinds of motions by the Kinect device
–
– bachelor and master theses
10/11

Jan Sedmidubsky
September 23, 2014
Motion Retrieval for Security Applications
Questions?
•Thank you for your attention.
•
•
•Try our online demo:
•http://disa.fi.muni.cz/motion-match/
11/11