Key-Frame Extraction for 3D Human Motion Sequence Segmentation Michal Balazia Laboratory of Data Intensive Systems and Applications Faculty of Informatics Masaryk University October 7, 2013 Michal Baläzia (DISA) Key-Frame Extraction October 7, 2013 1/9 3D Human Motion Capture Employment motion simulation and exposition - avatar graphics content-based retrieval in motion database biomechanical analysis - gait disorders detection, rehabilitation generating new motion instances Neck Right \ B.Hand Shoulder •-•- Lett Shoulder L Hand —• KOCH • 3D data capturable via Microsoft Kinect or on-body sensors • Human motion in the form of a sequence of body poses a Pose characterised by 3D coordinates of selected body points and time • Various extractable features • joint angles, their velocity or acceleration • body points' distances • relational features 1 -00.0 Michal Baläzia (DISA) Key-Frame Extraction October 7, 2013 2 /9 Key-Frames • Key-frames are time frames of motion sequence extracted according to specific sampling strategy. • Purposes • compression of motion data • motion retrieval » motion sequence segmentation • Two base key-frame sampling strategies • uniform - each two consecutive key-frames of equal time difference • adaptive - respect to motion sequence (extremal poses, turnover, etc.) • Approaches to key-frame extraction - Assa, MuIler, Gong, Xiao, Liu ■0 0.0 Michal Baläzia (DISA) Key-Frame Extraction October 7, 2013 3 /9 Assa Assa - curve averaging • Pose consists of skeletal joints and their associated aspects: (1) positions, (2) angles, (3) velocity, (4) angular velocity • x{ - value of aspect a in frame / • High-dimensional curve x{ (4x#joints) is reduced by RMDS algorithm to a curve C(f) of 5-8 dimensions a Point p in C(p) is projected onto average curve C(p) • rp = \C(p) — C(p)\ - distance at point p ^~ • Iterative key-frame extractor algorithm: 1. add pi of maximum rPi to key-frames 2. modify C(p) to touch C(p) October 7, 2013 Müller Müller - genetic learning • F = (Fi,..., Ff) - set of / relational features • F-segment of data stream D represented by matrix Mp[D] • Example: F = (FLeftKneeBent, FRightKneeBent), K = 5 F-segments • error tolerant (0/1 —> *) motion class patterns X G {0,1, *}fxK • Vk C {0,1}^ - subset of alternative feature vectors = fuzzy set • T+/~ - set of positive/negative training F-motions • Individual described by element of T+ and submatrix of Mp[D] • Mutations: change element of T+, change row or column of Mp[D] • Optimization in terms of recall and performance of fuzzy query V(X) = (Vi,...,VK) October 7, 2013 Gong, Xiao, Liu Gong - local-motion energy extremes • Of - angle between limb I and axis a • V = [cos(6f), cos(6f), cos(6f),..., cos(6f2), cos(6f2), cos(^i2)] " Pose • Ei = \tpi — ipi-i\2 - energy in 2-th frame of pose tpi • Key-frames are frames i of extremal Ei Xiao - angle extremes • of^ - angle between limb bone I and central bone in frame i • Key-frames are frames i such that 31 £ {1,..., 8} : of^ is extremal Liu - cluster centroids • rixi - rotational parameter of Ihip/rhip/chest in axis x/y/z in frame i • Ei — [flxi i Tlyi i flzi i T'rxi > T'ryi i frzi > T'cxi > fcyi > fczi] ~ frame •