SegmentCodeList: Unsupervised Representation Learning for Human
Skeleton Data Retrieval

D 2023

SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval

SEDMIDUBSKÝ, Jan, Fabio CARRARA and Giuseppe AMATO

Basic information

Original name

SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval

Authors

SEDMIDUBSKÝ, Jan (203 Czech Republic, guarantor, belonging to the institution), Fabio CARRARA and Giuseppe AMATO

Edition

Cham, 45th European Conference on Information Retrieval (ECIR), p. 110-124, 15 pp. 2023

Publisher

Springer

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10200 1.2 Computer and information sciences

Country of publisher

Czech Republic

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/23:00130177

Organization unit

Faculty of Informatics

ISBN

978-3-031-28237-9

ISSN

DOI

http://dx.doi.org/10.1007/978-3-031-28238-6_8

UT WoS

000995489700008

Keywords in English

3D skeleton sequence;segment similarity;unsupervised feature learning;Variational AutoEncoder;segment code list;action retrieval

Abstract

V originále

Recent progress in pose-estimation methods enables the extraction of sufficiently-precise 3D human skeleton data from ordinary videos, which offers great opportunities for a wide range of applications. However, such spatio-temporal data are typically extracted in the form of a continuous skeleton sequence without any information about semantic segmentation or annotation. To make the extracted data reusable for further processing, there is a need to access them based on their content. In this paper, we introduce a universal retrieval approach that compares any two skeleton sequences based on temporal order and similarities of their underlying segments. The similarity of segments is determined by their content-preserving low-dimensional code representation that is learned using the Variational AutoEncoder principle in an unsupervised way. The quality of the proposed representation is validated in retrieval and classification scenarios; our proposal outperforms the state-of-the-art approaches in effectiveness and reaches speed-ups up to 64x on common skeleton sequence datasets.

Links

EF16_019/0000822, research and development project

Name: Centrum excelence pro kyberkriminalitu, kyberbezpečnost a ochranu kritických informačních infrastruktur

Citovat

SEDMIDUBSKÝ, Jan, Fabio CARRARA and Giuseppe AMATO. SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval. Online. In 45th European Conference on Information Retrieval (ECIR). Cham: Springer, 2023, p. 110-124. ISBN 978-3-031-28237-9. Available from: https://dx.doi.org/10.1007/978-3-031-28238-6_8.

@inproceedings{2245929,
   author = {Sedmidubský, Jan and Carrara, Fabio and Amato, Giuseppe},
   address = {Cham},
   booktitle = {45th European Conference on Information Retrieval (ECIR)},
   doi = {http://dx.doi.org/10.1007/978-3-031-28238-6_8},
   keywords = {3D skeleton sequence;segment similarity;unsupervised feature learning;Variational AutoEncoder;segment code list;action retrieval},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Cham},
   isbn = {978-3-031-28237-9},
   pages = {110-124},
   publisher = {Springer},
   title = {SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval},
   year = {2023}
}

TY  - JOUR
ID  - 2245929
AU  - Sedmidubský, Jan - Carrara, Fabio - Amato, Giuseppe
PY  - 2023
TI  - SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval
PB  - Springer
CY  - Cham
SN  - 9783031282379
KW  - 3D skeleton sequence;segment similarity;unsupervised feature learning;Variational AutoEncoder;segment code list;action retrieval
N2  - Recent progress in pose-estimation methods enables the extraction of sufficiently-precise 3D human skeleton data from ordinary videos, which offers great opportunities for a wide range of applications. However, such spatio-temporal data are typically extracted in the form of a continuous skeleton sequence without any information about semantic segmentation or annotation. To make the extracted data reusable for further processing, there is a need to access them based on their content. In this paper, we introduce a universal retrieval approach that compares any two skeleton sequences based on temporal order and similarities of their underlying segments. The similarity of segments is determined by their content-preserving low-dimensional code representation that is learned using the Variational AutoEncoder principle in an unsupervised way. The quality of the proposed representation is validated in retrieval and classification scenarios; our proposal outperforms the state-of-the-art approaches in effectiveness and reaches speed-ups up to 64x on common skeleton sequence datasets.
ER  -

SEDMIDUBSKÝ, Jan, Fabio CARRARA and Giuseppe AMATO. SegmentCodeList: Unsupervised Representation Learning for Human Skeleton Data Retrieval. Online. In \textit{45th European Conference on Information Retrieval (ECIR)}. Cham: Springer, 2023, p.~110-124. ISBN~978-3-031-28237-9. Available from: https://dx.doi.org/10.1007/978-3-031-28238-6\_{}8.

Detailed Information on Publication Record