Tracking subjects and detecting relationships in crowded city
videos

J 2022

Tracking subjects and detecting relationships in crowded city videos

ELIÁŠ, Petr, Matúš MACKO, Jan SEDMIDUBSKÝ and Pavel ZEZULA

Basic information

Original name

Tracking subjects and detecting relationships in crowded city videos

Authors

ELIÁŠ, Petr (203 Czech Republic, guarantor, belonging to the institution), Matúš MACKO (703 Slovakia, belonging to the institution), Jan SEDMIDUBSKÝ (203 Czech Republic, belonging to the institution) and Pavel ZEZULA (203 Czech Republic, belonging to the institution)

Edition

Multimedia Tools and Applications, Springer, 2022, 1380-7501

Other information

Language

English

Type of outcome

Článek v odborném periodiku

Field of Study

10200 1.2 Computer and information sciences

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

References:

URL

Impact factor

Impact factor: 3.600

RIV identification code

RIV/00216224:14330/22:00129021

Organization unit

Faculty of Informatics

DOI

http://dx.doi.org/10.1007/s11042-021-11891-z

UT WoS

000743891900007

Keywords in English

Multi-subject tracking;Relationship detection;2D skeleton sequences;Video analysis;Smart cities

Abstract

V originále

Multi-subject tracking in crowded videos is an established yet challenging research direction in computer vision and information processing. High applicability of multi-subject tracking is demonstrated in smart cities (e.g., public safety, crowd management, urban planning), autonomous driving vehicles, robotic vision, or psychology (e.g., social interaction and crowd behavior understanding). In this work, we propose a real-time approach that reveals tracks of subjects in ordinary videos, captured in highly populated pedestrian areas, such as squares, malls, and stations. The tracks are discovered based on the proximity of detected bounding boxes of subjects in consecutive video frames. The reduction of track fragmentation and identity switching is achieved by the re-identification phase that uses caching of unassociated detections and mutual projection of interrupted tracks. As the proposed approach does not require time-consuming extraction of appearance-based features, the superior tracking speed is achieved. In addition, we demonstrate tracker usability and applicability by extracting valuable information about body-joint positions from discovered tracks, which opens promising possibilities for detecting human relationships and interactions. We demonstrate accurate detection of couples based on their holding hand activity and families based on children's body proportions. The discovery of these entitative groups is especially challenging in crowded city scenes where many subjects appear in each frame.

Links

GA19-02033S, research and development project

Name: Vyhledávání, analytika a anotace datových toků lidských pohybů

Investor: Czech Science Foundation

Citovat

ELIÁŠ, Petr, Matúš MACKO, Jan SEDMIDUBSKÝ and Pavel ZEZULA. Tracking subjects and detecting relationships in crowded city videos. Multimedia Tools and Applications. Springer, 2022, Neuveden, January, p. 1-23. ISSN 1380-7501. Available from: https://dx.doi.org/10.1007/s11042-021-11891-z.

@article{1824299,
   author = {Eliáš, Petr and Macko, Matúš and Sedmidubský, Jan and Zezula, Pavel},
   article_number = {January},
   doi = {http://dx.doi.org/10.1007/s11042-021-11891-z},
   keywords = {Multi-subject tracking;Relationship detection;2D skeleton sequences;Video analysis;Smart cities},
   language = {eng},
   issn = {1380-7501},
   journal = {Multimedia Tools and Applications},
   title = {Tracking subjects and detecting relationships in crowded city videos},
   url = {https://link.springer.com/article/10.1007/s11042-021-11891-z},
   volume = {Neuveden},
   year = {2022}
}

TY  - JOUR
ID  - 1824299
AU  - Eliáš, Petr - Macko, Matúš - Sedmidubský, Jan - Zezula, Pavel
PY  - 2022
TI  - Tracking subjects and detecting relationships in crowded city videos
JF  - Multimedia Tools and Applications
VL  - Neuveden
IS  - January
SP  - 1-23
EP  - 1-23
PB  - Springer
SN  - 13807501
KW  - Multi-subject tracking;Relationship detection;2D skeleton sequences;Video analysis;Smart cities
UR  - https://link.springer.com/article/10.1007/s11042-021-11891-z
N2  - Multi-subject tracking in crowded videos is an established yet challenging research direction in computer vision and information processing. High applicability of multi-subject tracking is demonstrated in smart cities (e.g., public safety, crowd management, urban planning), autonomous driving vehicles, robotic vision, or psychology (e.g., social interaction and crowd behavior understanding). In this work, we propose a real-time approach that reveals tracks of subjects in ordinary videos, captured in highly populated pedestrian areas, such as squares, malls, and stations. The tracks are discovered based on the proximity of detected bounding boxes of subjects in consecutive video frames. The reduction of track fragmentation and identity switching is achieved by the re-identification phase that uses caching of unassociated detections and mutual projection of interrupted tracks. As the proposed approach does not require time-consuming extraction of appearance-based features, the superior tracking speed is achieved. In addition, we demonstrate tracker usability and applicability by extracting valuable information about body-joint positions from discovered tracks, which opens promising possibilities for detecting human relationships and interactions. We demonstrate accurate detection of couples based on their holding hand activity and families based on children's body proportions. The discovery of these entitative groups is especially challenging in crowded city scenes where many subjects appear in each frame.
ER  -

ELIÁŠ, Petr, Matúš MACKO, Jan SEDMIDUBSKÝ and Pavel ZEZULA. Tracking subjects and detecting relationships in crowded city videos. \textit{Multimedia Tools and Applications}. Springer, 2022, Neuveden, January, p.~1-23. ISSN~1380-7501. Available from: https://dx.doi.org/10.1007/s11042-021-11891-z.

Detailed Information on Publication Record