Real-time Pattern Detection in IP Flow Data using Apache Spark

D 2019

Real-time Pattern Detection in IP Flow Data using Apache Spark

ČERMÁK, Milan; Martin LAŠTOVIČKA a Tomáš JIRSÍK

Základní údaje

Originální název

Real-time Pattern Detection in IP Flow Data using Apache Spark

Autoři

ČERMÁK, Milan; Martin LAŠTOVIČKA a Tomáš JIRSÍK

Vydání

Washington DC, USA, 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), od s. 521-526, 6 s. 2019

Nakladatel

IEEE

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL, URL

Označené pro přenos do RIV

Ano

Kód RIV

RIV/00216224:14610/19:00108993

Organizační jednotka

Ústav výpočetní techniky

ISBN

978-1-7281-0618-2

ISSN

UT WoS

000469937200092

EID Scopus

2-s2.0-85067062108

Klíčová slova anglicky

Anomaly Detection;IP Flow;Apache Spark;Stream Processing

Štítky

core_A, firank_A, rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 25. 4. 2022 14:31, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

Detection of network attacks is a challenging task, especially concerning detection coverage and timeliness. The defenders need to be able to detect advanced types of attacks and minimize the time gap between the attack detection and its mitigation. To meet these requirements, we present a stream-based IP flow data processing application for real-time attack detection using similarity search techniques. Our approach extends capabilities of traditional detection systems and allows to detect not only anomalies and attacks that match exactly to predefined patterns but also their variations. The approach is demonstrated on detection of SSH authentication attacks. We describe a process of patterns definition and illustrate their usage in a real-world deployment. We show that our approach provides sufficient performance of IP flow data processing for real-time detection while maintaining versatility and ability to detect network attacks that have not been recognized by traditional approaches.

Návaznosti

EF16_019/0000822, projekt VaV

Název: Centrum excelence pro kyberkriminalitu, kyberbezpečnost a ochranu kritických informačních infrastruktur

Přiložené soubory

2019-IM-real-time-pattern-detection-in-ip-flow-data-using-apache-spark-presentation.pdf

2019-IM-real-time-pattern-detection-in-ip-flow-data-using-apache-spark-paper.pdf Verze souboru

Citovat

ČERMÁK, Milan; Martin LAŠTOVIČKA a Tomáš JIRSÍK. Real-time Pattern Detection in IP Flow Data using Apache Spark. Online. In 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM). Washington DC, USA: IEEE, 2019, s. 521-526. ISBN 978-1-7281-0618-2.

@inproceedings{1484741,
   author = {Čermák, Milan and Laštovička, Martin and Jirsík, Tomáš},
   address = {Washington DC, USA},
   booktitle = {2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM)},
   keywords = {Anomaly Detection;IP Flow;Apache Spark;Stream Processing},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Washington DC, USA},
   isbn = {978-1-7281-0618-2},
   pages = {521-526},
   publisher = {IEEE},
   title = {Real-time Pattern Detection in IP Flow Data using Apache Spark},
   url = {https://ieeexplore.ieee.org/document/8717827},
   year = {2019}
}

TY  - CONF
ID  - 1484741
AU  - Čermák, Milan - Laštovička, Martin - Jirsík, Tomáš
PY  - 2019
TI  - Real-time Pattern Detection in IP Flow Data using Apache Spark
PB  - IEEE
CY  - Washington DC, USA
SN  - 9781728106182
KW  - Anomaly Detection;IP Flow;Apache Spark;Stream Processing
UR  - https://ieeexplore.ieee.org/document/8717827
N2  - Detection of network attacks is a challenging task, especially concerning detection coverage and timeliness. The defenders need to be able to detect advanced types of attacks and minimize the time gap between the attack detection and its mitigation. To meet these requirements, we present a stream-based IP flow data processing application for real-time attack detection using similarity search techniques. Our approach extends capabilities of traditional detection systems and allows to detect not only anomalies and attacks that match exactly to predefined patterns but also their variations. The approach is demonstrated on detection of SSH authentication attacks. We describe a process of patterns definition and illustrate their usage in a real-world deployment. We show that our approach provides sufficient performance of IP flow data processing for real-time detection while maintaining versatility and ability to detect network attacks that have not been recognized by traditional approaches.
ER  -

ČERMÁK, Milan; Martin LAŠTOVIČKA a Tomáš JIRSÍK. Real-time Pattern Detection in IP Flow Data using Apache Spark. Online. In \textit{2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM)}. Washington DC, USA: IEEE, 2019, s.~521-526. ISBN~978-1-7281-0618-2.

Přehled o publikaci