Semantically Partitioned Complex Event Processing Lasaris Filip Nguyen October 9, 2014 Filip Nguyen Complex Event Processing October 9, 2014 1/22 Q| Complex Event Processing Ql Semantically Partitioned CEP • Goals of Research • Model • Implementation • Benefits Ql Planned Experimental Datasets Filip Nguyen Complex Event Processing October 9, 2014 2/22 Ql Complex Event Processing Q Semantically Partitioned CEP • Goals of Research • Model • Implementation • Benefits Planned Experimental Datasets Complex Event Processing Motivation and Usage Stock Customers Stock Market Analysis tools Stock Manipulator • Event: An object that is a record of an activity in a system. • Figure shows a real-world concept Stock Market. From IT perspective it is a number of communicating systems • CEP is a tool to detect so called Complex Events e.g. Economic crisis is approaching Complex Event Detection Even1 Constraint Checker Figure 1: Illustrative Example of CEP Deployment 4 □ ► 43 * 4 1 -00.0 October 9, 2014 Complex Event Processing Motivation and Usage Stock Customers Broker 2 Analysis tools Complex Event Detection Event Constraint Checker Figure 1: Illustrative Example of CEP Deployment Filip Nguyen • Event: An object that is a record of an activity in a system. • Figure shows a real-world concept Stock Market. From IT perspective it is a number of communicating systems • CEP is a tool to detect so called Complex Events e.g. Economic crisis is approaching • Using temporal operators • Dealing with large amounts of events • Processing of events on-line October 9, 2014 4 / 22 Complex Event Processing Complex Event Processing Scaling and Performance Why study CEP performance? • In CEP goal of many research papers is to improve performance • DEBS Grand Challenge. Other public CEP benchmarks such as Fi neos What is traditional approach to achieve performance? • Carefully build Event Processing Network • A EPN consists of EPA. The agent might filter events ^> Aggregated CEP Events Map Aggregation Layer Filter Layer Adapter Layer Figure 2: An EPN Filip Nguyen Complex Event Processing October 9, 2014 5/22 Complex Event Processing Ql Semantically Partitioned CEP • Goals of Research • Model • Implementation • Benefits Planned Experimental Datasets Semantically Partitioned CEP Goals of Research Research Problems How to horizontally scale CEP? • Devise algorithms • Develop a Java based ecosystem for distributed CEP 1 -00.0 October 9, 2014 Semantically Partitioned CEP Goals of Research Research Problems How to horizontally scale CEP? • Devise algorithms • Develop a Java based ecosystem for distributed CEP How to capture this formally? « Using a graph representation 1 -00.0 October 9, 2014 Semantically Partitioned CEP Goals of Research arch Problems How to horizontally scale CEP? • Devise algorithms • Develop a Java based ecosystem for distributed CEP How to capture this formally? « Using a graph representation What if an engine wouldn't find every Complex Event? What if we prefer bigger performance to high accuracy? Definition Engine Accuracy A CEP engine A has accuracy Ca £ (0,1). Ca signifies percentage amount of Complex Events out of all Complex Events uncovered by ideal CEP engine. 1 -00.0 October 9, 2014 Semantically Partitioned CEP Goals of Research Research Problems How to horizontally scale CEP? • Devise algorithms • Develop a Java based ecosystem for distributed CEP How to capture this formally? « Using a graph representation What if an engine wouldn't find every Complex Event? What if we prefer bigger performance to high accuracy? Definition Engine Accuracy A CEP engine A has accuracy Ca £ (0,1). Ca signifies percentage amount of Complex Events out of all Complex Events uncovered by ideal CEP engine. How much is the accuracy/performance trade-off usable in today's EP problems? Filip Nguyen Complex Event Processing October 9, 2014 7/22 Semantically Partitioned CEP Model Peer to Peer Horizontal Scaling Trend in today's middleware is to leave centralized architecture of a solution and instead use decentralized peer-to-peer horizontal scaling. • Databases: NoSQL Databases - e.g. Apache Cassandra • Web Servers:Webserver session replication • Communication Middleware:Messaging systems such as JBoss HornetQ • Caching Middleware: in-memory caches like JBoss Data Grid 1 -00.0 October 9, 2014 Semantically Partitioned CEP Mode Theoretical Model • I remove the concept of centralized CEP engine • Graph node is so called Peer (producer, consumer and CEP engine) e.g. for a Stock Market server to become a peer it would need to have my Java daemon deployed Figure 3: Peers Exchanging Events _ 1 -00.0 October 9, 2014 Semantically Partitioned CEP Mode Theoretical Model • I remove the concept of centralized CEP engine • Graph node is so called Peer (producer, consumer and CEP engine) e.g. for a Stock Market server to become a peer it would need to have my Java daemon deployed • During processing of events, edges are dynamically added and removed Figure 3: Peers Exchanging Events _ 1 -00.0 October 9, 2014 Semantically Partitioned CEP Model Semantic Partitioning a The strategy that dynamically adds and removes oriented edges in the graph • In effect it is a strategy to disseminate events between sets of peers • Monte Carlo strategy, CEP Based strategy 1 -00.0 October 9, 2014 Semantically Partitioned CEP Model ulation Results • Matching accuracy and memory consumption was monitored • Monte Carlo performs better in regards to semantic power but suffers from memory consumption 100 2 50 - 800 CEPl -e- CEP2 -•-Montel 600 ^Monte2 >> 0 |400 200 - 1/ 0 - L J 0 5 10 15 20 Time 4 □ ►