Detailed Information on Publication Record
2016
Simulations and study of a new scheduling approach for distributed data production
MAKATUN, Dzmitry, Jerome LAURET, Hana RUDOVÁ and Michal ŠUMBERABasic information
Original name
Simulations and study of a new scheduling approach for distributed data production
Authors
MAKATUN, Dzmitry (112 Belarus), Jerome LAURET (840 United States of America), Hana RUDOVÁ (203 Czech Republic, guarantor, belonging to the institution) and Michal ŠUMBERA (203 Czech Republic)
Edition
United Kingdom, Journal of Physics: Conference Series, vol. 762, p. 1-7, 7 pp. 2016
Publisher
Institute of Physics Publishing
Other information
Language
English
Type of outcome
Stať ve sborníku
Field of Study
10201 Computer sciences, information science, bioinformatics
Country of publisher
United Kingdom of Great Britain and Northern Ireland
Confidentiality degree
není předmětem státního či obchodního tajemství
Publication form
printed version "print"
RIV identification code
RIV/00216224:14330/16:00088810
Organization unit
Faculty of Informatics
ISSN
UT WoS
000439689600023
Keywords in English
data transfer planning; distributed data processing; Grid; network flows; data production
Tags
International impact, Reviewed
Změněno: 14/5/2020 15:33, RNDr. Pavel Šmerk, Ph.D.
Abstract
V originále
Distributed data processing has found its application in many fields of science (High Energy and Nuclear Physics (HENP), astronomy, biology to name only those). We have focused our research on distributed data production, an essential part of computations in HENP. Using our previous experience, we have recently proposed a new scheduling approach for distributed data production which is based on the network flow maximization model. It has a polynomial complexity providing required scalability with respect to the size of computations. Our approach improves the overall data production throughput due to three factors: transfer input files in advance before their processing (allows to decrease I/O latency); balancing of the network traffic (includes splitting the load between several alternative transfer paths); and transfer files sequentially in a coordinated manner (allows to reduce the influence of possible network bottlenecks). In this contribution, we present the results of our new simulations based on the GridSim framework which is one of the commonly used tools in the field of distributed computations. In these simulations we study the behavior of standard scheduling approaches compared to our recently proposed approach in a realistic environment relying on the data from the STAR and ATLAS experiments and considering the influence of the background traffic. The final goal of the research is to integrate the proposed scheduling approach into the real data production framework. In order to achieve this we are constantly moving our simulations towards real use cases, study scalability of the model and the influence of the scheduling parameters on the quality of the solution.
Links
| GAP202/12/0306, research and development project |
|