Peer-to-peer Cooperative Scheduling Architecture for National Grid Infrastructure

MATYSKA, Luděk; Miroslav RUDA a Šimon TÓTH. Peer-to-peer Cooperative Scheduling Architecture for National Grid Infrastructure. In Simon C.Lin, Eric Yen. Data Driven e-Science. Academia Sinica Grid Computing Centre, 2011, s. 105-118. ISBN 978-1-4419-8014-4.

Základní údaje

Originální název

Peer-to-peer Cooperative Scheduling Architecture for National Grid Infrastructure

Autoři

MATYSKA, Luděk; Miroslav RUDA a Šimon TÓTH

Vydání

Data Driven e-Science, od s. 105-118, 2011

Nakladatel

Academia Sinica Grid Computing Centre

---

Další údaje

Typ výsledku

Stať ve sborníku

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Označené pro přenos do RIV

Ne

Organizační jednotka

Fakulta informatiky

ISBN

978-1-4419-8014-4

UT WoS

000310340000008

Příznaky

Mezinárodní význam, Recenzováno

---

Anotace

Anglicky

For some ten years, the Czech National Grid Infrastructure MetaCentrum uses a single central PBSPro installation to schedule jobs across the country. This centralized approach keeps a full track about all the clusters, providing support for jobs spanning several sites, implementation for the fair-share policy and better overall control of the grid environment. Despite a steady progress in the increased stability and resilience to intermittent very short network failures, growing number of sites and processors makes this architecture, with a single point of failure and scalability limits, obsolete. As a result, a new scheduling architecture is proposed, which relies on higher autonomy of clusters. It is based on a peer to peer network of semi-independent schedulers for each site or even cluster. Each scheduler accepts jobs for the whole infrastructure, cooperating with other schedulers on implementation of global policies like central job accounting, fair-share, or submission of jobs across several sites. The scheduling system is integrated with the Magrathea system to support scheduling of virtual clusters, including the setup of their internal network, again eventually spanning several sites. On the other hand, each scheduler is local to one of several clusters and is able to directly control and submit jobs to them even if the connection of other scheduling peers is lost. In parallel to the change of the overall architecture, the scheduling system itself is being replaced. Instead of PBSPro, chosen originally for its declared support of large scale distributed environment, the new scheduling architecture is based on the open-source Torque system. The implementation and support for the most desired properties in PBSPro and Torque are discussed and the necessary modifications to Torque to support the MetaCentrum scheduling architecture are presented, too.