Towards a Benchmarking Suite for Kernel Tuners

D 2023

Towards a Benchmarking Suite for Kernel Tuners

TØRRING, Jacob O, van Werkhoven BEN, Filip PETROVIČ, Floris-Jan WILLEMSEN, Jiří FILIPOVIČ et. al.

Základní údaje

Originální název

Towards a Benchmarking Suite for Kernel Tuners

Autoři

TØRRING, Jacob O (578 Norsko), van Werkhoven BEN (528 Nizozemské království), Filip PETROVIČ (703 Slovensko, domácí), Floris-Jan WILLEMSEN (528 Nizozemské království), Jiří FILIPOVIČ (203 Česká republika, garant, domácí) a Anne C ELSTER (578 Norsko)

Vydání

neuveden, 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), od s. 724-733, 10 s. 2023

Nakladatel

IEEE

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

URL

Kód RIV

RIV/00216224:14610/23:00131587

Organizační jednotka

Ústav výpočetní techniky

ISBN

979-8-3503-1199-0

ISSN

DOI

http://dx.doi.org/10.1109/IPDPSW59300.2023.00124

UT WoS

001055030700088

Klíčová slova anglicky

autotuning; benchmarking

Štítky

rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 20. 3. 2024 14:59, Mgr. Alena Mokrá

Anotace

V originále

As computing system become more complex combining CPUs and GPUs, it is becoming harder and harder for programmers to keep their codes optimized as the hardware gets updated. Autotuners try to alleviate this by hiding as many architecture-based optimization details as possible from the end-user, so that the code can be used efficiently across different generations of systems. Several autotuning frameworks have emerged, but a comparative analysis between these related works is scarce, owing to the significant manual effort required to port a tunable kernel from one tuner another. In this article we introduce a new benchmark suite for evaluating the performance of optimization algorithms used by modern autotuners targeting GPUs. The suite contains tunable GPU kernels that are representative of real-world applications, allowing for comparisons between optimization algorithms and the examination of code optimization, search space difficulty, and performance portability. Our framework facilitates easy integration of new autotuners and benchmarks by defining a shared problem interface. Our benchmark suite is evaluated based on five characteristics: convergence rate, local minima centrality, optimal speedup, Permutation Feature Importance (PFI), and performance portability. The results show that optimization parameters greatly impact performance and the need for global optimization. The importance of each parameter is consistent across GPU architectures, however, the specific values need to be optimized for each architecture. Our portability study highlights the crucial importance of autotuning each application for a specific target architecture. The results reveal that simply transferring the optimal configuration from one architecture to another can result in a performance ranging from 58.5% to 99.9% of the optimal performance, depending on the GPU architecture. This highlights the importance of autotuning in modern computing systems and the value of our benchmark suite in facilitating the study of optimization algorithms and their effectiveness in achieving optimal performance for specific target architectures.

Návaznosti

LM2023054, projekt VaV

Název: e-Infrastruktura CZ

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, e-Infrastruktura CZ

Citovat

TØRRING, Jacob O, van Werkhoven BEN, Filip PETROVIČ, Floris-Jan WILLEMSEN, Jiří FILIPOVIČ a Anne C ELSTER. Towards a Benchmarking Suite for Kernel Tuners. Online. In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). neuveden: IEEE, 2023, s. 724-733. ISBN 979-8-3503-1199-0. Dostupné z: https://dx.doi.org/10.1109/IPDPSW59300.2023.00124.

@inproceedings{2306557,
   author = {Tørring, Jacob O and Ben, van Werkhoven and Petrovič, Filip and Willemsen, FlorisandJan and Filipovič, Jiří and Elster, Anne C},
   address = {neuveden},
   booktitle = {2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)},
   doi = {http://dx.doi.org/10.1109/IPDPSW59300.2023.00124},
   keywords = {autotuning; benchmarking},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {neuveden},
   isbn = {979-8-3503-1199-0},
   pages = {724-733},
   publisher = {IEEE},
   title = {Towards a Benchmarking Suite for Kernel Tuners},
   url = {https://ieeexplore.ieee.org/abstract/document/10196663},
   year = {2023}
}

TY  - JOUR
ID  - 2306557
AU  - Tørring, Jacob O - Ben, van Werkhoven - Petrovič, Filip - Willemsen, Floris-Jan - Filipovič, Jiří - Elster, Anne C
PY  - 2023
TI  - Towards a Benchmarking Suite for Kernel Tuners
PB  - IEEE
CY  - neuveden
SN  - 9798350311990
KW  - autotuning
KW  - benchmarking
UR  - https://ieeexplore.ieee.org/abstract/document/10196663
N2  - As computing system become more complex combining CPUs and GPUs, it is becoming harder and harder for programmers to keep their codes optimized as the hardware gets updated. Autotuners try to alleviate this by hiding as many architecture-based optimization details as possible from the end-user, so that the code can be used efficiently across different generations of systems. Several autotuning frameworks have emerged, but a comparative analysis between these related works is scarce, owing to the significant manual effort required to port a tunable kernel from one tuner another. In this article we introduce a new benchmark suite for evaluating the performance of optimization algorithms used by modern autotuners targeting GPUs. The suite contains tunable GPU kernels that are representative of real-world applications, allowing for comparisons between optimization algorithms and the examination of code optimization, search space difficulty, and performance portability. Our framework facilitates easy integration of new autotuners and benchmarks by defining a shared problem interface. Our benchmark suite is evaluated based on five characteristics: convergence rate, local minima centrality, optimal speedup, Permutation Feature Importance (PFI), and performance portability. The results show that optimization parameters greatly impact performance and the need for global optimization. The importance of each parameter is consistent across GPU architectures, however, the specific values need to be optimized for each architecture. Our portability study highlights the crucial importance of autotuning each application for a specific target architecture. The results reveal that simply transferring the optimal configuration from one architecture to another can result in a performance ranging from 58.5% to 99.9% of the optimal performance, depending on the GPU architecture. This highlights the importance of autotuning in modern computing systems and the value of our benchmark suite in facilitating the study of optimization algorithms and their effectiveness in achieving optimal performance for specific target architectures.
ER  -

TØRRING, Jacob O, van Werkhoven BEN, Filip PETROVIČ, Floris-Jan WILLEMSEN, Jiří FILIPOVIČ a Anne C ELSTER. Towards a Benchmarking Suite for Kernel Tuners. Online. In \textit{2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)}. neuveden: IEEE, 2023, s.~724-733. ISBN~979-8-3503-1199-0. Dostupné z: https://dx.doi.org/10.1109/IPDPSW59300.2023.00124.

Podrobný výpis o publikaci