J 2024

A methodology for comparing optimization algorithms for auto-tuning

FLORIS-JAN, Willemsen; Richard SCHOONHOVEN; Jiří FILIPOVIČ; Jacob O. TØRRING; Rob van NIEUWPOORT et. al.

Základní údaje

Originální název

A methodology for comparing optimization algorithms for auto-tuning

Autoři

FLORIS-JAN, Willemsen (528 Nizozemské království); Richard SCHOONHOVEN (528 Nizozemské království); Jiří FILIPOVIČ (203 Česká republika, garant, domácí); Jacob O. TØRRING (578 Norsko); Rob van NIEUWPOORT (528 Nizozemské království) a Ben van WERKHOVEN (528 Nizozemské království)

Vydání

Future Generation Computer Systems, Amsterdam, The Netherlands, Elsevier Science, 2024, 0167-739X

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Nizozemské království

Utajení

není předmětem státního či obchodního tajemství

Odkazy

Impakt faktor

Impact factor: 6.200 v roce 2023

Kód RIV

RIV/00216224:14610/24:00137323

Organizační jednotka

Ústav výpočetní techniky

UT WoS

001249701400002

EID Scopus

2-s2.0-85193970927

Klíčová slova anglicky

Auto-tuning; Methodology; Optimization algorithms; Performance optimization; Performance comparison

Štítky

Příznaky

Mezinárodní význam, Recenzováno
Změněno: 4. 4. 2025 13:11, Mgr. Eva Špillingová

Anotace

V originále

Adapting applications to optimally utilize available hardware is no mean feat: the plethora of choices for optimization techniques are infeasible to maximize manually. To this end, auto-tuning frameworks are used to automate this task, which in turn use optimization algorithms to efficiently search the vast searchspaces. However, there is a lack of comparability in studies presenting advances in auto-tuning frameworks and the optimization algorithms incorporated. As each publication varies in the way experiments are conducted, metrics used, and results reported, comparing the performance of optimization algorithms among publications is infeasible. The auto-tuning community identified this as a key challenge at the 2022 Lorentz Center workshop on auto-tuning. The examination of the current state of the practice in this paper further underlines this. We propose a community-driven methodology composed of four steps regarding experimental setup, tuning budget, dealing with stochasticity, and quantifying performance. This methodology builds upon similar methodologies in other fields while taking into account the constraints and specific characteristics of the auto-tuning field, resulting in novel techniques. The methodology is demonstrated in a simple case study that compares the performance of several optimization algorithms used to auto-tune CUDA kernels on a set of modern GPUs. We provide a software tool to make the application of the methodology easy for authors, and simplifies reproducibility of results.