Reproducibility in Management Science

J 2024

Reproducibility in Management Science

FIŠAR, Miloš, Ben GREINER, Christoph HUBER, Elena KATOK, Ali OZKES et. al.

Základní údaje

Originální název

Reproducibility in Management Science

Autoři

FIŠAR, Miloš (203 Česká republika, garant, domácí), Ben GREINER, Christoph HUBER (40 Rakousko), Elena KATOK (840 Spojené státy), Ali OZKES a Management Science REPRODUCIBILITY COLLABORATION

Vydání

MANAGEMENT SCIENCE, UNITED STATES, INFORMS, 2024, 0025-1909

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

50200 5.2 Economics and Business

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Odkazy

Article

Impakt faktor

Impact factor: 5.400 v roce 2022

Organizační jednotka

Ekonomicko-správní fakulta

DOI

http://dx.doi.org/10.1287/mnsc.2023.03556

UT WoS

001132639700001

Klíčová slova anglicky

reproducibility; replication; crowd science

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 27. 3. 2024 14:55, Mgr. Pavlína Kurková

Anotace

V originále

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95 % of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29 % of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68 %. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12\% of articles voluntarily provided replication materials, out of which 55 % could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness.

Citovat

FIŠAR, Miloš, Ben GREINER, Christoph HUBER, Elena KATOK, Ali OZKES a Management Science REPRODUCIBILITY COLLABORATION. Reproducibility in Management Science. MANAGEMENT SCIENCE. UNITED STATES: INFORMS, 2024, roč. 70, č. 3, s. 1343-1356. ISSN 0025-1909. Dostupné z: https://dx.doi.org/10.1287/mnsc.2023.03556.

@article{2346526,
   author = {Fišar, Miloš and Greiner, Ben and Huber, Christoph and Katok, Elena and Ozkes, Ali and Reproducibility Collaboration, Management Science},
   article_location = {UNITED STATES},
   article_number = {3},
   doi = {http://dx.doi.org/10.1287/mnsc.2023.03556},
   keywords = {reproducibility; replication; crowd science},
   language = {eng},
   issn = {0025-1909},
   journal = {MANAGEMENT SCIENCE},
   title = {Reproducibility in Management Science},
   url = {https://pubsonline.informs.org/doi/full/10.1287/mnsc.2023.03556},
   volume = {70},
   year = {2024}
}

TY  - JOUR
ID  - 2346526
AU  - Fišar, Miloš - Greiner, Ben - Huber, Christoph - Katok, Elena - Ozkes, Ali - Reproducibility Collaboration, Management Science
PY  - 2024
TI  - Reproducibility in Management Science
JF  - MANAGEMENT SCIENCE
VL  - 70
IS  - 3
SP  - 1343-1356
EP  - 1343-1356
PB  - INFORMS
SN  - 00251909
KW  - reproducibility
KW  - replication
KW  - crowd science
UR  - https://pubsonline.informs.org/doi/full/10.1287/mnsc.2023.03556
N2  - With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95 % of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29 % of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68 %. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12\% of articles voluntarily provided replication materials, out of which 55 % could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness.
ER  -

FIŠAR, Miloš, Ben GREINER, Christoph HUBER, Elena KATOK, Ali OZKES a Management Science REPRODUCIBILITY COLLABORATION. Reproducibility in Management Science. \textit{MANAGEMENT SCIENCE}. UNITED STATES: INFORMS, 2024, roč.~70, č.~3, s.~1343-1356. ISSN~0025-1909. Dostupné z: https://dx.doi.org/10.1287/mnsc.2023.03556.

Podrobný výpis o publikaci