Title:
Comparison of the accuracy of bibliographical references generated for medical citation styles by EndNote, Mendeley, RefWorks and Zotero
Author: Jifi Kratochvil
Affiliation:    Masaryk University Campus Library, Brno, Czech Republic
Keywords:     accuracy, bibliographical references, citation styles, EndNote, medical journals, Mendeley, reference managers, RefWorks, Zotero
Comparison of the accuracy of bibliographical references generated for medical citation styles by EndNote, Mendeley, RefWorks and Zotero
Abstract
Bibliographical references to online and printed articles, books, contributions to edited books and web resources generated by EndNote, Mendeley, RefWorks and Zotero were compared with manually written references according to the citation instructions in 15 biomedical journals and the NLM citation style. The fewest mistakes were detected in references generated by Zotero for 11 journals and the NLM style, while the second fewest number of mistakes was found in Mendeley. The largest number of mistakes for 9 journals was found in references generated by EndNote and in the other 4 journals the largest number of mistakes was detected in RefWorks references. With regard to the individual types of resources, the lowest number of mistakes was shown by Zotero, while RefWorks had the greatest number of mistakes. All programs had problems especially with generating the URL and the date of access in the reference to online documents. It was also found that several mistakes were caused by technical limitations of the reference managers, while other mistakes originated due to incorrect setting of the citation styles. A comparison showed that Zotero and Mendeley are the most suitable managers.
Introduction
Bibliographical references are an integral part of all scientific publications. However, their authors constantly struggle with generating them and repeatedly make mistakes in creating them. These mistakes should not be treated lightly, because they may lead the reader to doubt the quality of the author's research; the author might also have shown the same carelessness in the references in the following parts of the article or in the research itself (Biebuyck, 1992). In the past, some authors have been revealed to have mentioned sources in the list of references which they had not read. Some authors have also brought over incorrectly formatted bibliographical references from different articles (Cronin, 1982; Garfield, 1990; Sweetland, 1989; Wyles, 2004). As a result, not only the reputation of the authors but also the reputation of the journals can suffer when poorly produced articles featuring mistakes are published (Spivey & Wilks, 2004). Mainly the cited authors and the cited journals are affected. Indexing these authors in citation databases can become more complicated or even impossible due to such mistakes (Garfield, 1990).
The causes of mistakes in references can vary. Apart from unethical ways of citing as mentioned, mistakes have also been reported for decades now to have been caused by mere carelessness and the authors' inconsistent writing (Bahar et al., 2012; De Lacey, Record, & Wade, 1985; Lok, Chan, & Martinson, 2001; Oermann, Cummings, & Wilmes, 2001). The blame lies with the journals themselves, because they do not contain very detailed citation instructions for authors. They may also refer to already invalid versions of the citation styles or they do not give any citation styles at all (De Lacey et al., 1985; Onwuegbuzie, Hwang, Combs, & Slate, 2012). Another reason for errors is the existence of too many citation styles where the authors cannot be sure what the best approach is (Liu, 1993; Moorthy, 1988; Park, Mardis, & Ury, 2011).
Standardisation of reference rules could provide a way out of this situation (Freimer & Perry, 1986; Garfield, 1990; Mansfield, 1984; Sweetland, 1989; Terbille, 1990). However, the situation in medical journals shows the reverse. Despite the Uniform Requirements for
1
Manuscripts being in existence for forty years already (ICMJE, 2015; Patrias, 2007), many medical journal editors request that authors follow the editor's own citation rules. No wonder that there is a high percentage of incorrect bibliographical references. For example, in five general surgical journals, such as Annals of Surgery and the British Journal of Surgery, 11% of references published in the July 2004 issues were incorrect. Three journals dealing with pediatric surgery (e.g. the Journal of Pediatric Surgery) had 33.7% incorrect references in the first issues of the year 2001, and the Archives of General Psychiatry and Journal of Clinical Psychiatry had 38.5% incorrect bibliographical references out of 420 randomly selected references published in the September 1980 and 1999 issues. Journals in the nursing field, e.g. the Journal of Pediatric Nursing, had as many as 42.7% incorrect bibliographical references in articles published in issues from the period between September 1999 and February 2000 (Celayir, Sander, & Celayir, 2003; Oermann et al., 2001; Reddy, Srinivas, Sabanayagam, & Balasubramanian, 2008). In the past, on average 23.6% incorrect references were found in such prestigious journals as the New England Journal of Medicine and The Lancet (De Lacey et al., 1985). In all these cases the mistakes were usually in authors' names, the titles of articles and journals, or incorrect information concerning the year of the journal or the pagination.
Under these circumstances, a growing interest in reference managers is understandable. Reference managers help to administer bibliographical records, text and picture files and above all they assist in inserting references into the text that are formatted in compliance with various reference styles (Zhang, 2012). A number of studies comparing the functions of the EndNote, Mendeley, RefWorks and Zotero reference managers have been published recently. However, these studies, a summary of which in connection with the results of this study can be found in the Discussion section, focused only on comparing technical aspects of the reference managers, not on comparing the accuracy of the bibliographical references generated from within these programs. Only a few studies have focused on such comparisons (Homol, 2014).
Kessler and Van Ullen (2005) compared 100 references produced in accordance with the APA style in the EasyBib, EndNote and NoodleBib programs, and found that the three applications generated 106 mistakes altogether. EndNote had the fewest mistakes in references to print publications while NoodleBib revealed the fewest mistakes in references to electronic publications. Brahmi and Gall (2006) focused their study on the quality of bibliographical references for 43 most prestigious medical journals created in EndNote and Reference Manager. They found that these applications were not able to generate references for 35-47% of the journals and the references which were generated differed in 33-43% of the cases from the recommended style. The bibliographical references made in the style of the remaining journals contained 33-46% of differences. Gilmour and Cobus-Kuo (2011) tested the CiteULike, Mendeley, RefWorks and Zotero managers for the ACS, AMA, APA, IEEE and Nature citation styles and they discovered that RefWorks shows the lowest error level in terms of the average number of mistakes. Homol (2014) compared the output from the EndNote, Basic, RefWorks and Zotero applications based on the APA and MLA citation styles with the references published in the EBSCO Discovery Service. She found that none of the programs generated faultless bibliographical references. RefWorks made the fewest mistakes for the APA style and EndNote Basic made the fewest mistakes for the MLA style.
None of these studies simultaneously compared the output from EndNote, Mendeley, RefWorks and Zotero, which are presently the applications most often tested. It is also necessary to verify the quality of bibliographical references to different types of documents, i.e. not only journal articles (Homol, 2014). Therefore, the aim of the study is to determine which of the reference managers generates the lowest number of mistakes for medical journals'
2
bibliographical references. The focus on medical journals is due not only to the author's role in a university library, namely providing services for the Faculty of Medicine and their employees in the faculty hospitals, but also this focus was chosen with the aim of providing a more detailed analysis of the situation for medical authors and medical librarians. This study will help medical authors to better decide which reference manager to use. This article can also guide medical librarians when choosing which reference manager(s) to stress in their information literacy classes. Last but not least, the aim of the article is to encourage librarians to perform further similar analyses of the quality of the input from reference managers for journals from other scientific fields.
Method
Between December 2015 and January 2016 the quality of bibliographical references generated for 15 medical journals and the National Library of Medicine (NLM) citation style by the reference managers EndNote (version X7.4, Bid 8818), Mendeley (version 1.15.2), RefWorks (version 4.4.1376) and Zotero (version 4.0.28.10) were compared. The journals (see Table 1) were chosen in the following way: the ten medical journals with the highest number of published articles in Journal Citations Reports in 2014 were added to the ten medical journals with the highest impact factor in Journal Citations Reports in 2014. These journals were chosen either because of the need for the medical authors to publish their articles in the most-referred journals or because the journals publish a high number of articles and therefore a large number of authors work with these journals' reference instructions. From these 20 journals, the following 5 titles were then excluded: CA: A Cancer Journal for Clinicians, Frontiers in Human Neuroscience, Molecular Medicine Reports, Oncology Letters and Oncotarget. This was done because in some citation managers it was not possible to generate the bibliographical references for these journals. The NLM style was added to these 15 journals for comparison. The format and how to adhere to NLM was agreed on by the editors of medical journals (ICMJE, 2015; Patrias, 2007). In this way, a list of journals and the NLM reference styles was created. The authors of medical journals very often follow these instructions.
Once the choice of the journals and the reference styles was made, the publications representing the commonly-mentioned types of print or online resources were chosen. As Homol (2014) pointed out, an analysis of other sources is needed. Although journal articles are and will probably continue to be the most frequently cited type of resources, various studies show that 16.5% of 81,834 references published in ten medical journals were to sources other than an article (Barrett, Helwig, & Neves, 2016; Delwiche, 2013; Rethlefsen & Aldrich, 2013). Therefore, references for various types of resources were tested in this study. These types were a journal article, a contribution to an edited book, a book, an edited book and a web resource. In addition, in the case of a journal article, a contribution to an edited book and a book, publications with different numbers of authors were also chosen. In the case of a journal article, a contribution to an edited book and a book, publications were chosen with different numbers of authors so that it could be verified whether the reference manager is able to generate the number of authors' names determined by the journal style. In this way, 17 publications (Table 2) were chosen. Bibliographical references were manually created according to the citation instructions of the NLM style and the example of all 15 journals. The reference instructions were found in the instructions for authors accessible on the websites of particular journals. These instructions for authors were the primary source of examples and were strictly followed. For example, according to the instructions of The Lancet a book title was set in capital
3
case and a journal title in italics, while according to the NLM style a book title was set in sentence case and a journal title without any changes. If the instructions did not contain a citation instruction or an example bibliographical reference for some type of resource, a citation of the given resource published in the relevant year of the particular journal was used. For example, reference 37 in the article "Tumor Regression After Brachytherapy for Choroidal Melanoma" served as an example of a bibliographical reference to a contribution to an edited book, because Investigative Ophthalmology & Visual Science does not provide any example in its citation instructions (ARVO, 2015; Rashid, Heikkonen, & Kivela, 2015). In this way, 17 examples of bibliographical references were made for individual journals and the NLM citation style.
After the examples of bibliographical references were created, manually-created records for the same publications were made in each of the reference managers. These were made manually so that they would contain all data in particular fields of records and so that mistakes due to importing records from different sources could be avoided (Basak, 2014; Kessler & Van Ullen, 2005). Following this, bibliographical references were generated from each reference manager using the citation formats for particular journals. The bibliographical references were compared with the examples created manually (Fig. 1 shows an example of such comparison).
Exemplar
BezuidenhoutD, Williams DF, Zilla P. Polymeric he art valves for surreal implantation, catheter-based technologies and he art assist devices. Biomaterials [Internet]. 2015 [cited 2015 Dec 2 2 ] ,3 : 6 —2 5. Available from: http://www.sciencedirect.com/science/article/pii/SO 142961214010114
EndNote
BezuidenhoutD, Williams DF, Zilla P. Polymeric he art valves for surreal implantation, catheter-based te chnolo gie s and he art a ssist devic e s. B iom aterials [I ntemet] .20152015/12/22;36":[6-25pp.]. Available from: http://www.sciencedirect.com/science/article/pii/30142961214010114.
Errors detected
Data: 2 errors (Incorrect form of the date of a citation, additional abbreviation of pages (pp.)) Punctuation: 3 errors (missing brackets in the date of a citation, additional brackets in pagination, additional period after the website address) Form atting none
Zotero
BezuidenhoutD, Williams DF, Zilla P. Polymeric he art valves for surgical implantation, catheter-based technologies and he art assist devices. Biomaterials [Internet]. 2015 [cited 2015 Dec 2 2 ] ,3 6:6 —2 5. Available from: http://www.sciencedirect.com/science/article/pii/SO 142961214010114
Errors detected Data: none Punctuation: none Formatting: none
Figure 1 - Comparison of a manually created example bibliographic reference to an online article with 3 authors or less with references generated from EndNote and Zotero.
In the same way as in the previous studies (Brahmi & Gall, 2006; Gilmour & Cobus-Kuo, 2011; Homol, 2014; Kessler & Van Ullen, 2005), different types of mistakes for individual
4
resource types were detected. Similarly to the Homol's study (2014) we divided the errors found in references into three categories. The first category comprised mistakes in data where data was missing or redundant (e.g. the rule for the number of authors allowed in references was not followed) or the data was incorrectly given (e.g. the wrong abbreviation for the word "editors" appeared). The second category comprised mistakes in the punctuation, such as missing or redundant punctuation, a missing or redundant space or a symbol. The third category consists of mistakes in formatting the data in references (wrong use of italics or bold type). After dividing the errors into categories, the number of mistakes in each of the categories for each manager were totalled for particular journals and particular kinds of resources. At the same time, the occurrence of mistakes in each of the categories for the reference managers was counted.
5
	EndNote					Mendeley					RefWorks					Zotero				
Journal Title	Data	Punctuation	Formatting	Average	Total errors	Data	Punctuation	Formatting	Average	Total errors	Data	Punctuation	Formatting	Average	Total errors	Data	Punctuation	Formatting	Average	Total errors
Annual Review of Immunology	34	13	4	3	51	27	5	2	2	34	25	15	4	2,6	44	44	0	5	2,9	49
Anticancer Research	27	19	7	3,1	53	14	12	4	1,8	30	28	22	4	3,2	54	16	21	4	2,4	41
Blood	35	3	1	2,3	39	48	0	4	3,1	52	40	4	4	2,8	48	46	4	4	3,2	54
BMC Cancer	47	34	30	6,9	111	42	34	29	6,6	105	45	37	21	6,4	103	42	25	25	5,8	92
Investigative ophthalmology	33	16	4	3,1	53	23	12	4	2,3	39	64	8	5	4,5	77	10	0	0	0,6	10
JAMA	26	4	3	1,9	33	12	0	0	0,7	12	23	1	0	1,4	24	7	0	0	0,4	7
Journal of Clinical Endocrinology	26	7	16	2,9	49	21	2	18	2,4	41	45	30	33	6,4	108	14	6	18	2,2	38
Lancet	34	14	0	2,8	48	28	10	5	2,5	43	29	18	0	2,8	47	13	9	0	1,3	22
Nature Medicine	45	12	3	3,5	60	29	12	0	2,4	41	44	9	2	3,2	55	27	6	0	1,9	33
Nature Reviews Cancer	45	12	3	3,5	60	29	12	0	2,4	41	44	9	2	3,2	55	27	6	0	1,9	33
Nature Reviews Immunology	45	12	3	3,5	60	29	12	0	2,4	41	44	9	2	3,2	55	27	6	0	1,9	33
Nature Reviews Neuroscience	45	12	3	3,5	60	29	12	0	2,4	41	44	9	2	3,2	55	27	6	0	1,9	33
Nature Reviews Drug Discovery	45	12	3	3,5	60	29	12	0	2,4	41	44	9	2	3,2	55	27	6	0	1,9	33
New England Journal of Medicine	34	16	0	2,9	50	44	20	0	3,8	64	43	8	0	3	51	42	15	0	3,4	57
NLM style	44	4	0	2,8	48	15	0	0	0,9	15	19	15	0	2	34	6	0	0	0,4	6
Vaccine	44	3	0	2,8	47	39	0	0	2,3	39	45	27	0	4,2	72	34	0	0	2	34
TOTAL	609	193	80	52,3	882	458	155	66	40,3	679	626	230	81	55,5	937	409	110	56	34,2	575
Table 1 - Number of mistakes in bibliographical references for medical journals (Data - the number of errors in data, Punctuation - the number of
errors in punctuation, Formatting - number of errors in formatting data, Average - the average number of errors in a reference)
	EndNote					Mendeley					RefWorks					Zotero				
Type of a document	Data	Punct uatio n	Formatting	Average	Total errors	Data	Punctuation	Formatting	Average	Total errors	Data	Punctuation	Formatting	Average	Total errors	Data	Punctuatio n	Formatting	Average	Total errors
Online article with 3 or less authors	36	12	5	3,5	53	22	12	6	2,7	40	27	17	5	3,3	49	33	4	4	2,7	41
Online article with 7 or more authors	62	14	6	5,5	82	25	12	6	2,9	43	30	18	5	3,5	53	36	4	5	3	45
Online book with 1 author	55	6	9	4,7	70	32	9	2	2,9	43	47	13	3	4,2	63	28	9	2	2,6	39
Online book with 2 authors	55	6	9	4,7	70	32	9	2	2,9	43	42	13	3	3,9	58	28	9	2	2,6	39
Online book with 3 authors	58	6	9	4,9	73	32	9	3	2,9	44	49	13	3	4,3	65	29	9	2	2,7	40
Online edited book	49	9	2	4	60	54	3	2	3,9	59	73	15	9	6,5	97	31	4	2	2,5	37
Web resource	80	12	2	6,3	94	63	5	3	4,7	71	66	13	2	5,4	81	50	7	3	4	60
Contribution to an online book with 3 or less authors	44	15	4	4,2	63	49	13	6	4,5	68	57	15	5	5,1	77	36	12	5	3,5	53
Contribution to an online book with 3 or more authors	44	15	4	4,2	63	50	13	5	4,5	68	56	15	5	5,1	76	37	11	5	3,5	53
Contribution to a printed book with 3 or less authors	12	13	4	1,9	29	22	13	6	2,7	41	31	14	5	3,3	50	27	7	5	2,6	39
Contribution to a printed book with 3 or more authors	12	13	4	1,9	29	23	13	5	2,7	41	30	14	5	3,3	49	28	7	4	2,6	39
Printed book with 1 author	24	9	3	2,4	36	9	9	2	1,3	20	22	7	4	2,2	33	8	5	2	1	15
Printed book with 2 authors	24	9	3	2,4	36	9	9	2	1,3	20	22	7	4	2,2	33	8	5	2	1	15
Printed book with 3 authors	24	9	3	2,4	36	9	9	2	1,3	20	22	7	4	2,2	33	8	5	2	1	15
Printed article with 3 or less authors	0	19	5	1,6	24	0	7	6	0,9	13	1	17	5	1,5	23	3	4	4	0,7	11
Printed article with 7 or more authors	7	17	6	2	30	2	7	6	1	15	4	18	5	1,8	27	7	4	5	1,1	16
Printed edited book	23	9	2	2,3	34	25	3	2	2	30	47	14	9	4,7	70	12	4	2	1,2	18
TOTAL	609	193	80	58,8	882	458	155	66	45,3	679	626	230	81	62,5	937	409	110	56	38,3	575
Table 2 - Number of mistakes in bibliographica the number of errors in punctuation, Formatting -
references according to the type of cited document (Data - the number of errors in data, Punctuation number of errors in formatting data, Average - the average number of errors in a reference)
Results
In this study the bibliographical references to a web resource were compared with those to print and online versions of a journal article, a book, an edited book and a contribution to it (Table 2). These references were made using the EndNote, Mendeley, RefWorks and Zotero reference managers for 15 medical journals and the NLM style (Table 1). For each of the journals, 17 bibliographical references were compared with the manually-produced example. The only exception was Anticancer Research. In its reference instructions and randomly chosen articles, no example of a bibliographical reference to a web resource was found. Therefore, 16 bibliographical references were compared there. A check of the accuracy of bibliographical references generated from the reference managers was made for 1,084 bibliographical references.
From Tables 1-2 and Fig. 2, it is clear that none of the four tested reference managers were able to correctly form all bibliographical references for any of the journals. The lowest number of mistakes (575) was found for the Zotero reference manager. It generated the fewest mistakes for the NLM reference style and 11 out of 15 journals (BMC Cancer, Investigative Ophthalmology & Visual Science, JAMA, Journal of Clinical Endocrinology, The Lancet, all Nature journals, Vaccine). The second lowest number of errors (679) was found in Mendeley, which had the lowest number of mistakes compared to the other three reference managers in the references for Annual Review of Immunology and Anticancer Research. After Zotero it showed the least number of mistakes for a further 9 journals and the NLM style. EndNote had the third lowest number of mistakes (882) and the fewest mistakes of all applications for Blood and the New England Journal of Medicine, but at the same time it showed the most mistakes for the biggest number of journals including the NLM style. RefWorks generated the most mistakes in references (937), but compared to EndNote it had the most mistakes in references for three journals. Regarding the total number of mistakes and the total number of journals for which the programs made the least mistakes, the best-quality reference output came from the program Zotero.
400 350
I n fl inL inlln n» InHn
4?
I EndNote   □ Mendeley   0 RefWorks   D Zotero
Figure 2 - Number of errors in bibliographic references according to the type of error
8
The Zotero program also achieved the best results in terms of the number of bibliographical references based on the type of cited resources (Table 2). Zotero generated the fewest mistakes in bibliographical references for 12 types of resources and it had the second lowest number of mistakes for the remaining five types of resources (both references to an online article, both references to a contribution to a print edited book and a reference to a print article with 7 or more authors). Here the Mendeley reference manager also produced the second lowest number of mistakes. It had the lowest number of mistakes in references to three types of resources and the second lowest number of mistakes in references for 10 out of 17 types of resources. On the other hand, the bibliographical references made by EndNote and RefWorks revealed the most mistakes for most sources. While EndNote made most mistakes in references for 11 out of 17 types of sources, RefWorks had the most errors in references for 6 types of sources. However, it is necessary to add that EndNote was the only reference manager with the least mistakes for both references to a contribution to a print edited book. EndNote also came in second place because it had the least number of mistakes for both references to a contribution to an online edited book. Overall, the Zotero program showed the best-quality output regarding the type of cited sources.
With regards to the error categories, it was found that RefWorks (99 mistakes) and EndNote (93) produced the most mistakes in references (Table 3), while Mendeley and Zotero both produced 70 kinds of mistakes in references. All reference managers produced mistakes predominantly in bibliographical data and RefWorks also had a markedly higher number of mistakes in punctuation and gaps in references. In connection with the frequency of the occurrence of individual mistakes (Fig. 2), these were mostly missing data in bibliographical references, incorrect structure for the bibliographic data or superfluous data. To a lesser extent there were mistakes in punctuation or incorrect formatting of some text (e.g. bold, italic) in the reference. All four reference managers had problems with generating the URL and the date of access in references to online resources. EndNote did not generate the URL for 120 references out of 135 references to online resources. For 63 references it did not generate the date of access. For 84 references RefWorks did not write the URL and for 68 the date of access. For 56 references Mendeley did not write the URL and for 61 the date of access. Zotero had the least number of such mistakes. For 34 references it did not generate the URL and the date of access. The reference managers also showed further problems with references to online resources. For example, EndNote generated the date of access in an incorrect form (the month was expressed as a number rather than the month name) and like RefWorks it did not generate information on the type of resource. Mendeley showed a mistake in the URL ("available from" instead "at") and in addition it unnecessarily generated the information on the type of resource. Zotero placed the date of access in an incorrect place in the reference.
Reference	Number of errors	Number of errors in the punctuation	Number of errors	Total
manager	in data		in formatting data	
EndNote	49	29	15	93
Mendeley	39	18	13	70
RefWorks	49	36	14	99
Zotero	42	15	13	70
Table 3 - Number of types of errors in particular categories
9
All four programs also had problems with generating the data on the particular edition in references to print and online books, that is, sometimes they did not generate the information at all. EndNote did not produce the information on the edition in 51 instances out of 90 references to print and online books. The other three managers do so for only 36 of them. EndNote then produced an incorrect abbreviation for the word edition" or it gave a cardinal number rather than an ordinal number (e.g. "2 edition" instead of "2nd ed.") in 42 references. In other cases, particular mistakes occurred only in some of the reference managers. For example, Mendeley was not able to generate the year of copyright (e.g. "c2009") because of the letter "c" before the date. Also, some individual reference managers generated superfluous data; EndNote added unnecessary information on the pagination in 72 references and extra information on the place of publication in 51 references. RefWorks added extra information on the pagination in 39 references and superfluous information on the place of publishing in 36 references. Zotero gave superfluous information on the pagination in 10 references and gave the wrong data on editors (e.g. "Edited by" instead of "editors" in the references for BMC Cancer) in 20 references.
In the category of mistakes concerning punctuation and whitespaces in references, all four managers made similar mistakes in punctuation after the name of the editor, the title of the journal and the year of publishing. The punctuation after this data was either not generated or it was wrong (e.g. a comma instead of a full stop). In other cases there were again individual mistakes for individual managers. For example, EndNote, Mendeley and Zotero gave the wrong punctuation after the names of books in some references. EndNote and RefWorks put an extra space before the pagination in some references to journal articles, and RefWorks did not put a bracket before the date of access to online resources.
In the category of errors in formatting the data in bibliographical references, all four reference managers usually showed the document titles wrongly, whether in italics or bold type. For example, EndNote, Mendeley and RefWorks, despite the reference instructions of BMC Cancer stipulating otherwise, generated the titles of contributions to an edited book and the titles of books, journals and edited books in bold type in references to online and print resources. Mendeley, RefWorks and Zotero also mistakenly generated the data on authors in bold type. This was found in references for the Journal of Clinical Endocrinology. In addition, EndNote did not give the names of books in italics in references to online books while it did so in print publications.
Discussion
In agreement with the previous studies (Brahmi & Gall, 2006; Homol, 2014; Kessler & Van Ullen, 2005) analysing the quality of the bibliographical references from reference managers, the results of this research show that none of the reference managers produces all bibliographical references without mistakes. The Zotero reference manager achieved the best results because it had remarkably excellent results in references for the JAMA journal and the NLM style. Nonetheless, even this reference manager as well as the other programs produced a number of mistakes in the references to online resources. All reference managers made on average 2.3-6.9 mistakes in producing references to online resources. This is a high number considering the usual 9 pieces of data in the references of an article (the author, the article title, the journal title, the date, the volume, the number, the pagination, the URL, the date of access), 10 pieces of data in the references of a contribution to an edited book (the author, the title of the contribution, the editor, the title of the edited book, edition, the place of publication, the publisher, the date, the URL, the date of access) and 8 pieces of data in the references of a book/
10
an edited book (the author/the editor, the title of the book/edited book, edition, the place of publication, the publisher, the date, the URL, and date of access). In this the previous findings were confirmed. This means that the reference managers are not always able to generate the data typical for online resources. Gilmour and Cobus-Kuo (2011) stressed this problem in connection with the CiteULike, Mendeley, RefWorks and Zotero reference managers. They tested the output from these managers for the APA style. Homol (2014) also found this for the APA style as well as the MLA style. She compared the accuracy of bibliographical references from EBSCO Discovery Service, EndNote Basic, RefWorks and Zotero. Kessler and Van Ullen (2005) came to the same findings while testing the EasyBib, EndNote and NoodleBib programs.
These studies put the blame for the mistakes on the reference managers themselves. However, our test revealed that the mistakes are not caused only by the program but the setting of a particular citation style. A check of the generated references showed the following: all four reference managers generated the references to the online resources without the URL and the date of access for the Annual Review of Immunology, but both pieces of data were generated for the NLM style. If the same record of the publication entered into the reference manager is generated incorrectly for one style and it is generated correctly for another style, then the reference mistakes cannot be caused by the reference manager. The fault is in setting the implemented citation styles.
The same applies to the next mistake found in this research. This was an alleged inability of these managers to generate the titles of documents correctly. In the past the RefWorks reference manager was blamed for this mistake (Gilmour & Cobus-Kuo, 2011; Homol, 2014) but our research found that the reason for such mistakes is the incorrect setting of the reference style. This means that in fact, while RefWorks generated proper names with lowercase initial letters for the Annual Review of Immunology and Journal of Clinical Endocrinology, the proper names were generated with capital letters in the references for other journals and the NLM style. The same error was also found in the Mendeley and Zotero reference managers when applied to the Annual Review of Immunology. If the reference managers generated the same record for different journals in different ways, the program itself cannot be at fault. There must be a mistake in the setting of the applied citation style. Likewise, the inability to generate a shortened title of the journal was previously taken as a mistake in the EndNote manager (Brahmi & Gall, 2006). However, our research found that EndNote showed this kind of problem not only with the Annual Review of Immunology and Blood, but the abbreviated title of the journal was generated for the other journals. The same mistake was found in the Mendeley reference manager which was unable to generate the shortened title only for the Journal of Clinical Endocrinology. It is again clear that the cause of the mistakes does not necessarily have to lie in the program itself but in the setting of the citation style. If there were a fault in the program, the data in references for all journals rather than just some of them would be generated incorrectly.
Some mistakes were undoubtedly also caused by the reference managers themselves. Mendeley showed an inability to generate the year of copyright in the form "c2009" because of the letter "c". After removing this letter, the program succeeded in generating the copyright date without problems. At the same time the JAMA and NLM instructions, for example, stipulate placing a letter "c" before the date in publications containing only the year of copyright (Iverson et al., 2007; Patrias, 2007). Zotero meanwhile had a problem with generating the place of publication and the publisher in the references to a web resource because there were no corresponding fields for this data in the record. However, the place of publication and the publisher are data required by the citation instructions provided in the Annual Review of
11
Immunology, BMC Cancer, The Lancet, the New England Journal of Medicine, Vaccine and the NLM style. RefWorks had problems with generating the first name of one of the editors of the edited book. This name was Klaus-Peter (i.e. names containing a hyphen) and RefWorks generated only the initial letter K while the other managers wrote K-P correctly.
Aside from these two sources of mistakes in references, a third was found as well: the directives in the reference instructions of a journal differed from its actual practice. Our research found that, for example, the reference instructions of Investigative Ophthalmology & Visual Science stipulate that a title of the book be written in the normal typeface (ARVO, 2015) while the title was actually written in italics in the published articles. Examples include references 46 and 47 in the introduction of the already-mentioned article on the regression of the tumour Melanoma (Rashid et al., 2015). Vaccine does not require in its citation instructions that the issue of the journal be given. In reality the issue can be mentioned, which can be seen, for example, in the article "Risk and outcomes of invasive pneumococcal disease in adults with underlying conditions in the post-PCV 7 era, The Netherlands" (Wagenvoort et al., 2016).
With this inconsistent approach it is understandable that some reference managers generate the references with data which do not correspond with the citation instructions for the authors or with the direct citation practice in the journal. On the other hand, mistakes were not always caused by different citation instructions and the usual citation practice. Especially in the case of print journal articles as the most often-cited type of resources in medical journals, it was therefore surprising to find that for more than half of journals and the NLM style, only two reference managers were capable of generating bibliographical references to a print article completely without any mistakes. This was the case of references to print journal articles, generally the most cited types of documents in medical journals, and namely references from Mendeley to Annual Review of Immunology, Anticancer Research, Blood, JAMA, Nature journals, Vaccine and the NLM style and a reference from Zotero to Blood, The Lancet, Nature journals, the New England Journal of Medicine and the NLM style. EndNote and RefWorks generated bibliographical references to print articles completely without any mistakes only for JAMA and The Lancet. EndNote also generated them for the NLM style.
Considering the above-stated facts, the question arises of which reference manager to recommend to authors intending to publish in medical journals. It is also necessary to take into account the functionality offered by the individual programs. Various studies (Gilmour & Cobus-Kuo, 2011; Muldrow & Yoder, 2009; Rapchak, 2012; Robbins, 2012; Steeleworthy & Dewan, 2013; Zhang, 2012) comparing different reference managers including EndNote, Mendeley, RefWorks and Zotero, have been published. The authors of these studies usually found similar functionality in all of these applications and they usually concluded that the program's suitability depends on the user. Despite this, they also mentioned the positive aspects and downsides of the individual programs.
In the case of EndNote the ability to look up information using the reference manager in external sources and databases and to fill in data in online archives in EndNote Basic were evaluated positively. The same applies to the problem-free functioning of the Cite While You Write plugin and the accessibility of installation files with the citation format on the websites of journals (Muldrow & Yoder, 2009; Rapchak, 2012; Zhang, 2012). The downsides, however, included the need to have a permanent internet connection, as well as the restricted number of external sources and databases for searching and downloading records and the charging of a fee for the use of the program (Steeleworthy & Dewan, 2013; Zhang, 2012). The cited studies also mentioned the impossibility of sharing the records with other users, which is presently possible, although the maximum number of users is 14 (Thomson Reuters, 2014b).
12
Features evaluated positively in Mendeley were the ability to save full texts and insert notes into them, the ability to also use this application on an iPad and the free access to the program. Mendeley was criticised for its inability to share records with other users (Gilmour & Cobus-Kuo, 2011; Robbins, 2012; Steeleworthy & Dewan, 2013).
RefWorks was criticised the most not only because its use is charged for but also because it requires a user to set up an account and have a permanent internet connection, as well as having institutional access. Also, the number of sources and databases for searching and downloading documents is restricted and the Write-N-Cite plugin is cumbersome (Gilmour & Cobus-Kuo, 2011; Muldrow & Yoder, 2009; Rapchak, 2012; Steeleworthy & Dewan, 2013). However, the possibility to archive data online and the simple import of records from accessible sources and databases have been evaluated positively (Muldrow & Yoder, 2009; Steeleworthy & Dewan, 2013).
In the case of Zotero, the possibility of using the program free of charge as well as the possibility to share records with other users, have been evaluated positively. Moreover, Zotero has a very simple method of downloading records from an unrestricted number of sources and databases, is simple to use and allows the editing of records and the possibility of using this reference manager on an iPad as well as Android devices (Gilmour & Cobus-Kuo, 2011; Muldrow & Yoder, 2009; Robbins, 2012; Steeleworthy & Dewan, 2013; Zhang, 2012). In the past, Zotero was criticised for not being able to archive data online, the necessity of manual copying of records for exchanging them among different devices and a restricted number of reference styles (Muldrow & Yoder, 2009).
The above-mentioned description of the functionality of individual reference managers is still relevant at the time of writing, except in the case of the Zotero program. Users of Zotero presently have the possibility to obtain an online storage space after creating an account. This online space can be synchronized with the manager installed on one's computer. There have also been changes in the selection of reference styles; at the time of creating this study (February 2016), Zotero contained 8,050 citation styles that are also used by the Mendeley program. EndNote offered 6,750 styles and RefWorks 4,015 citation styles (Mendeley, 2016; RefWorks Copyright, 2009; Roy Rosenzweig Center for History and New Media, 2016; Thomson Reuters, 2014a).
In connection with the repository, it must be noted that citation styles are constantly evolving, frequently being updated. The citation styles for EndNote and RefWorks are updated by the administrators of these programs (RefWorks Copyright, 2009; Thomson Reuters, 2014a), while the styles for Mendeley and Zotero are available in Zotero's Style Repository and can be updated by anyone with a knowledge of the Citation Style Language (CSL). Administrators of Zotero's Style Repository avoid the risk of incorrect citation style settings by means of an automated check of CSL source code as well as by publishing each citation style after consulting other members of the repository on it and approving it (Zelle, 2015a, 2015b). As the results in this study show, Zotero's Style Repository members devote considerable attention to the approval process. Moreover, any registered user can display the history of changes made to the settings of each citation style, which makes the repository transparent.
With respect to the findings concerning the quality of bibliographical references in this study and the technical possibilities of individual reference managers, Mendeley and Zotero seem to be the most suitable reference managers. This conclusion can be disputed, of course, as the results of the tests include types of publications which are not often cited in medical journals. Also, a comparison of the results for print articles only can bring different results. This objection is well founded, because it is journal articles that are mainly cited in medical journals (Barrett
13
et al., 2016; Delwiche, 2013; Rethlefsen & Aldrich, 2013). However, these articles are nevertheless cited as print resources even though scientific information sources are accessible to researchers in an online form these days. Furthermore, the mentioned studies found an increase of references to other sources published in the last 10 years compared to articles. Finally, current medical journals as well as the NLM style and the AMA styles, as essential citation styles for medicine, assume that reference may be made also to other kinds of resources and they include bibliographic reference templates for these in their instructions for authors.
Despite this, additionaly the analysis of the results relating to the references of printed articles was made and showed that Mendeley and Zotero achieve the best results with regard to the number of mistakes in bibliographical references to printed articles (Table 4). Although Zotero has the least number of mistakes in total, Mendeley could be regarded as a more accurate manager. In total Mendeley had one more mistake in references in comparison with Zotero but it had the least number of mistakes for most journals. Mendeley created the references to articles for the NLM style and for 10 out of 15 journals completely without mistakes while Zotero created references completely without mistakes for the NLM style and 8 journals. EndNote created references without any mistakes for the NLM style and two journals. RefWorks did the same for only two journals. It is evident then that also from the point of view of creating bibliographical references to printed articles Mendeley and Zotero are the reference managers which make bibliographical references with the lowest number of mistakes.
With regard to the above-mentioned facts, Mendeley and Zotero are the most suitable reference managers at the present time. Both of these programs provide the same basic functionality as EndNote and RefWorks, but they are capable of generating a larger range of bibliographical references without mistakes. In addition, they are free of charge, they have a bigger selection of reference styles and they can be used on mobile phones. The only fundamental difference between Mendeley and Zotero is the former's feature for inserting notes to downloaded full texts. Naturally, everything depends on how important this feature is to users, and this may influence one's decision in choosing between Mendeley and Zotero.
14
	EndNote				Mendeley				RefWorks				Zotero			
Journal Title	Data	Punctuation	Formatting	Total errors	Data	Punctuation	Formatting	Total errors	Data	Punctuation	Formatting	Total errors	Data	Punctuation	Formatting	Total errors
Annual Review of Immunology	2	6	0	8	0	0	0	0	0	4	0	4	1	0	1	2
Anticancer Research	0	2	0	2	0	0	0	0	0	2	0	2	3	0	0	3
Blood	1	0	0	1	0	0	0	0	0	0	2	2	0	0	0	0
BMC Cancer	1	5	7	13	1	6	6	13	1	8	4	13	1	6	6	13
Investigative ophthalmology	0	6	2	8	0	4	2	6	0	4	0	4	2	0	0	2
JAMA	0	0	0	0	0	0	0	0	0	0	0	0	2	0	0	2
Journal of Clinical Endocrinology	0	4	2	6	0	0	2	2	2	4	4	10	0	2	2	4
Lancet	0	0	0	0	0	2	2	4	0	0	0	0	0	0	0	0
Nature Medicine	0	2	0	2	0	0	0	0	0	2	0	2	0	0	0	0
Nature Reviews Cancer	0	2	0	2	0	0	0	0	0	2	0	2	0	0	0	0
Nature Reviews Immunology	0	2	0	2	0	0	0	0	0	2	0	2	0	0	0	0
Nature Reviews Neuroscience	0	2	0	2	0	0	0	0	0	2	0	2	0	0	0	0
Nature Reviews Drug Discovery	0	2	0	2	0	0	0	0	0	2	0	2	0	0	0	0
NEJM	2	2	0	4	0	2	0	2	1	2	0	3	0	0	0	0
NLM	0	0	0	0	0	0	0	0	1	0	0	1	0	0	0	0
Vaccine	1	1	0	2	0	0	0	0	0	1	0	1	1	0	0	1
TOTAL	7	36	11	54	1	14	12	27	5	35	10	50	10	8	9	27
Table 4 - Number of errors in bibliographical references to print articles in medical journals (Data - the number of errors in data, Punctuation - the number of errors in punctuation, Formatting - number of errors in formatting data, Average - the average number of errors in a reference)
Conclusion
Although this study has led us to recommend Mendeley or Zotero, the conclusion is not a definitive one. Both reference managers and reference styles are under continual development and the next test might show a different level of generated bibliographical references as well as the functionality offered by the reference managers. In addition, the results of the study are limited by the number of journals and the reference styles used for the comparison of the references. It is not realistic to make a complete analysis with the present number of professional journals. This is a further reason why it is necessary to perform regular tests to verify the ability of reference managers to generate bibliographical references with minimal mistakes. These tests should be performed primarily by librarians who, as information professionals, can best work with the structure of bibliographical records and data created from them. In the past, it has repeatedly been proven (Basak, 2014; Gilmour & Cobus-Kuo, 2011; Homol, 2014; Kessler & Van Ullen, 2005) that a number of authors had made mistakes in references as a result of incorrectly imported and insufficiently corrected records from different online sources and databases. Naturally, librarians can help the authors to check and correct mistakes, but a more sophisticated solution could lie in consistent verification of reference managers, their functionality and the quality of the bibliographical references they generate. The results of this study in fact reflect the need to test different reference managers to verify their suitability for users at a particular institution. Furthermore, while the subject of this study was to check the references generated for medical journals, it is at the same time important to carry out research on journals from other fields. Only in this way can librarians work effectively with the present capabilities of the reference managers in connection with authors' needs. In conjunction with this, they can adapt the range of their services, especially training on reference managers. They can at the same time provide the necessary feedback on the quality of the final products to the creators of reference styles.
References
ARVO. (2015). Online submission instructions for authors [Website]. Retrieved from
http ://io v s. arvoj ournals .org/ s s/forauthor s. aspx#msprep Bahar, Z., Beser, A., Elcigil, A., Karayurt, O., Vural, F., Ugur, O., & Kucukguclu, O. (2012).
Accuracy Of references in eight nursing journal. Healthmed, 6(6), 2066-2073. Barrett, A., Helwig, M., & Neves, K. (2016). Mapping the literature of hospital pharmacy.
Journal of the Medical Library Association, 104(2), 118-124.
http://doi.Org/10.3163/1536-5050.104.2.005 Basak, S. K. (2014). A comparison of researcher's reference management software:
Refworks, Mendeley, and EndNote. Journal of Economics and Behavioral Studies,
6(7), 561-568. Retrieved from http://ifrnd.org/Research Papers/J6(7)5.pdf Biebuyck, J. F. (1992). Concerning the ethics and accuracy of scientific citations. The Journal
of Anesthesiology, 77(1), 1-2. http://doi.org/10.1097/00000542-199207000-00001 Brahmi, F. A., & Gall, C. (2006). EndNote and Reference Manager citation formats compared
to 'instructions to authors' in top medical journals. Medical Reference Services
Quarterly, 25(2), 49-57. http://doi.org/10.1300/J115v25n02_04 Celayir, A. C, Sander, S., & Celayir, S. (2003). To the editor: Journal of Pediatric Surgery,
38(4), 653-654. http://doi.org/10.1053/jpsu.2003.50154 Cronin, B. (1982). Norms and functions in citation - the view of journal editors and referees
in psychology. Social Science Information Studies, 2(2), 65-11.
http://doi.org/10.1016/0143-6236(82)90001-1
16
De Lacey, G., Record, C, & Wade, J. (1985). Style matters - how accurate are quotations and
references in medical journals. British Medical Journal, 291(6499), 884-886.
http://doi.org/10.1136/bmj.291.6499.884 Delwiche, F. A. (2013). Mapping the literature of radiation therapy. Journal of the Medical
Library Association, 101(2), 120-127. http://doi.org/10.3163/1536-5050.10L2.007 Freimer, G. R., & Perry, M. M. (1986). Student problems with documentation. Journal of
Academic Librarians hip, 11(6), 350-354. Garfield, E. (1990). Journal editors awaken to the impact of citation errors. How we control
them at ISI. Current Comments, 13(41), 367-375. Retrieved from
http://www.garfield.library.upenn.edu/essays/vl3p367yl990.pdf Gilmour, R., & Cobus-Kuo, L. (2011). Reference management software: A comparative
analysis of four products. Issues in Science and Technology Librarians hip,
66(Summer). http://doi.org/10.5062/F4Z60KZF Homol, L. (2014). Web-based citation management tools: Comparing the accuracy of their
electronic journal citations. The Journal of Academic Librarians hip, 40(6), 552-557.
http://doi.Org/10.1016/j.acalib.2014.09.011 ICMJE. (2015). Recommendations for the conduct, reporting, editing, and publication of
scholarly work in medical journals: Updated December 2015. [USA]: International
Committee of Medical Journal Editors. Retrieved from http://www.icmje.org/icmje-
recommendations .pdf
Iverson, C, Chistiansen, S., Flanagin, A., Fontanarosa, P. B., Glass, R. M., Gregoline, B., ...
Young, R. K. (2007). AMA manual of style: A guide for authors and editors (10th ed.).
Oxford: Oxford University Press. Kessler, J., & Van Ullen, M. K. (2005). Citation generators: Generating bibliographies for the
next generation. Journal of Academic Librarianship, 31(4), 310-316.
http://doi.Org/10.1016/j.acalib.2005.04.012 Liu, M. (1993). Progress in documentation - the complexities of citation practice - a review of
citation studies. Journal of Documentation, 49(4), 370-408.
http://doi.org/10.1108/eb026920 Lok, C. K. W., Chan, M. T. V., & Martinson, I. M. (2001). Risk factors for citation errors in
peer-reviewed nursing journals. Journal of Advanced Nursing, 34(2), 223-229.
http://doi.Org/10.1046/j.1365-2648.2001.01748.x Mansfield, J. W. (1984). Why Not a Uniform Citation System. Journal of Academic
Librarianship, 10(4), 220-222. Mendeley. (2016). Adding new citation styles to Mendeley Desktop [Website]. Retrieved
from http://support.mendeley.com/customer/en/portal/articles/169935-adding-new-
citation-styles-to-mendeley-desktop Moorthy, A. L. (1988). Towards a Standard Style for Bibliographic References. DESIDOC
Journal of Library & Information Technology, 8(4). Retrieved from
http://publications.drdo.gov.in/ojs/index.php/djlit/article/view/2979 Muldrow, J., & Yoder, S. (2009). Out of cite! How reference managers are taking research to
the next level. PS - Political Science and Politics, 42(1), 167-172.
http://doi.org/10.1017/S1049096509090337 Oermann, M. H., Cummings, S. L., & Wilmes, N. A. (2001). Accuracy of references in four
pediatric nursing journals. Journal of Pediatric Nursing, 16(4), 263-268.
http://doi.org/10.1053/jpdn.2001.25537 Onwuegbuzie, A., Hwang, E., Combs, J. P., & Slate, J. R. (2012). Editorial: Evidence-based
guidelines for avoiding reference list errors in manuscripts submitted to Journals for
review for publication: A replication case study of educational researcher. Research in
17
the Schools, 19(2), i-xvi. Retrieved from
http://search.proquest.com/docview/1509202998 Park, S., Mardis, L. A., & Ury, C. J. (2011). I've lost my identity - oh, there it is... in a style
manual: Teaching citation styles and academic honesty. Reference Services Review,
39(1), 42-57. http://doi.org/10.1108/00907321111108105 Patrias, K. (2007). Citing Medicine: The NLM Style Guide for Authors, Editors, and
Publishers. (D. Wendling, Ed.) (2nd ed.). Bethesda, MD: National Library of
Medicine. Retrieved from http://www.ncbi.nlm.nih.gov/books/NBK7256/ Rapchak, M. (2012). RefWorks vs Endnote: A comparison of bibliographic management
software. Nursing & Allied Health Resources Section: Newsletter, 32(1), 2-4.
Retrieved from http://nahrs.mlanet.org/home/images/newsletter/v32nol2012.pdf Rashid, M., Heikkonen, J., & Kivela, T. (2015). Tumor regression after brachytherapy for
choroidal melanoma: reduction of thickness and cross-sectional area by shape and
regression pattern. Investigative Ophthalmology & Visual Science, 56(4), 2612-2623.
http://doi.org/10.1167/iovs.14-16322 Reddy, M. S., Srinivas, S., Sabanayagam, N., & Balasubramanian, S. P. (2008). Accuracy of
references in general surgical journals - an old problem revisited. The Surgeon:
Journal of the Royal Colleges of Surgeons of Edinburgh and Ireland, 6(2), 71-75.
http://doi.org/10.1016/S1479-666X(08)80067-4 RefWorks Copyright. (2009). Output style list [Website]. Retrieved from
http s ://w w w .refworks .com/ content/products/ output_style. asp Rethlefsen, M. L., & Aldrich, A. M. (2013). Environmental health citation patterns: mapping
the literature 2008-2010. Journal of the Medical Library Association, 101(1), 47-54.
http://doi.org/10.3163/1536-5050.101.L008 Robbins, L. P. (2012). Mendeley or Zotero: Which should the mobile researcher use? The
Charleston Advisor, 14(2), 5-11. http://doi.Org/10.5260/chara.14.2.5 Roy Rosenzweig Center for History and New Media. (2016). Zotero style repository
[Website]. Retrieved from https://www.zotero.org/styles Spivey, C. A., & Wilks, S. E. (2004). Reference list accuracy in social work journals.
Research on Social Work Practice, 14(4), 281-286.
http://doi.org/10.1177/1049731503262131 Steeleworthy, M., & Dewan, P. T. (2013). Web-based citation management systems: Which
one is best? Partnership: The Canadian Journal of Library and Information Practice
and Research, 8(1). Retrieved from
https://journal.lib.uoguelph.ca/index.php/perj/article/view/2220/2780 Sweetland, J. H. (1989). Errors in bibliographic citations: A continuing problem. Library
Quarterly, 59(4), 291-304. Terbille, C. (1990). Students and rules of style for reference citations. Libri, 40(3), 242-254. Thomson Reuters. (2014a). EndNote output styles [Website]. Retrieved from
http://endnote.com/downloads/styles Thomson Reuters. (2014b). Library sharing [Website]. Retrieved from
http://endnote.com/product-details/library-sharing Wagenvoort, G. H. J., Knol, M. J., de Melker, H. E., Vlaminckx, B. J., van der Ende, A.,
Rozenbaum, M. H., & Sanders, E. A. M. (2016). Risk and outcomes of invasive
pneumococcal disease in adults with underlying conditions in the post-PCV7 era, The
Netherlands. Vaccine, 34(3), 334-340. http://doi.Org/10.1016/j.vaccine.2015.ll.048 Wyles, D. F. (2004). Citation errors in two journals of psychiatry. Behavioral & Social
Sciences Librarian, 22(2), 27-51. http://doi.org/10.1300/J103v22n02_02 Zelle, R. M. (2015a). Criteria for accepting styles. Retrieved from https://github.com/citation-
style-language/styles/wiki/Criteria-for-Accepting-Styles
18
Zelle, R. M. (2015b). Quality control. Retrieved from https://github.com/citation-style-
language/styles/wiki/Quality-Control Zhane Y (2012) Comparison of select reference management tools. Medical Reference
Services Quarterly, 31(1), 45-60. http://doi.org/10.1080/02763869.2012.641841
19