The "Impact Factor" and Selected Issues of Content and Technology in Humanities Scholarship Published Online

Submission

Tötösy de Zepetnek, Steven. "The 'Impact Factor' and Selected Issues of Content and Technology in Humanities Scholarship Published Online." Journal of Scholarly Publishing 41.4 (2010): forthcoming.

The "Impact Factor" and Selected Issues of Content and Technology in Humanities Scholarship Published Online — Steven Tötösy de Zepetnek.

Abstract: Steven Tötösy de Zepetnek's article "The 'Impact Factor' and Selected Issues of Content and Technology in Humanities Scholarship Published Online" is about current developments in the US, Europe, and Asia with regard to metrics and the "impact factor" in humanities publishing and their relevance to scholarship as to its "value" for promotion and tenure. Long established in the sciences and medicine, the impact factor as to the "value" of published scholarship, this has become a much–debated and contentious issue in the humanities. Tötösy de Zepetnek presents examples of and a brief discussion about the situation of metrics in humanities scholarship with particular reference to online publishing including the example of the Purdue University Press peer–reviewed, full–text, and open–access humanities and social sciences online quarterly CLCWeb: Comparative Literature and Culture (1999–).

Keywords: Metrics, Impact Factor, Humanities Publishing, Humanities Publishing Online, Arts and Humanities Citation Index, Elsevier Scopus, European Reference Index for the Humanities, Council of Editors of Learned Journals, CLCWeb: Comparative Literature and Culture, Purdue University Press.

In this article I discuss issues of content and valuation in publishing humanities scholarship online. What I mean by content is not scholarship per se, but the "value" of an article published by a scholar as to its locus of publication. While in the U.S. and Canada peer review is accepted as a measure of quality for promotion and tenure (although humanities departments and faculties of arts at times keep their own lists of "important" journals ranked as to the importance of the publication of articles), in Europe and in Asia (for example in Spain, Belgium, increasingly in Germany, India, Taiwan, the People's Republic of China, etc.) there is a move in the humanities towards the principle of the "impact factor" as borrowed from the social sciences, the natural sciences, medicine, and other empirical and applied fields. I suspect the implementation of the impact factor in the humanities may play a role sooner rather than later also in the U.S., Canada, and Australia. It is of particular relevance that the said impact factor with regard to the humanities is tied to one particular indexing and abstracting service, the for–profit company Thomson Reuters Institute for Scientific Information Arts and Humanities Citation Index. Importantly, the valuation of scholarship via the impact factor creates a challenge for the humanities in general and for online publishing in the field in particular.

Before I discuss the situation I outline above, I would like to note that there is, a priori, a difference between those (universities and scholars themselves) who prefer the traditional way of publishing scholarship in print and those who believe in adopting the mode of online publishing using the advantages of new media technology: it is curious situation, indeed, that while most online journals' publication of scholarship is no different from print publication except the mode of delivery, scholars and the academe — often and still — regard online publishing of less relevance, value, and importance than print publication. This has several implications, including that in some humanities departments the publication of articles in online journals — even if and despite of peer–review — are not accorded the same value as print publications. Situations thus arise in which scholars are in fact discouraged from publishing online. There is, then, a further gap between those who prefer online publishing based on subscription fees and those who argue for online publishing in open access. At the same time, owing to pressures in funding and the overall constricted situation of academic publishing in particular in the humanities, as well as the dramatic increase in published scholarship, most humanities scholars recognize the inevitability and necessity of digital publishing. This recognition is grudging, however, and with objections at many levels.¹ One such issue is the said growing importance of the impact factor and its relevance for publishing humanities scholarship online.

The concept of the impact factor first arose in the 1960s with the science Citation Index (SCI) and, subsequently, with Journal Citation Reports (JCR). In its simplest terms, the impact factor is a measure of the frequency with which an article in a journal has been cited in a particular year or period. The impact factor is meant to indicate total citation frequencies and to eliminate some of the bias of counts that favor large journals over small ones, frequently issued journals over less–frequently issued ones, and older journals over newer ones. In the humanities, there are three indexing services considered increasingly as the primary carriers of the impact factor: 1) the Arts and Humanities Citation Index (AHCI, a subset of the Thomson Reuters Institute for Scientific Information [ISI] Web of Knowledge); 2) the European Reference Index for the Humanities (ERIH, recent and still in the process of development); and 3) Elsevier's Scopus, which does mostly science–based indexing, and is similar to the above ISI, a for–profit indexing and abstracting service.

With regard to the above three indexing and abstracting services for online humanities publications, the situation is not optional. For example, AHCI started to consider the indexing of articles published in online humanities journals only in 2005. An additional problem with the AHCI is that it neglects humanities journals in general, and humanities journals published online in particular. Christine L. Borgman, in her Scholarship in the Digital Age (2007), underlines the less–than–adequate coverage of humanities scholarship: "The depth of coverage in the ISI Web of Knowledge, which is among the most comprehensive online bibliography databases, is deepest in the sciences, shallower in the social sciences, and most shallow in the humanities … Indicators in the ISI citators are the least valid for the arts and humanities because they only include references made by journal articles."² "Complicating matters further," Borgman continues, "the Science, Social Science, and Arts and Humanities Citations Indexes are a closed system consisting of references made by established journals on a list selected by the editors at Thomson Scientific. Not all journals are included, and books and conference proceedings rarely are indexed."³ Borgman reaches the conclusion that "The simplest approach to clarifying the legitimacy of digital documents is to rely on traditional quality indicators such as the imprimatur of well–regarded publishers. This approach, however, cedes much of the control for legitimization to publishers and discourages experimental forms of publication. For example, if the only publications valued for promotion, tenure, and institutional reviews are those that appear in journals with high–impact factor as measured by ISI citation statistics … journals indexed by the ISI have an inordinate power over the quality–control system."⁴

The above situation is an a priori handicap in the valuation of humanities scholarship, but even if we accept the inevitability of the widely sanctioned primacy of the ISI and its citation indices, the neglect of humanities journals in general and those published online in particular, the additional obstacles of being accepted by the indexing service is a further matter of relevance. The example I employ to illustrate selected issues is the journal I have been editing since 1999 and publish with Purdue University Press, CLCWeb: Comparative Literature and Culture (ISSN 1481–4374). The journal was submitted for indexing by ISI's AHCI in 2001 at which time AHCI's response was that indexing would not be possible because the journal's material is not cited in print journals (this was an argument similar to the one I received in 1999 when I applied for funding to the Social Sciences and Humanities Research Council of Canada: the response was that unless texts published in an online journal are cited in print journals and unless the online journal can show subscription fees, no funding is possible). By 2005 ISI changed its approach to the indexing of online journals in the humanities and thus I re–submitted the journal. While the content of the journal was approved speedily, AHCI told me that there are technical issues to be resolved and that it would take some time to arrive at the approval of the journal for indexing, this because CLCWeb is published in html. At the same time Purdue University Press embarked on a number of projects with regard to its digital repositories and online publications and by 2006 the Press decided to publish its online journals in pdf in DigitalCommons with a third–party publisher, the Berkeley Electronic Press. Thus, between January 2006 and January 2007 I converted with the assistance of three editorial assistants all material of CLCWeb 1999–current for publication in pdf and re–submitted the journal to AHCI for indexing. In March 2009 I was told again that the journal has been approved with regard to its scholarly content and with the technical evaluation remaining CLCWeb would likely be indexed starting in 2009. On 11 November 2009 I received the approval to index from ISI with the statement that CLCWeb has been approved for indexing in AHCI as of volume 10 issue 1 (2008) (on this, namely backdating the indexing, see below). Overall, my experience with AHCI has been not the best because of the time span to arrive at the approval of indexing journal stretching from 2001 until 2009: while there may have been legitimate issues because of matters technical, the long stretch to receive the approval to index is less than acceptable (after all, ISI is a for–profit company whose institutional subscription fee for university libraries is USD 11,000/year) if for no other reason than the urgency of the prominence of ISI and the development to accord AHCI indexed journals the prominence and importance I mention above.

The European Reference Index for the Humanities (ERIH) is an undertaking by the European Union's European Science Foundation and "aims initially to identify, and gain more visibility for top–quality European Humanities research published in academic journals in, potentially, all European languages. It is a fully peer–reviewed, Europe–wide process, in which 15 expert panels sift and aggregate input received from funding agencies, subject associations and specialist research centres across the continent. In addition to being a reference index of the top journals in 15 areas of the Humanities, across the continent and beyond, it is intended that ERIH will be extended to include book–form publications and non–traditional formats. It is also intended that ERIH will form the backbone of a fully–fledged research information system for the Humanities."⁵ At ERIH's inception in 2006, the humanities experts chosen to decide which journal would be included in their ranking system numbered all of four scholars: this has changed since and now there are dozens of scholars appointed to select and review journals in many fields in the humanities. CLCWeb: Comparative Literature and Culture was submitted for indexing by ERIH in 2006, at which time I received no acknowledgement or any communication; the journal was resubmitted in September 2007 and ERIH's "Junior Science Officer"—who stated just having been hired—apologized in a June 2008 e–mail that my previous submission was not acknowledged and wrote also that by the end of 2008 the selection of submitted journals would be completed and that such selections will be made every four years after that. The last communication I received from ERIH was an e–mail of 17 October 2008. I have not heard anything since about whether the journal was approved for indexing or whether the evaluation process would be still ongoing.

In 2008, across Europe as well as in Australia and on the listserve of the Council of Editors of Learned Journals (CELJ), a discussion about ERIH's process of selection erupted. Editors of humanities journals and others criticized the "numerics" and "metrics" approach applied to humanities scholarship and ERIH's A, B, or C system of journal rankings (see also the report on ERIH by Jennifer Howard in The Chronicle of Higher Education in 2008). While ERIH argued that their system of ranking is not based on descending valuation but on different categories of journals, it remained clear to most that in the end a journal ranked A would end up as a top journal regardless of the category designation.⁶ ERIH, similar to AHCI, does not appear to have an interest in online journals in the humanities and the current list of the so–called initial 2008 rankings of ERIH has few of such.

Elsevier's Scopus is an abstracting service similar to ISI's AHCI, that is, it is a for–profit company. In my perception, the humanities rate low on their list of priorities, and until 2008 they did not even consider to include the humanities in their indexing services;⁷ however, as of 2009 they are starting the indexing of humanities journals. In comparison with AHCI and ERIH, my experience with Scopus has been better in the sense that after submission of CLCWeb for indexing in December 2008 I received the decision to include the journal in March 2009 and since October 2009 the journal is listed and indexed among Scopus' Arts and Humanities journals.

With the three indexing services discussed above, one further matter remains curious: once an online journal is accepted for listing and indexing, why would these services begin indexing only in the year of acceptance? Clearly, with online journals, whose history is no older than the birth of the world wide web in 1994, the indexing of all material published would be easy and of little expense—technically speaking—precisely because all its material is available online. I put this question to all three indexing services. AHCI responded that they would remain as is with the start of indexing with the year of acceptance (although, see above, when I received the decision to index, they approved the journal for indexing with 2008, thus backdated…), ERIH did not reply, and Scopus responded that indeed my proposal does make sense and that they are looking into the indexing of all material published in an online journal and that this may happen in the future.

As mentioned above, the move in the humanities towards valuation of work via the impact factor is occurring mostly in Europe and Asia, but this move has implications also for journals published in the U.S., Canada, and Australia.⁸ In many instances a journal published in the U.S., for example, is missing out on good work because scholars are reluctant to publish in a journal that is not indexed in the AHCI, and in the case of CLCWeb>: Comparative Literature and Culture this has occurred several times over the years. It remains without saying that the overall reluctance of many U.S. humanities scholars to publish in an online journal — whether AHCI–indexed or not — remains an indicator of the slow pace of the humanities alongside new media technology, an issue that has not changed significantly since 2001.⁹

Next, I discuss briefly a number of items relevant for online publishing. One of the most important of these, which in my opinion is often misapplied, is the matter of a journal's Universal Resource Locator (URL). There is the argument against long URLs; however, in my opinion the URL of a journal, whenever possible, ought to be a "narrative" URL and not an acronym. I think the reason that so many journals have acronyms and not narrative URLs, which would make the journal immediately recognizable, has much to do with the disjointed relationship between the scholar and the "techie." To illustrate this, here is again the example of CLCWeb: Comparative Literature and Culture: when the university press that publishes the journal decided to publish in .pdf instead of the previous HTML format, the URL was designated without consultation with me, the journal's editor, and a press technology expert made up the journal's new URL. In other words, it is important that the technology side and the scholarship side of a learned journal communicate with each other, and it should also be a standard that the editor of an online journal be someone who knows new media technology.

Another important issue is mirrors and archival preservation. Here, the U.S. is relying on the multiplication of institutional and commercial depositories of archives instead of on a national policy. The Library of Congress has no depository of online journals. In contrast, most countries have been creating national depositories of electronic material since the late 1990s. In addition to the archiving of a journal's material in a national digital depository, the use of mirrors is advisable not only for this type of preservation but also for the facilitation of the speed of download in other parts of the world. A mirror of a journal published in the U.S., for example, functions thus in Europe or Asia not only as facilitating a higher speed of download but also as a depository.

In conclusion, I would like to suggest that institutions such as university presses and associations and organizations in the humanities would make efforts to persuade and pressure indexing services such as the AHCI, Elsevier's Scopus, and the European ERIH as to their approach to humanities scholarship published online (or in print): a perpetually and traditionally underfunded field of scholarship, the humanities must receive due recognition and prompt inclusion for indexing not because I lament the traditional non–recognition of the humanities as not being "practical" and applied enough; rather, I am suggesting this because, after all, universities and university libraries subscribe world wide to the said indexing services to the tunes of thousands of dollars per year and since such subscriptions are package deals thus including the humanities, it is unacceptable that the humanities remain disregarded and on the second burner whether print or online journal indexing.

Note: Research for the above article and participation in the 2008 convention of the Modern Language Association of America panel on learned journals was funded by Purdue University Press and National Sun Yat–sen University's Center for the Humanities and Social Sciences. I thank Nathan L. Grant (Saint Louis University) for his coments on the paper.

¹ For more detail, see Steven Tštšsy de Zepetnek, "The New Knowledge Management and Online Research and Publishing in the Humanities," CLCWeb: Comparative Literature and Culture 3.1 (2001): Perdue University Press. See also Tštšsy de Zepetnek, "New Media, Publishing in the Humanities, and CLCWeb: Comparative Literature and Culture,"Formamente: Rivista Internazionale di Ricerca sul Futuro Digitale / International Research Journal on Digital Future 2.1–2 (2007): 255–73.

² Christine L. Borgman, Scholarship in the Digital Age: Information, Infrastructure, and the Internet (Cambridge, MA: MIT Press, 2007), page 158–59.

³ Borgman, Scholarship in the Digital Age, page 64.

⁴ Borgman, Scholarship in the Digital Age, page 85.

⁵ ERIH: European Reference Index for the Humanities.(2009):

⁶ Jennifer Howard, The Chronicle of Higher Education (10 October 2008): "New Ratings of Humanities Journals Do More Than Rank — They Rankle,".

⁷ Borgman, Scholarship in the Digital Age, page 215.

⁸ See also Howard, "New Ratings of Humanities Journals Do More Than Rank — They Rankle," (2008).

⁹ See Tštšsy de Zepetnek, "The New Knowledge Management and Online Research and Publishing in the Humanities," (2001) and Tštšsy de Zepetnek, "New Media, Publishing in the Humanities, and CLCWeb: Comparative Literature and Culture" (2007).

Steven Tštšsy de Zepetnek taught in the University of Alberta Department of Comparative Literature 1984–2000. Residing in Boston since 2000, he is professor of media and communication studies at the University of Halle–Wittenberg (Germany) and professor of literature at National Sun Yat–sen University (Taiwan). He is editor since 1999 of CLCWeb: Comparative Literature and Culture (ISSN 1481–4373), the Purdue University Press peer–reviewed, full–text, and open–access humanities and social science online quarterly and series editor since 2001 of the Purdue University Press hard–copy monograph series of Books in Comparative Cultural Studies. Contact: Comparative Literature and Culture.