Environmental Economics: "Even top economists publish in predatory journals, study finds" #RePEc

October 27, 2016

"Even top economists publish in predatory journals, study finds" #RePEc

Retraction Watch:

Top-ranking economists sometimes publish papers in open access journals deemed potentially “predatory,” according to a new analysis.

The findings contradict previous results that show that researchers who publish papers in “potential, possible, or probable” predatory journals (as defined by librarian Jeffrey Beall) are largely inexperienced.

According to the study, 27 of the most eminent economists (within the top 5% of their field) have published nearly 5% of their papers in predatory journals. These researchers published 31 papers in predatory journals in 2015 alone.

The finding — which is not yet peer reviewed — comes as a “big surprise,” co-author Frederick Wallace of the Gulf University for Science and Technology in West Mishref, Kuwait, told Retraction Watch.

At the end of last year, Wallace and Timothy Perri from the Appalachian State University in Boone, North Carolina noticed that 39 predatory journals were listed on the database Research Papers in Economics (RePEc). After some digging, the pair identified just under 1,300 papers published last year in predatory journals in RePEc, submitted by close to 2,800 authors.

Although only around 5% of the study’s sample authors who’d published in predatory journals (124 individuals) were registered with RePEc, 27 (around one-fifth) are listed among the top 5% of RePEc-registered authors. ...

Mark McCabe, an economist at Boston University in Massachusetts and SKEMA Business School in Sophia Antipolis, France, raised some doubts over whether the findings truly represent how often the field’s top researchers are publishing in predatory journals. For instance, the top 5% in RePEc may not represent the true top 5% of the field, he noted:

1. In what type of non-predatory journals are these (top 5%) authors publishing papers (the difference between predatory and non-predatory can be very small)? Impact Factors? Publishers? etc. Are any of them among the top 100 journals in economics?

2. What are the institutional affiliations of [these] (top 5%) authors? Any top-ranked economics departments or business schools?

3. Where is the list of the predatory journals in which these 148 papers were published?

Jeffrey Beall, a librarian at University of Colorado Denver and the creator of Beall’s lists, noted:

Actually, a broad range of people submit to predatory journals.

He added that he was also hesitant to draw too strong a conclusion from these findings:

I see this study as preliminary, and I am not sure it contains strong evidence.

Asked about the criteria RePEc uses to rank researchers, Wallace pointed us to this paper, which outlines the process RePEc uses to rate authors registered on the site using factors such as the number of papers, citations received, journal page counts, and views and downloads statistics.

Wallace argued that he believes that a high ranking in RePEC is something the field takes seriously:

I’d blow my own horn if I was listed as a top 5% researcher. I’d include it in my annual report every year.

via retractionwatch.com

I've read the paper (I prefer this link) and the evidence it presents is quite strong. The evidence is an accounting exercise, matching journals from Beall's list to economists on the RePEc list. There is no fancy analysis. If you don't like the evidence then you must think there is something wrong with one of the lists. I know of no other list of predatory journals. The RePEc economist ranking list is objective (here is a paper that critically examines the rankings) and includes everyone who wants to be included (you must be a registered RePEc author to be ranked). So, the questions from Dr. McCabe are easily answered:

For evidence that RePEc rankings are taken seriously, here is Matt Kahn:

From a worker's perspective, if you earn a good reputation at one firm --- how do you cash in on this fact with firms that don't know you have a good reputation? How do you signal your quality? In the past, an Ivy League degree or a Mensa membership card helped but in this economy --- will workers soon have a vector of crowd sourced metrics that they can display to potential employers? A future labor economist could then run hedonic wage regressions to test how the labor market rewards each of these metrics.

This world already exists in academic economics. REPEC does a pretty good job of ranking us. For those at public universities, you can look up our salaries and correlate it with our REPEC rankings. You will see a very high positive correlation. Academic economics can be modeled pretty well as a perfectly competitive labor market. There is an element of crowd sourcing to REPEC rankings as citations and paper downloads are some of the metrics used to create the single index of economist "quality".

And for evidence that "I'd blow my own horn if I was listed as a top 5% researcher", here again is Matt Kahn:

REPEC provides an objective measure of who is "Royalty" in the economics profession. The current list of the top 5% is here. I am ranked #681 out of 27,365 economists so that's not bad (and my 3 books aren't counted here).

Posted by John Whitehead on October 27, 2016 at 02:52 PM in Kahn, Research | Permalink

Comments

Retraction Watch:

Top-ranking economists sometimes publish papers in open access journals deemed potentially “predatory,” according to a new analysis.

The findings contradict previous results that show that researchers who publish papers in “potential, possible, or probable” predatory journals (as defined by librarian Jeffrey Beall) are largely inexperienced.

According to the study, 27 of the most eminent economists (within the top 5% of their field) have published nearly 5% of their papers in predatory journals. These researchers published 31 papers in predatory journals in 2015 alone.

The finding — which is not yet peer reviewed — comes as a “big surprise,” co-author Frederick Wallace of the Gulf University for Science and Technology in West Mishref, Kuwait, told Retraction Watch.

At the end of last year, Wallace and Timothy Perri from the Appalachian State University in Boone, North Carolina noticed that 39 predatory journals were listed on the database Research Papers in Economics (RePEc). After some digging, the pair identified just under 1,300 papers published last year in predatory journals in RePEc, submitted by close to 2,800 authors.

Although only around 5% of the study’s sample authors who’d published in predatory journals (124 individuals) were registered with RePEc, 27 (around one-fifth) are listed among the top 5% of RePEc-registered authors. ...

Mark McCabe, an economist at Boston University in Massachusetts and SKEMA Business School in Sophia Antipolis, France, raised some doubts over whether the findings truly represent how often the field’s top researchers are publishing in predatory journals. For instance, the top 5% in RePEc may not represent the true top 5% of the field, he noted:

1. In what type of non-predatory journals are these (top 5%) authors publishing papers (the difference between predatory and non-predatory can be very small)? Impact Factors? Publishers? etc. Are any of them among the top 100 journals in economics?

2. What are the institutional affiliations of [these] (top 5%) authors? Any top-ranked economics departments or business schools?

3. Where is the list of the predatory journals in which these 148 papers were published?

Jeffrey Beall, a librarian at University of Colorado Denver and the creator of Beall’s lists, noted:

Actually, a broad range of people submit to predatory journals.

He added that he was also hesitant to draw too strong a conclusion from these findings:

I see this study as preliminary, and I am not sure it contains strong evidence.

Asked about the criteria RePEc uses to rank researchers, Wallace pointed us to this paper, which outlines the process RePEc uses to rate authors registered on the site using factors such as the number of papers, citations received, journal page counts, and views and downloads statistics.

Wallace argued that he believes that a high ranking in RePEC is something the field takes seriously:

I’d blow my own horn if I was listed as a top 5% researcher. I’d include it in my annual report every year.

via retractionwatch.com

For evidence that RePEc rankings are taken seriously, here is Matt Kahn:

From a worker's perspective, if you earn a good reputation at one firm --- how do you cash in on this fact with firms that don't know you have a good reputation? How do you signal your quality? In the past, an Ivy League degree or a Mensa membership card helped but in this economy --- will workers soon have a vector of crowd sourced metrics that they can display to potential employers? A future labor economist could then run hedonic wage regressions to test how the labor market rewards each of these metrics.

This world already exists in academic economics. REPEC does a pretty good job of ranking us. For those at public universities, you can look up our salaries and correlate it with our REPEC rankings. You will see a very high positive correlation. Academic economics can be modeled pretty well as a perfectly competitive labor market. There is an element of crowd sourcing to REPEC rankings as citations and paper downloads are some of the metrics used to create the single index of economist "quality".

And for evidence that "I'd blow my own horn if I was listed as a top 5% researcher", here again is Matt Kahn:

REPEC provides an objective measure of who is "Royalty" in the economics profession. The current list of the top 5% is here. I am ranked #681 out of 27,365 economists so that's not bad (and my 3 books aren't counted here).