Category Archives: The Profession

Debate on Misleading Literature Reviews

Richard P. Phelps wrote, “To save the research literature, get rid of the literature review,” in the London School of Economics (LSE) Impact Blog:

“The laziest literature reviews are those falsely declaring previous work on a topic to be nonexistent — “dismissive reviews”. When I first began to search for the magnitude of the dismissive review problem, I expected to find hundreds; I found hundreds of thousands…

You can see for yourself. Access a web database that allows searching by phrase (e.g. Google, Yahoo Search, Bing) and try some of these: “this is the first study”; “no previous studies”; “paucity of research”; “there have been no studies”; “few studies”; “little research”; or variations thereof.”


“I don’t believe that journals can responsibly review manuscript literature reviews for accuracy. And, because they cannot, literature reviewers are free to trash or ignore competing work. Thus, ironically, literature reviews may degrade our knowledge base more than preserve it.”

Dr. Phelps then gives six reasons for journals to drop the literature review. I encourage you to read his article, linked above.

In response to Dr.  Phelps, Arnaud Vaganay wrote, “To save the research literature, let’s make literature reviews reproducible,” in the same publication outlet.

Dr. Vaganay agreed with the problem of literature reviews, and wrote:

“…editors and peer reviewers do little to ensure that LRs tell the whole story rather than a good story. It is hard to blame them; by some accounts the scientific output doubles every nine years. As it becomes increasingly difficult to stay on top of the literature – even in one’s own field – editors and peer reviewers should pay more attention to the way LRs are conducted.”

He departs from Dr. Phelps in his argument that literature reviews (LR) can be reformed by making some formal requirements:

“Readers who are as frustrated with the current state of LRs as Richard Phelps and me might be interested in Cumulative Literature Reviews (CLRs) (see from slide seven onwards). CLRs start from the premise that LRs and their components (literature dataset, LR protocol) are research assets, just like the protocol and dataset used in the empirical part of a study… The methodology is still at the development stage and is being piloted in two ongoing studies.”




More Good Advice for Publishing in Journals

Hugh McLaughlin, a journal editor, wrote “How to increase your likelihood of publishing in peer reviewed journals.” Some highlights:

Address “SO WHAT?”

“You might want to consider some key questions: does your article contribute new knowledge? Does it offer a unique way to address a social problem or policy dilemma? Will it have implications for practice?”

Revise, Revise,  Revise

“Once you start writing it’s important to remember that each paragraph and page is only a draft.”

Know Your Journal

“Editors like to see that potential authors are aware of the journal and its content.”

Theory is Important

“Infusion refers to the infusion of theory into the manuscript, providing readers with new ways of provoking discussion, challenge and applications to practice. It was noted in our recent publication that “articles that are theoretically strong receive the most citations”. ”

Respond to Each Reviewer

“Respond to each reviewer separately showing how you have addressed the points they have raised.”

Rejection is Good, but Time Is Better

“You may want to consider putting the manuscript aside for a period so that in time you may gain a fresh perspective on it, consider how it can be improved and where you can resubmit it.”

On Reviewing Articles for Journals

One of the assignments for the “From Idea to…” class was to review an article.  Several classes were devoted to discussions of the structure of empirical articles in the social sciences, and one class was devoted to writing article reviews; students were given a review of an article as an example. Everyone in the class (N = 15) reviewed the same article.

What is striking is how many different views there are about the same article.  Here are some examples of student reviews.

On the style of writing:

— “Article is written very clearly and precisely, all data is analyzed very carefully, research question is answered, all hypotheses are verified.”

— “The structure of the article is clear and simple. The language is academic, hence demanding, but not too hard to understand even for a general public. ”

— “The paper is written clearly and in a manner that makes it easy to follow also for a non-specialist in the field.”


—  “Values of this in a way pioneer analysis are undermined however by serious shortcomings in the presentation of the study and its results, its chaotic structure, lack of definitional clarity and theoretical profoundness.”

— “My main comment towards the article is that it uses too many intellectual shortcuts, both in theoretical explanation of hypothesis and in the results interpretation. Therefore, I perceived it as more journalistic than academic, despite of being based on the survey data.”

— “The results that follow each other are sometimes describe in such a modest way, that a reader could easily miss some important information or understand it incorrectly.”

Most identified problems such as:

— Article fails to adequately answer, “So what?”

— Article does not cite enough previous literature, and thus the theory and expectations section suffers

— Article does not describe the data used in adequate detail

— Author should not assume that “party image” is a valid indicator of “party system”; a clearer link should be made.

Others thought that the political parties were not adequately described:

—  “The descriptions of the political parties are very journalistic and too simplistic. What does it mean that PO is pragmatic? Or PiS nationalistic? Again it sounds more like repeating the clichés.”

— “[need] to improve the characteristics of the Polish parties, which is not very precise (Samooborona as a “traditionalist Catholic” party, or controversial characteristic of PiS as a nationalistic party).”

Others question whether women and the old are disadvantaged groups:

— “Why women, the old and the poor have been chosen? Are they really disadvantaged in Poland? How do we know that?”

— “The [assumption of the] article is … the idea that women, the old and the poor are disadvantaged social groups and that those groups are less visible during the political process. In terms of a large number of socio-economic indicators those groups definitely have certain disadvantages in their social life, but their little visibility in political process is very questionable.”

A very interesting criticism is this:

— “Another point is that data was collected in 2008, one year after Parliamentary elections: it is known that parties care much more about their image before elections. Perhaps, research during ongoing PR campaign could show different results.”

Most recommended to revise & resubmit or even conditionally accept the article.  Perhaps the students thought that the course instructor is also the author of the article, and decided to be kind; or, the students are too kind anyway, even if their criticisms are direct and rather unfavorable to the work.

On writing article reviews:

Key to understanding how to write an article for review for a journal is that the editor of the journal, the reviewer and the author are all engaged in a debate over the author’s work.  The author presents their research and is supposed to do so clearly and in a detailed fashion.  The editor of the journal asks a “reviewer” to enter this debate; the reviewer’s job is to (a) demonstrate that they understood the article as best they could, (b) present their criticisms in as clear and detailed way as the author had done (or sometimes in an even clearer and more detailed way if the author’s article is insufficiently clear and detailed), and (c) tell the editor whether the article should be rejected outright, rejected but invited to revise the article and resubmit it for a second review (so-called “Revise & Resubmit”), conditionally accept the article (meaning the article has so few flaws that only minor revisions are needed), or accept as is (this almost never happens).

The reviewer has a professional and ethical responsibility to the editor, the author, and the larger scientific community to be clear and detailed in their criticisms.

Remember, too, the audience for reviews: it is for the editor and the author, both of whom are your colleagues in the scientific community.  You have a responsibility to be respectful to them.  Reviews should be direct and honest, but language used by the reviewer can impede the points the reviewer wants to make.  Reviewers should be as careful in their reviews as the author was in writing the article.  Criticisms should be constructive, an attempt to help the author of the article do better.

Other notes:

The beginning of a professional article review necessarily begins with a summary of the article.  This is so that the editor of the journal – the one who makes a decision as to whether the article will be published in their journal – and the author who receives the review are both confident that the reviewer understood the article.

Important to note that in articles which contain tables and figures, editors and reviewers expect that the tables and figures are listed as “–  Table 1 about here –“ in the place where the table would be, and that the table or figure is placed in the back of the paper.

In modern scientific writing in English, it is acceptable to use the first person singular, „I.”  (if the work is co-authored, then “we” is used). This is called, “active writing,” and most modern journals prefer active writing.

As an example, let’s take this sentence from the December 2010 issue of the American Sociological Review, the most prestigious English language journal in sociology (emphasis added):

“Applying this strategy to several different cases, we identify three trajectories scientific propositions assume on their way through contestation to consensus among practicing scientists…”

And these two sentences from a different article in this same issue:

“In this article, I explain variation in token experiences by demonstrating that tokenism is contingent on the local cultural context in which it is embedded. I develop this argument by comparing the experiences of female and African American male investors in the leveraged buyout (LBO) industry.”

You can see these examples here:

and here:

Unfortunately or not, in modern science writing it is not necessary for authors to state how their research might be “useful for the world.”  World-usefulness is not a criterion for inclusion into social science journals.

The point that the writing style is too “journalistic” can be valid, but you would need to provide a specific example of a sentence where the style impedes on understanding the author’s thought.   At what point is “too journalistic” also “too unscientific?”