Debate on Misleading Literature Reviews

Richard P. Phelps wrote, “To save the research literature, get rid of the literature review,” in the London School of Economics (LSE) Impact Blog:

“The laziest literature reviews are those falsely declaring previous work on a topic to be nonexistent — “dismissive reviews”. When I first began to search for the magnitude of the dismissive review problem, I expected to find hundreds; I found hundreds of thousands…

You can see for yourself. Access a web database that allows searching by phrase (e.g. Google, Yahoo Search, Bing) and try some of these: “this is the first study”; “no previous studies”; “paucity of research”; “there have been no studies”; “few studies”; “little research”; or variations thereof.”

and

“I don’t believe that journals can responsibly review manuscript literature reviews for accuracy. And, because they cannot, literature reviewers are free to trash or ignore competing work. Thus, ironically, literature reviews may degrade our knowledge base more than preserve it.”

Dr. Phelps then gives six reasons for journals to drop the literature review. I encourage you to read his article, linked above.

In response to Dr.  Phelps, Arnaud Vaganay wrote, “To save the research literature, let’s make literature reviews reproducible,” in the same publication outlet.

Dr. Vaganay agreed with the problem of literature reviews, and wrote:

“…editors and peer reviewers do little to ensure that LRs tell the whole story rather than a good story. It is hard to blame them; by some accounts the scientific output doubles every nine years. As it becomes increasingly difficult to stay on top of the literature – even in one’s own field – editors and peer reviewers should pay more attention to the way LRs are conducted.”

He departs from Dr. Phelps in his argument that literature reviews (LR) can be reformed by making some formal requirements:

“Readers who are as frustrated with the current state of LRs as Richard Phelps and me might be interested in Cumulative Literature Reviews (CLRs) (see from slide seven onwards). CLRs start from the premise that LRs and their components (literature dataset, LR protocol) are research assets, just like the protocol and dataset used in the empirical part of a study… The methodology is still at the development stage and is being piloted in two ongoing studies.”