I want to pass this post by Peter Boettke for reactions, although I don't have time to think in detail about it now. He mentions a New York Times article about a more open, blog-like "crowd-sourcing" of the peer review process. I didn't even get a chance to read the article in detail yet, but it sounds interesting.
I do have my doubts, though. I have very little experience with the peer review process, and the experience I have had doesn't entirely reaffirm the value of the status quo, but I worry that this is the wrong stage in the process for introducing a blog-like approach.
It seems to me peer-review is simply a way of ensuring quality and making sure important points in research were hit, right? I don't see how this blog-like approach helps with that. What blogging can offer is a way to formalize a much wider range of comments and critiques than the normal journal format for replies and comments. A good example of this is what Econ Journal Watch is doing to collect a series of critiques of the Buturovic and Klein article on economic ideology (which I participated in). We could have more of this sort of thing, and we could presumably have each of these critiques as blog posts with comment sections, methods of voting on the salience of different critiques, etc.
But is it really necessary as a gate-keeper? I don't think so.
This is not to say the current peer review process is perfect - it's not. But it seems to me the biggest problem with peer review are issues of (1.) elevating the most sexy, counter-intuitive findings and empirical tricks, regardless of whether they are the right findings, and (2.) having a bias against insignificant findings. I don't see how these would be solved with a blog-like approach.
I also imagine it would be a nightmare to respond to everybody in an equitable way. I had only two reviewers for my recent Review of Austrian Economics article, and on some issues they advocated exactly the opposite change! And this is RAE - presumably the reviewers of other journals are even less homogenous than the reviewers of this one.
So my initial reaction is to move forward with these possibilities and definitely experiment with them, but I'd hesitate to use them at the peer review stage.
Do others have thoughts?
Tuesday, August 24, 2010
Subscribe to:
Post Comments (Atom)
Posts
Peer is merely step say four in the process of the reviewing, etc. epistemological claims (particularly novel claims). It has become the focus of far too much importance in the scientific process. Anyway, peer review has been the subject of some controversy lately due to the level of autoplagiarism, etc., so this proposal may be a reaction to that.
ReplyDelete