Re: Research into Wikimedia content and communities

Actually, I don't think the visibility of public reviews would be likely to result in higher quality
reviews. Blind review exists (at least in part) to protect the reviewers and enable them to provide an
honest opinion on a submission without having to worry about the retribution for a negative review (e.g.,
negative reviews on their work in return, not getting recommended for funding or promotion, etc.). Such
worries would be particularly problematic for junior faculty or students. I would expect signed reviews
to be much more anodyne. You might also imagine the reverse, that someone might submit a falsely positive
review hoping for some reward down the road. Blind reviewing does create some problems, but AFAIK it's
nearly universal in academic publishing. It would be easy to permit or re
 quire anonymous comments though. Or perhaps revealing the names only if the paper is accepted. 

Double blind review (meaning that the authors' names aren't known to the reviewers) would also be more
difficult in this system. The intent of double blind reviewing is to encourage reviewers to review the
paper and not the author. On the other hand, it's often pretty easy to guess who the author is even without
the name attached to the paper, which may be why double-blind reviews are not universal. 

It's also worth noting that the main problem with peer review is getting reviewers at all, since there's
little reward for reviewing and it takes a fair amount of effort to write a good review--note that a paper
might be 40 pages and a review several pages long (at least in my field--there's a lot of variation in
publication norms from field to field). One argument for reviewing is that you get to read papers earlier,
but if everyone can do that anyway, then there's not much incentive to spend the effort crafting a careful
review afterwards. It could be though that the volume of comments or the discussion among commenters
would compensate for less depth in any single review--it would be interesting to see how that balanced
out. Given the small size of research communities and the ratio of 
 readers to contributors on Wikipedia, I wouldn't expect a flood of comments on papers in most
subdisciplines though. 

On 22-May-2012, at 6:46 PM, FT2 <ft2.wiki <at> gmail.com> wrote:

>  3. A key change would be that reviewers' identities would be public.
>   Although this would remove the usual complete separation of author and