Darryl McAdams sends me this link
from the American Statistical Society. It discusses a possible rather attractive alternative to
the traditional academic publishing regime. In particular, it explores how the suppleness of
the blog format might be used to enhance dissemination of novel results and
improve the quality of discussion by encouraging useful critical commentary. The contrast between Jane 2.0 and her
unfortunate 1.0 avatar is striking. The
latter is disadvantaged in myriad ways: she is cut off from interesting peer
commentary, has to wait an excessively long time for reviews, is limited to
submission to one journal at a time and only gets her work widely read if
published. Jane 2.0 is far better off in virtually every respect. A further benefit of 2.0 is
that it re-empowers the community that does most of the work: the research community
that reviews the papers and which cares about the work (as opposed to the
publishing houses that are mainly interested in turning a profit). I think that
open source journals are where academic publishing is (and should be) heading. Jane
2.0 seems like an attractive alternative to what we have now. I say this
knowing full well that the utopian vision 2.0 describes will surely engender
problems of its own.
This is happening already in some fields. For example, in geophysics there are some journals that have sort of a two-tiered format. First, one submits a paper to Journal Part A. The paper is minimally checked for adhering to formatting guidelines, etc. The paper is then opened for online, non-anonymous commenting for some number of weeks, and I believe there are a small number of "designated reviewers" who are required to comment during this time. Some of the reviews are even so detailed that they become squibs/papers of their own. After the review period, the author then revises the manuscript and it is published in Journal Part B. I am commenting in vagaries here because I'm going on my half-remembered impressions from having this process explained to me by friends who have gone through it, but if anyone is curious about the details, I can get them. I leave it as an exercise for the reader to think through both positive and negative ramifications of this peer-review format.
ReplyDeleteI think this would be an extremely positive sign of the times. Coupled with current open access journals (which could be the place where papers are published in their final form), the web-based discussion forum/review process is a very good idea. That's what conferences were for in the beginning, when people had no other choice if they wanted to discuss something in real time. The legacy and power of the big publishers are the only things keeping something like this from happening. Of course, without research(ers) there are no publications, so maybe it is possible to change the picture.
ReplyDeleteRemember the episode of the animated Star Trek where Spock goes back in time through the Guardian of Forever and saves himself as a child, thus allowing himself to live on to later make said time-travel rescue? (No? Really?) Perfectly coherent state of affairs once you're in it. Question is, how does one **reach** this state of affairs? If young Spock is _first_ eaten by a fearsome creature in the deserts of Vulcan, then he can't go on to save himself. If young Spock is first _not_ eaten, then how is it that when older Spock goes back in time young Spock is about to be eaten? If young Spock is saved by older Spock, no problem. But you don't seem to be able to get there from here.
ReplyDeleteThis strikes me as similar. This is a great suggestion, much better than the suggestion that we should ditch peer review and start posting papers on LingBuzz or aRxiv - that's just the worst of both worlds. But once we ditch the "clearinghouse for papers" format and start putting things in a discussion format, then we're talking. People follow links, people comment on blog posts. There will be pesky loose ends to clean up (how do you evaluate someone for promotions, jobs, etc? what is the incentive for people to actually follow up and make the changes suggested?) - but that's not the main problem. The main problem is how you get there from here. Because at the moment you do not want to do this with your research. If your research is good, you want to hide it until it can get published as a paper (or a conference paper, whatever your field fancies) and make some meaningful impact on your career. If papers don't matter anymore, then fine, people will do this. But papers do matter now. Right?
This comment has been removed by the author.
DeleteOn said note, Norbert, I wonder: why don't you do this experiment? See if it can work. Right now you're posting high-level musings that you admit sometimes meander. Post a genuine down-in-the-weeds type deduction about syntax, worked out, with some genuine judgment data. A squib-size thing. Doesn't have to be spelled right, but it has to be digestible and sensible. Then see what people do. If people don't provide helpful (i.e. reviewer-level or better) comments, trash it, suggest further data, etc, to the point where you feel like if you clean it up it's journal worthy, then, end of experiment. If it's successful, we start calling you "Norbert 2.0".
DeletePart of the point is that this sort of thing solves all those problems in one fell swoop. Tim Gowers has a nice extended discussion of this.
Deletehttp://gowers.wordpress.com/2011/10/31/how-might-we-get-to-a-new-model-of-mathematical-publishing/
The point is, you need to make sure the thing you build (the thing I'm building! :)) can act as a transitional tool, and as a destination for the transition.
As mathematicians and physicists and so on are well aware, people are *already* posting stuff to arXiv prior to publication. It's not a matter of *should* -- it's already happening. Journals aren't *going* to give way, they've already given way, and we just haven't realized it yet, because there is no coherent replacement. We have blogs and arXiv and Reddit, but none of these things really works the way it ought to, not right now, because all of the important stuff is just too scattered. arXiv doesn't let you give feedback nor does it let you aggregate, blogs don't let you promote nor does it let you aggregate, and Reddit is just a giant blob of immaturity that can't be used for serious science. And none of them have very good feedback systems even when they *do* let you give feedback.
So we're in a transitional period, where journals are demonstrably dead, but nothing has taken their place.
We'll see how far I can get, and how much promoting I can get real names to do. :)
Ah, why not? I will consider doing this in the near future. Frankly, I am running out of musings so maybe this will help matters.
ReplyDeleteMore seriously, the main reason for avoiding shop talk has been to try and make the blog directed at the larger intellectual issues that I feel have no place for discussion. We have venues for discussing "serious" syntax, but few to discuss the larger animating ideas, that, I at least, believe make the field worth doing. If I do follow your suggestion it will likely be on another blog entirely. Maybe I can call it, with your permission, Norbert 2.0.
Stefan Müller recently received a grant from Die Deutsche Forschungsgemeinschaft (DFG) to try to set up an entirely open access publishing system (with no author fees!) for linguistic books. I know this post seems to be more about journals, but, if the project comes to light, it seems like the developed software could also be used for journal publishing.
ReplyDeleteThe proposed project, Language Science Press, would function as a full-fledged publisher, and one thing that they talk about in the grant is the "gamification" of the review process. I'd recommend reading the grant, as they describe it in more detail, but the software that they are trying to develop would essentially open up the review process to the (linguistic) community---in addition to the standard blind review that a monograph would go through before being accepted for publication as a book---which might help achieve some of the Jane 2.0 features that your post and the link talk about.
For anyone interested in the project, you can sign up as a supporter (and receive updates about the project), and you can also donate to the project.
(NB: I'm not affiliated with the project in any manner (other than having signed up as a supporter), so this isn't meant to be an advertisement; I just thought that some of the followers of this blog might find the project interesting/worthwhile.)