More on Scientific Publishing

John Baez has some recommendations about publishing

What can we do to keep academic discourse freely available to all? Here are some things:

1. Don’t publish in overpriced journals.

2. Don’t do free work for overpriced journals (like refereeing and editing).

3. Put your articles on the arXiv or a similar site before publishing them.

4. Only publish in journals that let you keep your articles on the arXiv or a similar site.

5. Support free journals by publishing in them, refereeing for them, editing them… even starting your own!

6. Help make sure free journals and the arXiv stay free.

7. Help start a system of independent ‘referee boards‘ for arXiv papers. These can referee papers and help hiring, tenure and promotion committees to assess the worth of papers, eliminating the last remaining reasons for the existence of traditional for-profit journals.

The nice thing is that most of these are easy to do! Only items 5 through 7 require serious work. As for item 4, a lot of math and physics journals not only let you keep your article on the arXiv, but let you submit it by telling them its arXiv number! In math it’s easy to find these journals, because there’s a public list of them.

There is much more over there, including another post.

Andrew Gelman had something to say (Eli came late) about how to deal with the speculative and sexy stuff that the top journals fall for

This is a hard problem!

There’s a general sense that the system is broken with no obvious remedies. I’m most interested in presumably sincere and honest scientific efforts that are misunderstood and misrepresented into more than they really are (the breakthrough-of-the-week mentality criticized by Ioannides and exemplfied by Bem). As noted above, the cases of outright fraud have little scientific interest but I brought them up to indicate that, even in extreme cases, the groups whose reputations seem at risk from the unethical behavior often seem more inclined to bury the evidence than to stop the madness.

If universities, publishers, and editors are inclined to look away when confronted with out-and-out fraud and plagiarism, we can hardly be surprised if they’re not aggressive against merely dubious research claims.

Gelman points out that this is not a victimless crime

it’s worth briefly reviewing who is hurt by the publication of flawed research. It’s not a victimless crime. Here are some of the malign consequences:

- Wasted time and resources spent by researchers trying to replicate non-findings and chasing down dead ends.
- Fake science news bumping real science news off the front page.
- When the errors and scandals come to light, a decline in the prestige of higher-quality scientific work.
- Slower progress of science, delaying deeper understanding of psychology, medicine, and other topics that we deem important enough to deserve large public research efforts

Eli remembers that when Fleishman and Pons had their cold fusion meltup, essentially all government scientists were told a) they were stupid and b) to drop everything and do cold fusion. Well, it raised the price of palladium, but it killed off a lot of science and cynified a whole bunch of scientists.

Dr. Gelman, is a good diagnostician, but the Rabett fears that his prescription will not work

Instead of publishing speculative results in top journals such as JPSP, Science, Nature, etc., publish them in lower-ranked venues. For example, Bem could publish his experiments in some specialized journal of psychological measurement. If the work appears to be solid (as judged by the usual corps of referees), then publish it, get it out there. I’m not saying to send the paper to a trash journal; if it’s good stuff it can go in a good journal, the sort where peer review really means something. (I assume there’s also a journal of parapsychology but that’s probably just for true believers; it’s fair enough that Bem etc would like to publish somewhere that outsiders would respect.)

Under this system, JPSP could feel free to reject the Bem paper on the grounds that it’s too speculative to get the journal’s implicit endorsement. This is not suppression or censorship or anything like it, it’s just a recommendation that the paper be sent to a more specialized journal where there will be a chance for criticism and replication. At some point, if the findings are tested and replicated and seem to hold up, then it could be time for a publication in JPSP, Science, or Nature.

Right, Science and Nature won't go for the big spash. Hohoho. On the other hand, the EGU open review system could go a long way to deal with this issue. It combines the fast availability of arXiv, with stringent peer reviews. The open reviews force referees to put some skin in the game. They are competing with one another. Even if they remain anonymous they will know if they blew it, and they can gain general respect by providing top notch reviews.

A beauty of the open review system (see EGU publications) is that you get to see the reviews. The reader can gain an appreciation of the problems as well as the strengths of the paper before or after (tastes differ) reading the paper. Difficult issues can be thrashed about in an interactive and professional forum. (Eli is thinking of a particular case which had tens of comments but he can't remember the details. It was discussed on several blogs. Eli is old and his memory never was much.)