ResearchGate, preprints, and open access

I am not much of a social media person, so I don't usually sign up for new social sites, even when work related. Recently, I set up a ResearchGate site for myself, mostly as an experiment. A colleague of mine swears by it, and claims that his citations have really increased since he started using the site. It is an easy experiment to try, so I just set up my page, and let it go. So far, nothing I have seen convinces me that more people are reading my publications, but it is early days yet. Also, I am not illegally uploading my publications, so there is that. If I am still blogging here, I'll revisit my ResearchGate experience in a year or so.

My field is not really one that uses preprint servers much, though I have no real objection to doing so. I much prefer to read the nicely formatted journal versions (when available), rather than the preprints on the arXiv myself. I think preprints are a good idea, but peer-reviewed journals serve a beneficial purpose both in acting as gate keepers for junk (and I think anyone who has done a review knows that), and also in improving manuscripts. All of my own publications were improved during the peer-review process.

I have a few open access publications, and they are not cited at a higher rate than my other publications. All the data I have access to seems to confirm my own experience that it is not hard to get access to a paper, even if ProdigalU's library doesn't have it. Legally even. Interlibrary loan works, but it it slow. Faster is to just email the corresponding author. In my experience, they are happy to send a pdf (which is usually fine under the licensing terms--I have never even come close to sending out as many as I am entitled to as an author). Not instant gratification, but still pretty easy access to the literature. And that is before the quasi-legal or downright copyright violating methods. The actual fastest method is to just google the title, and often a non-pay walled link will show up. I just don't see access to publications as a major issue for most people with an Internet connection.

I get that some people have open access as a near and dear issue in their hearts. I get that extortionate journals are a problem, and that libraries are being squeezed by publishers to take on (and pay for) crap journals they don't want to get the ones they do through packaging. The thing is, open access doesn't solve this problem and adds new ones. Quite frankly, I can't pay page fees. I just don't have the money. Even if I did, there are not many non-predatory open access journals in my field, so I would just be paying more money to those same journals already extorting my library. Furthermore, at this stage in my career, the work that can go into high impact journals needs to go into high impact journals (mostly for the benefit to my CV), which means no open access.

Interdisciplinary science

This study in Nature looking at grant success rates for the Australian Research Council's Discovery Programme over a 5 year period confirms with data what has long been suspected--interdisciplinary projects are less likely to be funded, and the effect is stronger the more interdisciplinary the proposal is. This is an issue not only for those of us who do a lot of interdisciplinary science (who of course want to be funded), but also for science in general, since more and more modern science is at the interface between disciplines.

Anecdotally, what is true for proposal evaluation also appears to be true (for me at least) in getting manuscripts published. My more interdisciplinary work needs to be shopped around a bit at different journals, sometimes 3 or 4 times before even going out for review. In contrast, my work that fits into a "traditional" discipline is apparently easier to match to a journal, since it tends to go out for review straight away (even if it isn't accepted at the first journal).

It is much harder to find appropriate reviewers for interdisciplinary work--I often end up recommending a list half in one field, half in another. Even so, referee reports often come back with serious misconceptions about the parts of the manuscript that are obvious out of the referee's area of expertise. The system of using 2 referees means that if I am lucky, I will get referees with 2 different areas of expertise. Alas, more often both reports are from reviewers in the same general "traditional" area, who then either ignore out of area issues, or don't appreciate the novelty, difficulty, or significance of the out of discipline results. It is hard to know how much of this is caused by my issues (writing style, not a strong enough introduction, lack of clarity, too much in love with the data, etc) and how much is lack of core knowledge in the referee at times.

If this is the case for manuscripts, it is almost certainly also the case for proposal evaluation. Moreover, I suspect that interdisciplinary proposals have a harder time attracting a strong advocate who can sell the research to the rest of the panel. Having served on panels myself, if a proposal does not attract a champion, it can be easily overlooked even if the science is top notch and the writing is clear.

Unfortunately for me, the problems I am interested in and the methods I use to solve them are highly interdisciplinary in nature. I often collaborate with colleagues in other subfields and departments. A long time ago, I realized that many people would be extremely skeptical of some aspects of my work. I am confident that the research I do is exciting and important--the problem is in getting others to see it, of course. To counter out of hand rejection of some of our admittedly very unusual research combinations, a good cover letter is crucial. I also find that it is extremely important to attend conferences and have student attend conferences so that potential reviewers see the work presented BEFORE they get a manuscript or proposal to review. Hopefully, that little bit of familiarity helps establish enough benefit of the doubt for people to read the work with an open mind, rather than dismissing it out of hand. With the increasing number of manuscripts and proposals submitted, there is less and less time to consider a manuscript/proposal, so snap judgements are important. As an aside, a side benefit of doing unusual research is that we don't worry much about being scooped on our very interdisciplinary stuff no matter how much we talk about it prior to publication.

I am not sure what to do about the problem of evaluating interdisciplinary work, but this will be an ongoing problem for the scientific community. Anyone else have strategies for helping others appreciate the novelty/beauty/significance of their work?

Things "everybody knows"

You don't have to look far on the Internet to find people complaining about peer review and all of its frustrations and flaws. My peer review frustration of the day is when reviewers come back with "everyone knows this" or "this is the expected result" and don't back up the assertion with any form of reference or previous literature. If there is something that "everybody knows" AND it is actually backed up by experiment (and not something people just assume), then it should be easy to provide the authors with a reference to prior work. If the result is something people have assumed for years, but is not (yet) backed up by data in the literature, then the experiment is possibly worth publishing, assuming the methodology and analysis are sound. I find this situation annoying both when I am the author and also when I am one of the other referees and the editor sends everyone the reports (something I whole-heartedly support--a good way to help calibrate reviews, and I often learn something from reading the opinions of people with complementary expertise).

I see this most frequently when the work is interdisciplinary, or when someone publishing is new to the field. New researchers don't have the biases and inherited wisdom of their predecessors, and are in a good position to question assumptions. They may also be bringing new tricks to an older problem that illustrates things taken for granted. In my own reviews, I try to provide at least one reference when I comment that a result is not new, or is expected based on previous work. Yes, the authors should do a thorough literature search, but sometimes people miss things, and if a result really is widely known, it takes less then 10 minutes to pull up an appropriate reference.

In my own research, I find that pretty much whenever I move into a new area, there are things that "everybody knows" that follow most people's science intuition, but are completely unsupported experimentally. Sometimes those things are trivial, and no one really cares, other times they are foundational to interpreting results or designing experiments. Probably 9 times out of 10, the results of an experimental test will mostly align with the expectation. But the real fun comes when the results are completely unexpected, and that is why we do the work. Confirming or denying a hypothesis is what research should be all about. As a reviewer, call that 9 times out of 10 result incremental if you want to (which often it is), but don't say it isn't a new result, even it is something "everybody knows". Now we have experimental confirmation that "everybody knows" something that is actually correct.