A real (and perhaps not uncommon) anecdote: Two sets of researchers, using a respected national longitudinal data set, have been working on related but distinct topics, employing mainly descriptive approaches in their research. In the interests of informing their own work, one group asked the other to share its paper. Reading it, they see that some findings look unrealistic at best, if not inaccurate. While continuing to work on their own paper, they attempt some replication. The numbers are, in fact, almost certainly wrong.
The paper is soon to be published at a journal regularly read by both groups and many of their peers.
The second group finds itself in a quandary. Convinced that the findings of the first team are erroneous, what, if any, actions should they take? What is the appropriate professional response, if any?
This is a useful anecdote for thinking about sharing knowledge, professional expectations, scarce resources, “public good” aspects of academic inquiry and professional responsibility, especially because neither the data nor the methodology is in question. (The disputed findings involve the mean of a commonly used descriptor variable.) It also comes up at a time when replication of research is a very visible issue in academia and beyond.
There are several stakeholders:
- The academic and professional community—in this case, economists and demographers—and public policymakers;
- The journal that accepted the paper;
- The researchers who wrote the paper;
- The other researchers who read the paper while working on their own (related) paper, which they want to publish in an academic journal.
Professional ethics and, perhaps, courtesy might suggest that the second group immediately notify both the members of the team that wrote the paper and the editors of the journal that accepted the paper. This would remedy publication of misinformation. In addition, it might save embarrassment to both the journal and the paper’s authors, and even to the field of those who rely on such research.
In an ideal world, the researchers who wrote the paper, the journal about to publish it and the larger community likely to read the article, would appreciate the second team’s intervention.
Yet, the second group faces mixed incentives. On one hand, they support academic integrity and professionalism. However, if they notify the first team, who did not ask for a response before or after sharing their paper, or the journal in question, they would likely be asked to provide detailed feedback requiring them to enlarge the replication they did after reading the paper, which would involve additional data analysis on their part. To the second team, being obligated to re-analyze and replicate the first team’s research would interfere with and distract them from their own research. From their perspective, doing their own research and getting it published best serves their interests; re-doing others’ work, especially with only secondary credit at best, is an unattractive alternative.
On a deeper level, the second team faces a number of hazards. Neither the first team nor the journal is likely to welcome their information, even if ultimately the larger professional community might, especially as the paper is about to go to press. Their comments are likely to be perceived as (and actually be) disruptive to the review and publication process.
In addition, and perhaps more worrisome, efforts by the second team to share conclusions about the validity of the first team’s research may be perceived as competitive rather than assistive, at least initially. This is likely to be the case not only with the soon-to-be-published authors but also other researchers and colleagues in their field, as well as journal editors and reviewers. In particular, the journal that had reviewed the paper and is about to publish it would be discomfited by news of the error Alienating existing and potential colleagues, and not to mention future opportunities to publish, is not an attractive option for most academic researchers who necessarily rely on publication to further their professional goals and careers.
In the world of academic research—indeed, as in any number of professional spheres—the options available to the second team are limited, not to mention unenviable. At last observation, the second team is considering waiting until the article has been published and they have completed their own work and submitted it for publication. At that time, or perhaps when they submit their own work, they will make public their knowledge of the other paper’s deficiencies. They find themselves hoping that because “academic journals [seeking to improve their visibility and readership] love conflict,” as one team member commented, their research will be more likely to be readily published when it is completed in the near future.
Addendum: After a spirited discussion, the second group of researchers informally notified the first group of its error and expressed their hope that the authors would be able to remedy it before the article was published.
Jane (Sjogren) O’Neil has been in higher education for nearly 40 years, as an economics faculty member, administrator and consultant. She is a trustee of the E. Eugene Carter Foundation.
[ssba]