Normalizing paper rejections: Thoughts on scientific writing
Scientific writing is one of the most important skills for a researcher. All achievements in a research career – paper publications, grants, promotion – all require convincing writings proving your contributions. So, how to write well? After a year of writing paper, I synthesize my writing lessons into two tips.
Writing tip 1: Use reviews as guiding questions
For effective scientific writing, I believe the core task is to derive at good guiding questions. That is half way towards having a good writing. All paragraphs must answer some guiding questions, there should be no redundancy. The challenge lies in having effective guiding questions. I think there are two practical ways to do so.
Start from the criticism of the reviewers
By reviewers, I mean the conference reviewers of your earlier submission of the paper. In principle, they represent the scientific standard that you need to uphold in your paper. They are a mirror reflecting all the strengths and weaknesses in your work in a highly objective way. Therefore, there reviews, even though sometimg heart-breaking, is arguably the best source of guiding questions to write the next revisions, and to improve your methodology.
This is why authors should normalize paper rejections. I used to take paper rejections with a fixed mindset, i.e., I thought the rejection means “your idea is trash, don’t ever think about publishing it here unless you redo the entire thing”. But as I said above, a paper rejection always comes with valuable feedback. Sometimes, getting rejected is the only way to gather such kind of feedback. The fact that the paper gets rejected is actually not that horrible because they literally give you a good set of guiding questions for the writing (and doing) of the next revision.
Make up the reviews when not available
In one of my earlier research experiences, I saw my advisor reasoning about our project by walking in the shoes of the reviewers. They kept asking “what would the reviewers say if we do this?”. I was puzzled by that practice, thinking that it was too practical, that good researchers should not work to please only a few reviewers, but should be about exploring new ideas and establishing new knowledge that is universally true. But now, I think that is a very good heuristic for reasoning about the work – the reviewers are a concrete proxy for the scientific audience who you actually interact with during the review process.
Writing tip 2: On how to go from the guiding questions
Once having the guiding questions, what are the mind tips to answer them?
First, be honest about the work. It is very tempting to oversell your contributions and handwave criticisms about your issues, such as those about generalization. That causes biases, making it hard to be honest about your work even when you want to.
To keep me honest, my advisor usually asks me about what I try to claim – “Are you really confident to say this is a challenging benchmark? That this is a strong model? That we outperform SOTA? That we have conducted comprehensive experiments?” Only when I can really convince myself about those claims that I may be able to convince the scientific audience.
Second, establish clear scope. This is usually a next step when being honest with our work. We may realize that our contributions are not as big as we thought. Keeping claim so would only make you receive more criticisms. Dr. Trung Phan (FUV) has once told me that, an appropriate statement on your scope of work is like a magic headband that protects you from crticisims (think of Wukong). When you really stay true to your scope, no one can say that your contributions are inadequate.
Moral of the story is that good writing is actually about good thinking. And to think well, be humble and learn from criticism. Don’t make a big deal out of paper rejections.