My major life update: I submitted my first research paper as a PhD student.

Context

  • For those who don’t know: PhD students are supposed to produce publications. That’s why we are generously supported financially by the schools.
  • I am studying in Natural Language Processing.
  • We submitted our paper to COLING.
  • I had co-written 3 conference papers before this (one accepted to SIGCSE, one rejected at ICSE, one rejected at AAAI Undergrad Consortium, IJCAI/ECAI, and NeurIPS). I will compare this experience to those.

The experience

My co-authors are editing the paper with me on Overleaf.

My co-authors editing the paper with me on Overleaf. The green one is my advisor. To me, it feels like you’re playing tennis with Roger Federer.

It took a huge amount of time. During the last week before the deadline, I shut down all other responsibilities, including classes and teaching assistantship, to focus on the completion of the paper. Everyday, regardless of my wake up time, I knew I would spend 12 hours, if not all waking hours, of that day for the research. On the last two nights, I went to bed at 3am and 7am, respectively.

  • This is unhealthy, but worth it. We originally didn’t know if we could submit the paper on time. But when we did, it felt good because we surprised ourselves.
  • However, I absolutely want to avoid this working pattern for future projects. My advisor told me that this is possible. We just need to be more strategic.
  • It was pretty stressful. We had a high hope to submit to this conference because this is relatively the “easieast” venue in the following 6 months. If we missed that, we would be likely to wait for another 6 months for the next chance. So the stake was quite high. I really started writing about 3 days before the deadline because I was occupied with running experiments before that.

As I was pretty average in ML engineering, I struggled to make the computational experiments that I was in charge perfect.

  • I failed to be fluent in quantization and weight off-loading (to fit large models in small multiple GPUs)
  • I failed to have a working implementation for fine-tuning vision-language models.
  • I still feel like not speaking the language of the open-source LLM world when it comes to downloading a random model checkpoing, wrapping it inside a HuggingFace class, change the interface to Pytorch, etc.
  • These things have actually happened during my ICSE paper. Although I made very good progress in learning about training a LLM, the experimental results were not good. Later, a co-author told me there were actually bug in the code.
  • I honestly don’t feel too bad about this. I know being principled and scientific is more valuable and rarer than being fluent with the technicalities. But I need to be a better hacker for sure. Maybe to learn it fast, I need to care less about the principles and just do more practice?

This is my first time writing the majority of a conference paper.

  • My advisor allowed me to lead the writing although I never be in that role before. I was both worried and excited. When we finished the paper, my advisor told me my writing was not bad and he did not have to “rewrite every sentence”.
  • When writing, I feel like summoning a broad set of writing experiences I had at my undergrad.
    • I first academic writing was in a pre-college course. I wrote about the linguistic landscape of a special building in Ho Chi Minh City. I learned about the overall structure and how to develop each section, about citations, and most importantly, about the research mindset.
    • I of course learned academic writing a lot more during the next 4 years of undergraduate. But the most prominent experience was in Scientific Inquiry by the beloved Dr. KinHo Chan (who researches in Neuroscience). I actually learned about the point of doing science before learning about scientific writing. As in everything, Kinho is very concise in his writing. He told us the 4 fundamental parts of a scientific paper – Intro, Method, Results, and Discussion. Literature Review and Background belongs to Intro. Conclusion is kinda not important. In Results, we let the data speak, while in Discussion, we interpret the data.
    • Other courses where I learned to write and got specific training in just effective writing: Rhetoric, Visual Studies, Ethics and Moral Philosophy, Digital Anthropology, “Language, Culture, and Thoughts”, etc. All of them are social science and humanity courses. Surprisingly, those experiences was highly useful duing this COLING paper because I had to analyze specific memes and interpret them, which requires subtle and grounded observations, a respect for subjective interpretations, and an objective voice. Fulbright’s education philosophy was correct in this aspect.
    • I really get into CS-styled writing during my REUs.
      • This game playing paper was I my first real research, where I did not write much but instead observed my advisor writing. He has a beautiful way of telling the story and weaving different pieces in our projects together in a scientific way. I also learned his nonchalance about writing even the deadline is close, and his writing techniques (e.g., leaving outline or ideas in \textcolor{red}{<content>} within the Overleaf doc).
        • To remark, I enjoy learning to do science with him. He has a non-hustle vibe so that he does not publish too frequently. In fact, he chooses to be a good teacher and goes to a liberal arts college. However, he still has a great curiousity that makes great scientists. His “math language” is also very effective.
      • Another unpublished paper in software engineering was when I actually allowed to write something. I got positive feedback during writing that. But we were in a rush, so the final paper was not that well presented.
  • I want to say I like writing, but only when I have things to share. Like this blog. Or journals (where I share with my future self). Because I have been holding a lot of ideas for a while, these informal writings is a good reinforcement learning environment for me to train the “vec-to-text” module of my brain.

The Aftermath

When the deadline was over, my head became sooooo light. I felt my life became so much more peaceful and stress-free. It the feeling like the pause between in-breath and out-breath been you’re breathing deeply (yeah, try it).

I guess my life in the coming years will be this kind of seasonal, which should be fun. I am excited to grow my technicalities and scientific mindset in the coming projects.