Tips about writing systems papers

by Lin Zhong, May 2015


If I can give one tip, that would be: respect your readers.

Put yourself into your readers' shoes

It is important to visualize your readers and how they read your paper. You hope they would enjoy your paper like you enjoyed the Harry Potter books; you hope they would study your work like you studied your Physics 101 textbook; at least you hope they would read your paper like how you read a classic paper recently. Such hopes are completely false, because nobody reads technical papers for recreation, you are obviously not writing a textbook, and more importantly, your paper is not yet published, let alone being a classic.

So how does a reader read your paper? Or more precisely, how does a reviewer read your paper? They try to finish the job as fast as possible. Peer-review is a service that does not pay. Reviewers try to understand the work as fast as possible so that they can do their job: write a review and decide if they want to champion for reject, accept or they don't really care. Often they have a dozen or so papers to review and likely are already behind the deadline. So you can imagine their most likely mood at this point: impatient, perhaps irritable. Well if you are really really lucky, one of your reviewers is reading your paper one month before the deadline on the beach with a glass of sparkling wine. But I can tell you that rarely happens.

Importantly, top publication venues usually have a very low acceptance rate, accepting one out of five submissions or less. That means the majority of the papers read by a reviewer are likely to be rejected. Therefore, it is not unusual a reviewer starts reading each paper with a subconscious bias looking for reasons to reject it. So much so that many TPC chairs these days try very hard to guide the reviewers to read submissions with a positive mindset and to focus on identifying values, instead of flaws. Until they are successful, you can safely assume your reviewers are hostile in that they are looking for reasons to reject your paper. Your job is to win them back.

In the rest of this article, readers include reviewers, the very first outside readers of your paper.


General techniques

Once you decide to make readers' life easier, you would immediately realize there are numerous ways, each of which, of course, requires you to spend more time with your draft. Below I summarize a few useful general techniques, roughly categorized as presentation and logic.

Presentation

Logic


Specialized techniques

Now I describe tips for important structural components of system papers.

Title

A good title captures both the problem addressed by the paper and the novel solution contributed. To avoid a title too long, it often suffices to capture the problem only.

Abstract and Introduction

Many readers only read the abstract and introduction. Many a reviewer makes up their mind after reading them. The introduction is so important that even the most hands-off advisors will read, revise, and sometimes rewrite it. I suspect that's a major source of surprises and inconsistencies. The introduction should summarize the entire paper crisply and bring out major points you want readers to walk away with.

Background

Unless you are writing for a very narrow community, it is very likely your readers do not have all the background to appreciate your work. So educate them. A nicely written background section is always enjoyable. Indeed, learning something new so quickly is one of the perks of doing reviews. The challenge is: the background section must be self-contained, concise, clear. This often requires a deep understanding of who your intended readers are. For a paper submission, it often helps to take a look at the program committee and imagine which members are likely to review your paper.

Design

For a systems paper, it is often nice to describe design and implementation separately. The design section often presents ideas that are agnostic to any specific implementation. These include the overall architecture, principles and algorithms.

A common mistake of the design section is lack of structure, e.g., a section of seven or more subsections each describing a component of the design. As mentioned above, the lack of structure often results from the lack of focus or abstraction. For example of lack of focus, the authors simply describe each system components, regardless of how interesting/novel they are. If the system has a lot of components, this strategy leads to a flat section with many subsections. One naturally solution is to use a subsection to provide an overview of the design and then use subsections to explain only important components that are novel and interesting. Presenting the design by component is often a sign of lack of abstraction. System component is just one possible dimension along which you can present the design. There are other dimensions, e.g., design objectives and principles/invariants followed. Discovering these dimensions require you to think about your design in an abstract way.

Another common mistake is failure to place a design choice in the context, e.g., simply describing it. There are two types of context that add depth. First, how is your choice related to prior work? Perhaps no one has solved the exactly same problem but it is highly likely others have faced similar challenges in solving a related one. It is important to be aware of how others made the choice and explain how their choice affected your choice. This is also a perfect place to give prior work credit. Second, what alternatives have you considered? It is highly unlikely your choice is straightforward or obvious (if it is, you should not spend a lot of space highlighting it). Share with readers what other choices you have considered and why you passed them in favor of the one ended up in the design.

Implementation

The section of implementation should not be a boring description. It should present interesting things about your implementation. If there is nothing interesting, the section should be very brief, providing the necessary information for readers to assess the quality of your evaluation, which is presumably based on the implementation.

What are the interesting things about the implementation? As the implementers, you should know. For example, any nontrivial challenges have you had to overcome? How does the implementation have to deviate from the design due to practical limitations? Any platform-specific optimization?

It is important to let the readers know how significant the implementation effort is. You can quantify it with the number of lines of code and the number of man-hours.

Evaluation

Since this article is about writing, instead of research itself, I will focus on writing, assuming you have done all the right things in evaluating your work.

Related work

The related work section is important in many ways. It sets your work in the larger intellectual context and explains how it advances the state of the art. For novice readers, it points out a quality collection of good works that they can follow up. For expert readers, it is the perfect place to check the authors' mastery of the topic area. For the authors of the cited work, it is where their own work is discussed, credited and criticized. A well-written section of related work always wins respect from its readers. Below are a few tips.


Recommended books

I ask all my students to buy and read the following two books:

The following book is more specialized for Computer Science but I have not read it as carefully as the above two: