from Guide to the PhD on Aug 13, 2023

How to read papers

I remember being asked to conduct a "literature review" for the first time many years ago. I had so many questions: How do I read papers? What am I looking for? Why is this paper "Best Paper"? Heck, how do I find papers? Here's the guide I wish I had.

A large part of conducting research is understanding what state-of-the-art even is, to make contributions to the field. The matter is further complicated by highly-opinionated researchers all with different ideas of what state-of-the-art is. Fortunately, especially with recent hype in AI, there are more and more accessible in-roads to keeping up with research developments.

This is the guide I wish I had, for reading papers. Rather than write for my past self, I'm writing for a modern version of myself, if I had started research today. Fact is, news, tools and accessibility of the field broadly has drastically improved. We'll go over how to maximize this, in this post.

How to find papers

Generally, paper searches shouldn't be done alone. Rather than seek out the most relevant papers in a field, you should actually seek out relevant experts or curated lists.

  1. Find an expert colleague. As a PhD student or a researcher in a large industry lab, this is more straightforward: There's bound to be some colleague with expertise in that field, somewhere in your organization. There are two ways you can achieve this: through word-of-mouth, or through a list of publications from your organization.

    1. Ask your manager or adviser. Back in 2020, I needed to learn more about explainable AI broadly. This was fortunately simple to do: My adviser recommended I consult lab-mate Lisa, who had been working on explainability methods for a year. Right after, we chatted in lab, and Lisa then suggested I speak with Prof Bargal, who had built a career in explainable methods. This outreach had the side benefit of a collaboration with both experts, culminating in an ICLR 2021 publication.
    2. Browse publications from colleagues. Now at Apple, I browse publications that are specifically authored by colleagues at the company, via forums such as In short, I'll treat publications as proxies for expertise, then contact authors for the relevant field. Fortunately, given the size of the company, there are plenty of experts to consult.
  2. Find sources of expertise. Rather than find papers directly, look for troves of papers that are peer-reviewed or otherwise filtered by a curator of some kind. This can come in multiple forms, either in traditional academic forums or even via social media.

    1. Known peer-reviewed venues. If you have no papers to begin with, start from a known venue or publication. For machine learning broadly, see ICML, ICLR, NeurIPS. Or, lookup conferences by field, such as computer vision (CVPR, ICCV, ECCV), graphics (SIGGRAPH), NLP (EMNLP), robotics (ICRA, IROS, CoRL) or systems (EuroSys, MLSys).
    2. Popular social media reviewers. Several folks on social media are known for consistently promoting good work. There is certainly noise in entrusting curation to a single account, but any curation is helpful when dealing with a firehose of information. There are known industry labs (@GoogleAI, @DeepMind, @OpenAI, @MetaAI), university labs (@StanfordAILab, @berkeley_ai, @MIT_CSAIL, @CMU_Robotics), and known curators (@AK).
    3. Use awesome lists. "Awesome" has become codename for "comprehensive list on a topic", on Github. Not to mention, there's an awesome list of all the other awesome lists; here's the meta-awesome list on Github (link). You can also simply Google Github awesome <topic> to find lists on almost any topic. For example, I used awesome-pruning while doing a literature review for my recent ICML 2023 paper.

At this point, you should have a set of candidate papers to consider, either from experts or other sources across the web.

Why read papers

There are a variety of reasons to read a paper, but there are three broad "personas" you can assume. Each one requires a different type of reading or skimming.

  1. Read as a critic, to assess the paper's validity. The primary goal is two-fold: (1) Find the core claim in the paper, and (2) check that the core claim is supported by the experimental results. This is the core function of a reviewer, and for publications that are not yet peer-reviewed, this is your role as a reader.
  2. Read as a user, to build off of it. The primary goal here is to understand the key bits of the method and results, just enough to use the work effectively. For example, in my own research, I accelerate model inference speed. To do so, I'll read and reference "fundamental" papers like Attention is all you need repeatedly — not to pick at the paper, but to find details in the architecture that are hardware-unfriendly and need some post-hoc optimization.
  3. Read as a competitor, to compare against it. The primary goal here is to position the paper relative to other work in the area, meaning the motivation in the abstract, introduction, and related works are key. Often times, your goal is also to position the paper relative to your own work. This is a part of assessing the paper as well — to understand how its contribution adds to the pool of knowledge in a field.

For the most part, reading as a user or a competitor is straightforward: For the former, you already know which parts you're looking for. The goal is to generally understand the method. For the latter, your goal is to find a weakness to exploit — in the motivation, the method, or the results. In light of that, we'll focus below on how to read papers as a critic.

How to read as a critic

As a critic, your main function is assess the paper's main claim; there are four components to this task. You can treat this as four steps in an assessment. If the paper does not pass any stage, there's no reason to continue onto the next stage of assessment.

Function #1. What is the main claim? At this stage, your goal is to simply identify and regurgitate the main claim — not to debate its worthiness or assess its validity. Here are several pointers for identifying claims.

Function #2. Is the claim valid? There are several requirements for a claim to be valid.

Function #3. Is the claim intuitively true? Look for an intuition for why the claim is intuitively true; even a hypothesis works. As long as the paper doesn't appear to throw at the wall and see what sticks. In this case, there isn't much insight that can be carried forward: the paper reads like a laundry list of random techniques and results. See Is my project paper-ready? for a more detailed description of the insight.

Function #4. Is the claim supported? Once you've found the intuition, your goal is to then see if the experimental results support the main claim. There are two aspects to assessing experimental results:

Notice a curious property of the above four components. They mirror the typical first four sections of a paper: Introduction for stating the claim, related works for validating and situating the claim, method for explaining the claim's intuition, and experimental results for supporting the claim. This particular framing helps you understand what to get from each section of the paper. I talk more about the purpose of each section in How to write a paper.

There are certainly other aspects to assessing a paper but these other rubric items are all secondary to the above.

With that, you should now be equipped to read as a critic. Use this as a reviewer for a venue, a reader for a peer's submission, or even for your own paper.


As a researcher, staying on top of the field is a must. Use the tips above to keep afloat, but also recognize that you don't need to always be on top of day-to-day Twitter threads. The goal is to keep tabs on state-of-the-art just enough to situate your own contributions to the field.

This ties into my final tip: Conduct periodic deep dives into related work when it comes time to write, publish or publicize your work. Concentrate your reading into specific focus periods or schedule a 5-minute reading period each morning. Don't waste hours each day just skimming; there's much more content on arXiv than you could possibly catchup on.

back to Guide to the PhD

  1. Note this practice of presenting a central idea and hiding the details is not always a good one. Several Google researchers tend to take this to the extreme. For example, EfficientNet and its subsequent papers repeated this practice to an egregious degree, leading to very poor reproducibility of the paper. With that said, for a number of years, EfficientNet exploded in popularity simply because other researchers used their pre-trained checkpoints instead of reproducing them.