November 6, 2022

How to prepare for research, without prior experience.

Making yourself an attractive candidate for research mentorship is hard. What even is "research"? Here is a brief guide to the foundations you need, to conduct research.

Preparation for research is effectively about building fundamentals — you don't need to be the world's expert, nor should you wait that long to begin. Having just enough understanding of the necessary components suffices. In this post, I'll skip over defining what "research" really is; this is a longer conversation, and instead, we'll focus on building fundamentals to conduct research.

Note: Examples below are tailored to AI/ML, and this bias may have bled over into the tips themselves. However, these tips likely generalize to any applied field with a coding and math component.

🔗Step 1 - Pick a field of study.

Your first step is to find a problem space, so you can focus your efforts on becoming the ideal candidate for that field. If you have no idea how to pick a field, don't worry: My advice here is to arbitrarily pick a domain and begin preparing for that domain. If you lose interest halfway, then there's no need to try research in the area. Your success in a field will be defined by your interest in that field's problems.

Your success in a field will be defined by your interest in that field's problems.

There are of course many broad areas of study, such as biology, computer science, statistics, etc. If you have not yet landed on a major, you can use your university's list of departments as a very general list of options. Within each department, you can then use colleges or the introductory classes to get a sense of broad topics. For artificial intelligence and machine learning (AI/ML) in particular, here are several problem spaces; I'll focus only on the parts of these fields that overlap with AI/ML today:

There are also many researchers that study a blend of two or more of the above fields. For example, combinations of computer vision and natural language are very common, with text-to-image models, such as OpenAI's DALLE-2, being a very hot topic in the community at the moment. In sum, pick a general field at this point. Either you have an inclination, or you don't; if you fall into the latter category, pick one at random, so you can start knocking items off your list.

🔗Step 2 - Evaluate your interest.

Once you've picked a field of study, evaluate your interest in that field. For AI/ML, you can use this 1-hour primer on "Tools to Learn Machine Learning". After this primer, you'll be much better equipped to process the deluge of information available. Here are several ways to identify interest in a field, which I suggest following in this order, either sequentially or in parallel.

  1. Hack on some code related to that field. This advice is pretty specific to the AI/ML field, where open-source code is now a de facto standard. This could be a framework tutorial such as PyTorch's object detection tutorial, a repository for general tasks such as HuggingFace's transformers tour, or a repository from a random paper like a deep learning adaptation of decision trees. To start, you could treat these repositories as black boxes, and utilize pre-trained models to build a nifty tool. This could be a gateway to exploring the problem space. As you try to deploy the model, you may run into problems: Maybe the model is too slow. Too memory-intensive. How would you fix that? Maybe the model performs extremely poorly on certain protected groups. Why is that? Or, perhaps the model is just underperforming. Toying with these repositories will help you gain confidence, learn about problems, and provide a starting point.

  2. Join seminars for that field. If you're early in your undergraduate career, you may not have room to add more classes. I suggest looking for topical seminars that your graduate department offers. For UC Berkeley, you can find seminars and classes for computer science and electrical engineering. You don't need a fully fledged class, and a weekly gathering for research talks suffices. Then, simply see which presentations interest or bore you. I should caution you that I find this method faulty — evaluating interest in the field using interest in talks. This is mostly because you're relying on engaging speakers to spark interest. However, I have several peers who picked their fields because of that one engaging, enthralling speaker. So, you could try your luck using this method. Sign up for a bunch of seminars initially, and after a week of talks, keep only the most interesting ones. Be sure to drop the ones you don't want!

  3. Read paper synopses from that field. This was not the path I took, as paper summaries were not as prolific in my undergraduate. However, you can find distilled and sometimes sensationalized research summaries on social media — popular sources include @akhaliq, and @TwoMinutePapers. You can also find high-level research perspectives and random research highlights from famous researchers like @karpathy. In the past, Twitter has been a fairly good stream of papers, especially after conference submission deadlines, as submissions are posted on Twitter. This may no longer be true when you read this article.

  4. Apply willy nilly and let fate decide. This was my technique of choice. I took classes that led me the quickest to graduation and haphazardly emailed a number of faculty in AI/ML, hoping to land an interview. (This went nowhere. I was immediately denied by Pieter Abbeel and subsequently failed an interview with Anca Dragan). However, chance encounters later on led me to research opportunities with other groups and projects. Then, the first paper I ever got to co-author effectively defined my research field moving forward — namely, computer vision. So, in short, you could ignore all my advice above and do the same I did: apply willy nilly, and let fate guide your interests. I recognize this isn't very useful advice, which is why I put this bullet point last.

It's worth noting that the above points aren't silver bullets for discovering research interest. It took me my entire PhD to finally find a subfield that I can call my passion, so don't sweat it if you only have vague interests. Vague is all you need. If you've found some level of interest, it's time to start thinking about preparing for research in that field.

🔗Step 3 - Prepare with classes, papers, and internships.

To prepare for research, hone the relevant skills with established resources for doing so. For AI/ML, hone your coding and math skills, focusing your efforts based on your research interests:

These steps above will give you the fundamental knowledge needed to hold technical discussions and conduct research in your field of choice. To start developing research qualities off the bat, see qualities of a good researcher and tips to build those muscles.

🔗Step 4 - Decide when to apply.

There are several "levels of interest" in research, which can help dictate how much time you want to sink in. It will also dictate the latest point you should begin conducting research. Here are the different distinctions and when you should begin research, for each objective:

These are the most common reasons for interest in research, but I'm sure there are many others I haven't covered. In general, my advice would be to start conducting research in your sophomore year to maximize research productivity. After you've taken these steps to prepare yourself, (or in my case, after having followed nearly none of these steps), start reaching out about research opportunities. You can find a fairly straightforward breakdown of how to do this in "How to get into research, as an undergraduate".