from Guide to Undergraduate on Nov 6, 2022

How to prepare for research, without prior experience.

Making yourself an attractive candidate for research mentorship is hard. What even is "research"? Here is a brief guide to the foundations you need, to conduct research.

Preparation for research is effectively about building fundamentals — you don't need to be the world's expert, nor should you wait that long to begin. Having just enough understanding of the necessary components suffices. In this post, I'll skip over defining what "research" really is; this is a longer conversation, and instead, we'll focus on building fundamentals¹ to conduct research.

Step 1 - Pick a field of study.

Your first step is to find a problem space, so you can focus your efforts on becoming the ideal candidate for that field. If you have no idea how to pick a field, don't worry: My advice here is to arbitrarily pick a domain and begin preparing for that domain. If you lose interest halfway, then there's no need to try research in the area. Your success in a field will be defined by your interest in that field's problems.

Your success in a field will be defined by your interest in that field's problems.

There are of course many broad areas of study, such as biology, computer science, statistics, etc. If you have not yet landed on a major, you can use your university's list of departments as a very general list of options. Within each department, you can then use colleges or the introductory classes to get a sense of broad topics.

For artificial intelligence and machine learning (AI/ML) in particular, here are several problem spaces; I'll focus only on the parts of these fields that overlap with AI/ML today:

• Computer Vision (CV): This involves tasks that both generate and extract information from images. The field today has grown beyond images to include videos and various sensors, such as radar, ultrasonic, and LiDAR. Related fields include graphics and computational photography. Try a generative computer vision demo, using the HuggingFace stable diffusion demo.
• Natural Language Processing (NLP): This involves generating and understanding audio and text — in addition to transferring between the two. Related or subfields include linguistics and question-answering. Try a question-answering demo, using the HuggingFace large language model online demo.
• Robotics: This involves controlling a robot to perform tasks, such as swing a wine glass around or sort objects. Related fields include controls and reinforcement learning (RL). Although not strictly related to robotics, try a library for building game agents, using OpenAI Gym tutorial. You can use Google Colab notebooks to avoid the hassle of a local setup on your computer.
• Systems (SYS): This includes a wide variety of infrastructure challenges with training and deploying models, including but not limited to distributed systems, serverless deployments, and databases. Try deploying a machine learning model by following a PyTorch Flask tutorial. You can use Replit to avoid the hassle of a local setup. There are also UI-only alternatives, such as Apple's CreateML or Banana.
• There are other related fields that the Berkeley AI Research (BAIR) website enumerates, including learning theory, security, biology, science, human interaction, and signal processing. You can also find a list of Berkeley AI Research (BAIR) faculty and their interests on the BAIR website faculty page.

There are also many researchers that study a blend of two or more of the above fields. For example, combinations of computer vision and natural language are very common, with text-to-image models, such as OpenAI's DALLE-2, being a very hot topic in the community at the moment. In sum, pick a general field at this point. Either you have an inclination, or you don't; if you fall into the latter category, pick one at random, so you can start knocking items off your list.

Step 2 - Evaluate your interest.

Once you've picked a field of study, evaluate your interest in that field. For AI/ML, take this 1-hour primer on "Tools to Learn Machine Learning". After this primer, you'll be much better equipped to process the deluge of information available. Then, follow the below steps to evaluate interest in your field. I suggest following these in order, either sequentially or in parallel.

• Hack on some code related to that field. This advice is pretty specific to the AI/ML field, where open-source code is now a de facto standard. This could be a framework tutorial such as PyTorch's object detection tutorial, a repository for general tasks such as HuggingFace's transformers tour, or a repository from a random paper like a deep learning adaptation of decision trees. To start, you could treat these repositories as black boxes, and utilize pre-trained models to build a nifty tool. This could be a gateway to exploring the problem space. As you try to deploy the model, you may run into problems: Maybe the model is too slow. Too memory-intensive. How would you fix that? Maybe the model performs extremely poorly on certain protected groups. Why is that? Or, perhaps the model is just underperforming. Toying with these repositories will help you gain confidence, learn about problems, and provide a starting point. Right now, try one of the tutorials above. Alternatively, pick another PyTorch tutorial. Try any of them. Get your hands dirty.
• Join seminars for that field. If you're early in your undergraduate career, you may not have room to add more classes. I suggest looking for topical seminars that your graduate department offers. For UC Berkeley, you can find seminars and classes for computer science and electrical engineering. You don't need a fully fledged class, and a weekly gathering for research talks suffices. Then, simply see which presentations interest or bore you. I should caution you that I find this method faulty — evaluating interest in the field using interest in talks. This is mostly because you're relying on engaging speakers to spark interest. However, I have several peers who picked their fields because of that one engaging, enthralling speaker. So, you could try your luck using this method. Sign up for a bunch of seminars initially, and after a week of talks, keep only the most interesting ones. Be sure to drop the ones you don't want! Right now, identify one seminar in your field of interest, and add the next meetup to your calendar. Use the links above to find a seminar.
• Read paper synopses from that field. This was not the path I took, as paper summaries were not as prolific in my undergraduate. However, you can find distilled and sometimes sensationalized research summaries on social media — popular sources include @akhaliq, and @TwoMinutePapers. You can also find high-level research perspectives and random research highlights from famous researchers like @karpathy. In the past, Twitter has been a fairly good stream of papers, especially after conference submission deadlines, as submissions are posted on Twitter. This may no longer be true when you read this article. Right now, follow or subscribe to a content creator that creates content for your field. You can use conference proceedings to find academics to follow. See the list of conferences below, in the next step. Admittedly, social media is not the ideal place to consume research, but if you're already spending hours on social media, might as well blend in a productive mix of short-form research content.
• Apply willy nilly and let fate decide. This was my technique of choice. I took classes that led me the quickest to graduation and haphazardly emailed a number of faculty in AI/ML, hoping to land an interview. For what it's worth, this went nowhere. I was immediately declined by Pieter Abbeel and subsequently failed an interview with Anca Dragan. However, chance encounters later on led me to research opportunities with other groups and projects. Then, the first paper I ever got to co-author effectively defined my research field moving forward — namely, computer vision. So, in short, you could ignore all my advice above and do the same I did: apply willy nilly, and let fate guide your interests. I recognize this isn't very useful advice, which is why I put this bullet point last. If this is your tactic of choice, right now, fill out the email template in How to get into research, as an undergraduate.

It's worth noting that the above points aren't silver bullets for discovering research interest. It took me my entire PhD to finally find a subfield that I can call my passion, so don't sweat it if you only have vague interests. Vague is all you need. If you've found some level of interest, it's time to start thinking about preparing for research in that field.

Step 3 - Prepare with classes, papers, and internships.

To prepare for research, hone the relevant skills with established resources for doing so. For AI/ML, hone your coding and math skills, focusing your efforts based on your research interests:

• Take relevant classes. Classes are not a perfect proxy for research interest but definitely contain fundamentals you'll need for the field. Admittedly, for applied fields especially, some of the theory is not needed, but you won't know which ones are necessary until later. The goal for these classes is to prepare you broadly for any subfield, so there's a great deal of breadth. If you've identified faculty with shared interests, make sure to read their personal websites, which often lists required classes. For AI/ML at UC Berkeley in particular, here are a list of relevant classes for each field. Even if you aren't at Berkeley, the syllabi for these courses can help you find the analogous course at your university:

  • Machine Learning: For the machine learning field as a whole, take Machine Learning (CS189). Probability (EE126) and optimization (EE127) are also often cited as must-haves. This pair, especially the latter, is fairly helpful. Even if you don't yourself use this daily, it gives you the technical background to read papers.
  • Computer Vision: I recommend the official computer vision course (CS280), although it's difficult to get into as an undergraduate as a graduate course. If you can't register, just read the slides or book. Get to know your stereo geometry! There are other courses, such as computational photography (CS194-26), which are designed for undergraduates. I took and very much enjoyed this course — made NSFW art, face morphed alyosha into a cartoon character, and made Atlas hold up a pusheen. Some of the covered concepts were later useful for my research as well. I wish I'd taken graphics (CS184) and signal processing (EE120) though.
  • Natural Language Processing: Take the aptly-named courses "Natural Language Processing" (Info 159, offered by the iSchool, or CS288, offered by the EECS department). Neither are offered very often, so seize the opportunity when they are. I took Introduction to Linguistic Science (LING 100), but my impression is that after deep learning's takeover, linguistics has played a minimal role.
  • Robotics: Artificial Intelligence (CS188) is a widely accepted must-take, and despite the name, the course is mostly geared towards fundamentals needed to study reinforcement learning. There is also an official Deep Reinforcement Learning (CS285) course that I would recommend. The instructors for this course are especially good.
  • Other courses: I was advised to take more math courses, so I took real analysis (MATH 104) and abstract algebra (MATH 113) but to be honest, I didn't need either course's contents for my more applied work. I was also advised to take statistics courses, so I took Theoretical Statistics (210A, 210B); these courses completely blew over my head, and I haven't used theoretical statistics since. With that said, if statistics is your jam, these classes are taught by the best lecturers you may ever find. They're charismatic, clear, and concise.
  • One last note about classes: If a class is boring, that field might not be for you. However, it might also be perfect for you. Classes are not a great proxy, and some classes are vastly different experiences from conducting research in the field. If you, like me, performed poorly in a class for your field, it's not the end of the world. Make sure to To maximize lecture, listen like you're about to reteach it. Know what you need to work on, and keep going.

• Read papers from that field. Depending on your field, papers are either (a) dense, as the field is rigorous like computer science theory or (b) dense, as the field likes using meaningless math, such as deep learning. I'm exaggerating a bit, but the point is that papers can end up being too dense to gauge general interest. However, you'll need to learn how to read papers to conduct research in the area. I've listed the top conferences in each field below. For each new field you're digging into, Google "literature review for X" — this should give you arXiv summary posts. After getting an overview, I suggest pulling papers from the below venues to read, as a conference proceeding has been previously vetted by researchers in the field:

  1. Machine Learning: Conference on Neural Information Processing (NeurIPs), International Conference on Machine Learning (ICML). These are catch-all conferences which include papers from all of the below fields.
  2. Computer Vision: Computer Vision and Pattern Recognition (CVPR 2023), European Conference on Computer Vision (ECCV 2022), International Conference on Computer Vision (ICCV 2023). Each year alternates between ECCV and ICCV.
  3. Natural Language Processing: Empirical Methods in Natural Language Processing (EMNLP 2022), Association for Computational Linguistics (ACL 2023)
  4. Robotics: Conference on Robot Learning (CORL 2022), International Conference on Robotics and Automation (ICRA 2023)
  5. Systems: Machine Learning and Systems (MLSys), EuroSys

• Finish an internship to solidify your coding skills. Participate in a software engineering internship at a company. Any company big or small company works, as long as you're properly working with a production-level codebase and not just coding on your own. It's important to learn how to code with others and write maintainable, readable code. This last part is extremely important: Make sure you know how to organize and write code others can read. No need for doc tests and docstrings, but the code should not be a tangled mess. With the field right now being very open-source heavy (as it should), other researchers will need to make sense of your code to use it. Furthermore, coding is just a means to an end. The better an engineer you are, the faster you'll be able to knock out code to get to real results.

• Miscellaneous resources that are worth bookmarking and keeping in your back pocket:

  • One of my good friends—Sinho Chewi, who is now doing his PhD at MIT—wrote a large number of notes for various classes across math, statistics, electrical engineering, and computer science. You can find his course notes online. These were especially help for my statistics courses, where official notes were not available.
  • Another one of my friends—Garrett Thomas, who is doing his PhD at Stanford—wrote a "Mathematics for Machine Learning" (Math4ML) resource that's really comprehensive. It's not meant to be read from top to bottom, but it's extremely useful for jogging your memory or identifying what you don't already know. I referenced this resource many many times.

These steps above will give you the fundamental knowledge needed to hold technical discussions and conduct research in your field of choice. To start developing research qualities off the bat, see What defines a "good" researcher?

Step 4 - Decide when to apply.

There are several "levels of interest" in research, which can help dictate how much time you want to sink in. It will also dictate the latest point you should begin conducting research. Here are the different distinctions and when you should begin research, for each objective:

• I want to try research out of curiosity. Or pride. This is a perfectly reasonable justification for taking a shot at research. In my undergraduate, saying you belonged to a research lab was a point of pride; it was also fairly common to try research at some point in your undergraduate career. If you belong to this category of students with "casual interest," starting research even your last semester of undergraduate is perfectly a-okay. Your work may not bear much fruit in just a semester, but it's definitely enough to get a taste. However, note that many graduate students are looking for undergraduates that can commit more than just a semester, so looking sooner is always better.
• I want to pursue a master's degree. Depending on your goal terminal degree, you may tackle research with different levels of urgency. If your ultimate goal is industry and thus are interested in industry-oriented master's programs (for example, the Berkeley EECS 5th-years masters), then you should begin research by your senior year. For this example, students are theoretically expected to continue their senior-year research through the master's. With that said, this specific program is very class-heavy and is not optimized for research. Most other master's programs are similar, with a focus on industry and classes rather than academia.
• I want to pursue a PhD. It's worth noting that if your ultimate goal is industry research, you may not need a PhD. See Why pursue a PhD? Is it for me? If you're still set on pursuing one, however, you should ideally begin conducting research in your sophomore year. In your freshman year, you will likely not have enough background or time; instead, I would suggest spending that first year exploring and acclimating to the undergraduate life. In your sophomore year, you still have years of research potential and have just enough working knowledge under your belt. You could start at any point in your junior year too; for what it's worth, I didn't truly begin making research progress until my junior year too. Starting in your senior year is fairly late, as PhD applications begin in your senior year fall. This changes if you plan to take a gap year before applying — by pursuing a master's in the interim, joining industry, or conducting research in a lab.

These are the most common reasons for interest in research, but I'm sure there are many others I haven't covered. In general, my advice would be to start conducting research in your sophomore year to maximize research productivity.

If you haven't already, complete as many todos as you can, from the first and second steps above. If you're busy, set a timer for 10 minutes, and knock out as many as you can. Piquing your interest and finding a new curiosity is certainly worth 10 minutes of digging.

After you've taken these steps to prepare yourself, (or in my case, after having followed nearly none of these steps), start reaching out about research opportunities. You can find a fairly straightforward breakdown of how to do this in How to get into research, as an undergraduate.

← back to Guide to Undergraduate