from Guide to Undergraduate on Nov 13, 2022

How to succeed as a (research) mentee.

Making the most of your research mentee experience is no easy task: The definition of a strong researcher is ill-defined — just like the definition of strong research mentee.

Research mentees¹ include anyone conducting research with a senior researcher present; this makes the advice below broadly applicable. The advice is geared towards two specific scenarios: (a) when the mentor is making high-level decisions and leading the project, as well as (b) when the mentee is—you are—making high-level decisions. This could also apply to undergraduates, junior graduate students, research interns, or even newly-minted industry researchers. Expectations are of course higher the more years of experience you have, but the rigor and steps to growth are just the same.

This guide is mostly written from a mentor's standpoint, capturing the trends I've noticed in mentees that successfully publish papers—including those that went on to pursue PhDs at top computer science programs and those that likewise found immense success in industry. I'm very proud of the undergraduate and graduate researchers I've had the chance to mentor over the years, and to all research mentees out there, I hope this guide can help you grow into someone you're proud of too.

Pillars of a successful mentee.

Before making a laundry list of all the minutiae to take care of, I should enumerate the pillars of a successful mentee. Successful mentees accomplish the following:

• Keep your mentors informed. This enables your mentors to chime in at critical moments, giving you the feedback you need and giving mentors insight into the direction of the project. This doesn't mean pinging mentors daily. Instead, it means keeping a document, channel, or group with daily status updates. In this way, mentors can ‘pull' updates on their own schedule, rather than rely on you to ‘push' updates to their inbox. The best mentees also offer updates at various granularities — daily, weekly, and maybe monthly. On top of keeping mentors informed, this has the additional benefit of keeping you in tune with the high-level direction, saving you from venturing down rabbit holes. Furthermore, as your mentors poke holes and ask questions, you should get better over time at providing concise and clear updates.
• Seek high-level guidance. This high-level oversight is primarily what you're looking for as a mentee, as it's easy to get lost in the weeds and lose sight of the 30,000-foot perspective. This manifests itself in specific ways: Say you get stuck debugging a particular slow operation in your model. This particular operation is difficult to build a custom CUDA operation for, and 90% of your model latency is due to this operation — ultimately slowing your training and progress. Mentors can certainly provide debugging advice, but taking a step back, your mentors may realize that you don't need to use this particular implementation. Simply try a different one. Or, maybe you don't even need to use this architecture; build on top of a more widely-used architecture. This is ultimately the role of the mentor — to ensure you don't get lost in the weeds, so you can keep making forward progress. The most important guidance changes your direction in light of the overarching goal. As you become more experienced, you'll learn how to pivot on your own, and over time, you should be able to handle more and more low-level problems.
• Define or contribute to the story. If you're curious about what the "story" is, see "Tell Tried and True Stories" from What defines a "good" researcher?. This could involve challenging the problem you're tackling—for example, by questioning its applicability to different domains; reading about related methods that tackle the same problem; or finding new baseline methods to compare against. If you're leading your own project, these are necessary to keep a lookout for. If you're helping a senior researcher with their project, these are necessary for them to watch out for and a huge plus if you can help them with this. This is a step beyond the call of duty, but this is the half step between following and defining a research agenda.

Every piece of advice in this post is grouped into one of the above three categories. Distinct from being a good researcher, these pieces of advice focus on maximizing how much you can learn from your mentors.

Setting up on day one.

On day one, establish routines and habits that will enable you to ask for help when you need it, and present results when you have then. On your first day on the job, get all your ducks in a row: what you're working on, how you'll collaborate, and what you'll need.

Here are specific todos for each category above:

• Keep your mentors informed. Under this category, get setup with communications and establish a home base for updates.

  • Ask how you will communicate. Get added to any slack organizations and channels that you need. Get email addresses should you need them. Join Google drives, calendars, and anything else you may need.
  • Ask how you should provide updates. Ideally, create a slack channel instead of direct messaging each mentor individually. Or, setup a Google document with status updates. Find some place to put more frequent status updates. Even if your mentors say a weekly meeting suffices, find a place to do this anyways.
  • Join group meetings, reading groups, and seminars. Ask if these exist in the lab and if you can join. Generally, these are open events that are sent to general mailing lists.
  • Ask if there are resources you'll need. You may need access to AWS, badge access, or cluster access. You may also need access to Github repositories, possibly any tooling licenses. Post updates as you get access to tooling that you need, or if you're blocked by lack of access.

• Seek high-level guidance. Under this category, setup times and places where you can easily ask questions.

  • Setup weekly meetings. Even the meetings are short, these are necessary. It provides a time for you to ask questions and for them to ask questions. Verbal communication will always be more efficient. Try to put together basic slides for these meetings. It shows preparedness, and you don't need fully-fledged slides with text. Just include key figures and plots—any visualizations you need to make your point. Alternatively, write down a list of questions in advance. Any sort of prep work will make your meetings go more smoothly.
  • Schedule in-person time. Find a physical space where your mentor will be, and try to schedule a regular weekly working time. This should be outside of your weekly meeting, and it should be for active coding sessions. This doesn't mean your mentor needs to code with you; it's just a more office-like environment where you're working right next to your mentor.
  • Determine objectives and timelines. Ask your mentor which deadlines and venues are appropriate for your work. This can help determine your pace.
  • Clarify your role and how you'll work together. Is this your project and they're advising? Or is this their project and you're helping? This will help you better understand if they're expecting to code alongside you, or if you're coding on your own. This is an important distinction.

• Define or contribute to the story. Under this category, get setup to understand the framing for your work and what impact it should make.

  • Ask if there is material you can read. There may be related papers, or if you're lucky, a literature review that provides a high-level overview of the field. This will give a broad sense of what competing methods do, and how others have tackled the problem space.
  • Ask specifically about the problem and motivation. If this is your mentor's project or idea, ask them what the motivation is for the project. They should have a clear idea of the problem space and the intuition to leverage for the project. For example, the problem may be "Existing segmentation networks are very slow to run on mobile devices." The intuition may be "They distribute computation evenly throughout the image, meaning they are wasting resources on uninteresting portions of the image like sky. We can improve efficiency by not wasting computation on uninteresting image regions."

How to meet expectations.

There are certainly basic expectations for a mentee. Know that the mentee's job is to put in work, but ultimately, it's the mentor's job to ensure that the work you put in goes in a meaningful direction. Here is how to meet expectations in each of the three categories above.

• Give regular updates — whether progress or not. If you run into low-level trouble (e.g., third-party repository does not run, slurm system has locked you out of a node with your dataset) throughout the week, post in your slack channel. Make sure your mentors are aware of blockers, and at the same time, try to pivot your way out of these blockers. Over time, you should be able to tackle an increasing number of low-level problems.

  • Most importantly, don't run experiments blindly. Experiments take babysitting. After launching an experiment, periodically check on its loss values — ensuring loss is decreasing over time. Check on its performance. If the model is training on ImageNet, ensure its classification accuracy is increasing over time. The worst is when you run a 3-day job, only to discover the model never achieved a training accuracy beyond guessing — 0.1% on ImageNet. Monitor and check early to catch problems early.

• Show up to meetings. At a bare minimum, attend the weekly meeting and provide status updates¹. During the weekly meeting, try your best to provide high-level updates so that your mentor can guide as necessary. The goal is to keep meetings as more productive discussions. In-person work sessions should be used for debugging.

  • Don't ask questions just for the sake of asking it. For example, asking what you should name your custom bash function, or how often trash in the lab is taken out. Keep questions focused.

• Understand the story. Ensure that you understand the problem you're tackling and how your work connects to it. If you forget, ask; your mentor should have no problem regurgitating this. In fact, they may have thought more about this and changed the story up. The more senior you are, the more you're expected to be able to answer this yourself. As a budding researcher, the basic expectation is to understand the story.

The mentee's job is to put in work, but ultimately, it's the mentor's job to ensure that the work you put in goes in a meaningful direction.

Here is more specific advice that is based on my own lab's setup at Berkeley; you should plan to learn these as you go, since these will be big productivity boosters later on. You also might need these tips to conduct research if your lab uses a local cluster of GPUs:

• Learn how to code with others on a Github repository. You can also pick up these skills in a software engineering internship. If not, now is the time. Standard protocol is to create a new git branch, work on that branch, then make a pull request (PR) against the main branch when you're done. Once others have approved your PR, you can then merge into the main branch for everyone to use. Generally, code quality shouldn't matter too much, as long as it's readable somewhat and not obviously broken.
• Learn how to use the cluster's filesystem. Understand which mounts are used for which kinds of files. For example, nfs mounts often perform poorly with many small files; if you plan to run ImageNet experiments, you may need to host the ImageNet dataset locally on the target node. There may also be a shared directory across all nodes; if that's the case, place your code there, which will only need to run once per job launch. Make sure you know which directories are slow to access as well. For example, at Berkeley, the EECS home directory is exceptionally slow, so don't put your anaconda environment there.

• Learn commands and tooling for a remote GPU. There are several tips and tricks for working with a remote GPU:

  • You can connect VSCode to your remote filesystem using VSCode Remote Host. For a long time, I wrote code in PyCharm locally, then scp'ed my files up. Worse, I would even sometimes commit, push, then pull. I did this until one of my undergraduate collaborators Henk enlightened me, and my life has been drastically improved.
  • Know your GPU commands. Use nvidia-smi to check which GPUs are used. Use CUDA_VISIBLE_DEVICES=0,1 python ... to restrict your process to a subset of GPUs. This is pertinent only really for the login node in a slurm system. Know your lab's etiquette as well; in general, don't run your jobs on a GPU where someone else's script is also running.
  • Know how to port forward from your local machine to your node of choice. You may need to go through your login node and shared filesystem. This will enable you to track experiment progress if you're using a tool like tensorboard.

• Learn slurm. If your cluster uses a slurm job submission system, run a basic job on it as soon as you can. There are many references online for interacting with a slurm system. Ensure you know which node is designated for login, and get acquainted with best practices.

  • Get to know how to submit a job sbatch <script>.sh, check job status scontrol show job <id>, cancel a job scancel <id>, and list your jobs by node squeue -u <username>. Be a good slurm citizen and don't fill the queue with thousands of high-priority jobs.
  • There are many other slurm commands I use constantly but couldn't be bothered to memorize. I put these in my ~/.bash_profile. Here are two commands that I used, which let me start an interactive CPU-only job like scpu node1 or an interactive GPU-only job like sgpu node2 2 for a 2-gpu job:

~/.bash_profile

scpu () {
    srun --nodelist=$1 --pty bash -i
}

sgpu () {
    srun --nodelist=$1 --pty --gres gpu:${2:-1} bash
}

In short, here are the three basic expectations of a mentee: (a) Give regular updates, (b) Show up to meetings, and (c) Understand the story. Do your best to tackle low-level problems and keep discussions high-level, but bring up blockers as you run into them.

How to exceed expectations.

Exceeding expectations involves providing your own direction. The more you can guide yourself, and potentially bring up ideas that change the direction of the project, the more you'll grow as a researcher and surprise your mentor. All of these tips below fall in the third category, "Define or contribute to the story". To become an invaluable collaborator, follow these tips:

• Read relevant papers and do a literature review. Bring up papers and suggest what they may be doing wrong, better, or differently. Each paper should have a takeaway that aids your decision making; either it affirms your direction or alters it. Some of my strongest mentees and collaborators have kept on top of recent papers, posting papers that address the same problem space or utilize a similar solution. These have been very helpful with guiding the project's direction, especially as the rest of team has dug deep into the code. Right now, Google the topic for your project, and read the abstract for 1 paper. Keep reading if it's interesting or highly-related.
• Suggest baselines or sanity checks to run. Find baselines to test and sanity checks to invalidate your assumptions. Keep an eye out for key assumptions your methods may be making, and test those assumptions directly. It's not common but also not too uncommon for mentees and collaborators to note these (and it makes me proud every time); not long ago, I proposed re-training a model to fix the first model's mistakes. My collaborator immediately asked, "Doesn't that assume each model makes different mistakes?" She was spot on in fact, so this led us to far better experiment: Train the same model twice, and see if the two models make mistakes for different samples. This was a spot-on question to ask, precisely because it helps us fail or prove assumptions quickly, a hallmark of an efficient researcher. For details, see "Fail Fast" from What defines a "good" researcher?. Right now, come up with a new baseline or sanity check for your project. Check your assumptions, and test them directly.
• Think about ways to frame what you're doing. A convincing motivation for your problem space is difficult to come up with, and your team members are constantly thinking about it. Your mentor should help guide you in developing a strong motivation and a strong story. With that said, you can practice and help develop the story too. It's simply a matter of pitching your story to your peers, when they ask what you're working on. If you can't pitch the project convincingly, you may not truly understand its impact. Ask your mentors how to frame the story, and try again. As you pitch the story more and more, you should get a better sense of what parts stick and what parts don't. Use this experience to help your team members write a convincing story. Right now, find a friend and a time to practice your pitch. It could be casual hangout or a study session. Just explain what you're working on.

In short, here are three accessible ways to start exceeding expectations: (a) Read relevant papers, (b) Suggest sanity checks to run or question assumptions, and (c) Practice pitching your research. That's it for tips to succeed as a research mentee. Best of luck on your research journey!

← back to Guide to Undergraduate