Researcher Breaks Down Path to Award-Winning Papers: 'Luck Plays a Role, But Not The Only One'

Nicholas Carlini, who recently received a best paper award at EuroCrypt for his model stealing research, has published an extensive blog post detailing his approach to conducting research that matters. The post, which took him eight months to write, offers both humility about the role of luck and concrete advice for researchers aiming to make significant contributions.

"Whether any particular paper wins an award is mostly luck," Carlini admits, "but it's clearly not all down to luck, and I actually owe it to others to give a more actionable response."

The article breaks down the research process into four main sections: coming up with good research ideas, performing good technical research, writing understandable papers, and navigating what happens afterward.

Developing Good Taste in Problems

The single most important skill for high-impact research, according to Carlini, is developing good taste in what problems are worth solving. Researchers with good taste find clever approaches and are pulled toward solutions that feel "right" before they can fully articulate why.

"Researchers who have developed good taste find clever and elegant approaches, and find themselves pulled toward solutions that feel 'right' before they can fully articulate why," Carlini explains. "At a macro scale, they pick problems that will matter; at a micro scale, they take approaches likely to succeed."

He notes that teaching taste is difficult, but it comes from practicing research while keeping focus on identifying what works and what doesn't.

The Power of Collaboration

Carlini emphasizes that no one does science in isolation. Good collaborators catch mistakes before others do, push back on bad ideas, bring complementary skills, and provide sounding boards for developing research directions.

"Saying 'have great collaborators' is not very actionable," Carlini acknowledges. "But you have more control over this than most things in life. Research is extremely amenable to collaborations that span the world."

He shares examples of successful collaborations initiated through thoughtful emails, including his EuroCrypt paper with Adi Shamir, which began after Carlini worked out a partial solution to a problem and reached out with his findings.

Strategic Reading and Idea Selection

While reading the literature is essential, Carlini also advises researchers to eventually "ignore all the papers" to avoid being constrained by existing approaches and potentially flawed directions the field has taken.

"Once you've read everything, the second step is to forget it all," he writes. "The reason is simple: everything that's already been done has already been done. If you constrain yourself to thinking only about what's been done, you'll never come up with something clever and new."

Carlini recommends approaching research with the objective of discovering something interesting, important, and new, rather than simply writing a conference paper. He also suggests trying to do something only you can do, finding areas where you have a comparative advantage, and positioning yourself to get lucky.

Executing Research Effectively

When it comes to performing the research itself, Carlini emphasizes the importance of killing papers that aren't working, either technically or in terms of potential impact. He compares the process to forming a startup, advocating for the "fail fast" principle.

"Great papers don't have poorly performed experiments," Carlini states. "Part of doing high quality science is not only that you've got a really good idea, but that you went to unreasonable lengths to execute on it better than anyone would reasonably expect."

He also stresses the importance of maintaining focus, ensuring the paper represents a "maximal" version that doesn't leave obvious improvements undone, while still leaving room for others to build on the work.

The Art of Scientific Writing

For many researchers, writing is the most challenging part of the process. Carlini, who admits to having struggled with writing throughout his academic career, offers practical advice for making papers more accessible and impactful.

"Research papers should have exactly one idea," he emphasizes. "You're allowed to support this idea through multiple experiments, and you're allowed extensions. But you should have One Singular Idea. Everything you write should connect to it."

He suggests knowing your reader, writing a compelling introduction that tells a story, creating self-contained figures, and writing conclusions that provide reflection rather than simply summarizing the paper in the past tense.

Navigating the Publication Process

Finally, Carlini addresses the unpredictable aspects of research publication, including the challenges of being "too early" with ideas, maintaining relevance in fast-moving fields, and competing with others working on similar problems.

"At any conference, something like 2-10% of papers are truly excellent and deserving of recognition," Carlini notes. "But only a smaller fraction can receive an award."

Despite these challenges, Carlini encourages persistence, noting that many of his award-winning papers were rejected at least once before acceptance.

"The award, if it comes, is just someone noticing where your distribution ended up," he concludes. "Focus on the distribution. Because of this, you shouldn't go into research with the objective of winning a best paper award. It's unhealthy, counter-productive, and out of your control. But you also shouldn't go into research just trying to accumulate publications that don't matter. Instead, write papers with the goal of having an impact. That's what matters, is entirely under your control, and is lots of fun."

For those interested in reading Carlini's complete thoughts, his blog post can be found here.