Guides · 5 min read

What a student team can actually build in a semester

Ten papers in this thread came out of undergraduate project teams. This is the method behind them, written for the student about to choose a final-year project.

Pick a problem with a data source attached

The most common way a student project dies is that the idea is good and the data does not exist. Before you commit, answer one question: where does the first hundred labelled examples come from, and can you get them yourself in two weeks?

If the answer is that you will collect them, that is a strong project rather than a weak one. The mushroom work in this thread started with a team photographing infected and healthy oyster mushrooms in a real farm, because no public dataset of Trichoderma infection existed. That dataset is now published openly and is the part of the project other people can build on. It also won the paper its award.

Baseline before you reach for the biggest model

Every team wants to start with the largest architecture they have heard of. Start with the smallest thing that could work and measure it, because that number is what makes the rest of the project legible. If a simple classifier gets you to seventy per cent, the contribution of the complicated one is the difference, not the whole.

It also protects you from the most common failure in a student result: a number that looks excellent because the test set leaked, or because one class dominates. If your accuracy is far above your baseline on the first attempt, something is usually wrong with the split.

Measure something beyond accuracy

The projects that turn into publishable work are almost always the ones that measure a second thing. What does inference cost? How long does it take on the device it will actually run on? What does it consume?

That is the whole point of the Eco-Transformers work: the accuracy of a large language model is well studied and its energy cost per inference is not, so the contribution is in the second measurement. The same instinct drove the on-device categorisation work, where the constraint was the phone, and the solar work, where the constraint was cloud cover in real time rather than in a clean dataset.

Write it up as you go

A project written up in the last fortnight reads like one. Keep a running record of what you tried and what it scored, including the attempts that failed, because the failed attempts are what make a comparison section honest and they are impossible to reconstruct from memory afterwards.

If you would like to work on something in this direction, the student path on the work-with-me page says what to send. A message that names a paper here and the question it left you with will get a real reply.

What to read next

Outside work, not mine. Each title links to a Google Scholar search, which will find you whichever version is open access.

2015
Hidden Technical Debt in Machine Learning SystemsD. Sculley and colleaguesWhat happens to a model after the project ends. Read it before you promise anyone a deployment.
2018
Datasheets for DatasetsTimnit Gebru and colleaguesIf you are collecting your own data — and you should — this is how to document it so it outlives your project.