Guides · 5 min read

Sharing a model instead of the patients

Health data is the most sensitive data there is, and the usual answer — de-identify and release — has failed publicly more than once. There are two better answers, and they compose.

Why de-identification is not enough

Latanya Sweeney showed in the late 1990s that a postcode, a date of birth and a sex are together enough to pick out a large majority of the American population. Hospital releases that had removed every name were matched against a public voter roll, and the governor of Massachusetts found his own records in one of them.

This is the same lesson as the Netflix result, arriving earlier and with higher stakes. The fields that look harmless are the ones that identify you, because identification does not need a name — it only needs a combination that belongs to one person.

Medicine cannot respond by keeping everything locked up. A model trained at one hospital on one population is a model that fails at the next. The question is how to move what was learned without moving who it was learned from.

Answer one: release the model

Instead of publishing records, train on them and publish only the trained model. This is obviously better than releasing the data, and it is not automatically safe: a model can memorise. Given access to it, an attacker can often tell whether a particular person's record was in the training set — a membership inference attack — and for a rare condition that alone is a disclosure.

Making the release itself differentially private closes that gap. You spend a privacy budget during training rather than on the data, and what comes out carries a bound on how much any one patient contributed. That is the subject of my most-read paper in this thread, which measures what the bound costs in clinical accuracy rather than assuming it is free.

Answer two: never move the data

Federated learning, introduced by Brendan McMahan and colleagues in 2017, inverts the pipeline. The model travels to each hospital, trains locally, and only the updates come back to be averaged. No record ever leaves the building where it was created, which matters as much for regulation and institutional trust as it does for mathematics.

It is not privacy on its own. Gradients are derived from data and leak accordingly; there is a body of work showing that training images can be reconstructed from the updates alone. So the two answers belong together: federate so the records stay put, and add a differentially private mechanism to the updates so what does travel carries a bound.

The practical obstacles in Indian hospital settings are rarely the cryptography. They are that each site labels differently, that the network is unreliable, and that one large site can dominate the average. Those are the problems that decide whether a federated project survives contact with a real ward.

What to read next

Outside work, not mine. Each title links to a Google Scholar search, which will find you whichever version is open access.

2002
k-Anonymity: A Model for Protecting PrivacyLatanya SweeneyWhere the field started, and the clearest statement of why quasi-identifiers matter.
2017
Communication-Efficient Learning of Deep Networks from Decentralized DataBrendan McMahan and colleaguesThe federated averaging paper. The whole field of federated learning starts here.
2017
Membership Inference Attacks Against Machine Learning ModelsReza Shokri, Marco Stronati, Congzheng Song and Vitaly ShmatikovWhy releasing a model is not automatically safe. Read it before you promise a hospital that it is.