Robustness and Privacy Trade-offs in Federated Multimodal Brain Tumor Segmentation: Missing Modalities, Differential Privacy, and Membership Inference

A companion paper builds a federated learning
pipeline for brain tumor segmentation. It trains across four
simulated hospitals with uneven data, and one hospital is always
missing a scan type. It also adds security: TLS, node checks, and
secure aggregation. That paper leaves three questions open. Does
a proximal FedAvg variant (FedProx) help with the uneven data?
Does a model built for missing scans really recover the accuracy
a normal model loses? What does adding formal differential
privacy (DP) cost, both in accuracy and in real protection? This
paper answers all three, and adds a fourth question: does DP
actually stop a real privacy attack, or does it just cost accuracy
for nothing? We find that FedProx performs worse than plain
FedAvg at both dataset sizes we test. We build a modality-aware
fusion model, trained to handle missing scans. It matches the
normal model’s accuracy when every scan is present. But when
the FLAIR scan is missing – the most important scan in our
data – it recovers 0.26 Dice points (0.4785 → 0.7382) that the
normal model loses. On privacy, we find a real problem. Our
DP method adds noise to a ∼971K-parameter model update.
The amount of noise needed scales with the square root of the
parameter count, not with the clipping norm. Every noise level
that keeps accuracy usable gives a privacy guarantee (ε) so large
it means almost nothing. Every noise level with a meaningful
ε (around 16–111) breaks the model completely. We also run a
membership-inference attack, which checks whether an attacker
can tell if a patient’s scan was used in training. This attack
barely works at all (AUC 0.50–0.52), even with no DP applied.
So the accuracy DP costs is not clearly buying any real protection
against this attack. We report all four results honestly, including
the two that did not work as hoped, because a federated system
that is honest about where privacy tools fail is more useful than
one that only reports wins.