Build the control that could change your mind

The model had a clean story and good numbers: top of the benchmark, near fold free, a fraction of the parameters of the transformer baselines. Everything supported the claim that the continuous representation was the reason.

Then someone asked the question I should have asked myself much earlier. How do you know it is the representation, and not your training objective?

The only honest way to answer

A baseline comparison cannot answer that question. Every baseline differs from your model in a dozen ways at once, so beating one tells you your whole system is good, not which part of it is doing the work.

What answers it is a control that shares everything except the single thing you are claiming credit for. So I built one: an ordinary voxel lattice model, trained with my loss, my integrator, my data splits, my schedule, my seeds and my own tuned hyperparameters. The only difference was the representation.

If the contribution is real, a control like this is the strongest evidence you can produce. If it is narrower than you thought, you would much rather learn that yourself than read it in a review.

What it told me

On isotropic 1 mm brain MRI, the two are extremely close. Much closer than the baseline comparison had suggested. Which meant the large gap to the published baselines was substantially explained by the training objective, and the representation was contributing something smaller and more specific than the headline implied.

It took about a day to sit with and then became the most useful thing I had. The claim was not mistaken, it was imprecise. It sent me looking for the regime where the representation should matter on mechanistic grounds, which is anisotropic sampling, and that is the study now underway.

What I do differently now

I design the attribution experiment at the same time as the method, not after. If I cannot describe the control that would isolate my contribution, I do not yet understand what my contribution is.

I also run removal studies on my own components rather than only on the baselines. A component that does nothing is not neutral. It is cost, complexity, and one more thing a reader has to take on trust.

The baseline I did not run, and why

The comparison covers ten methods. A reader who knows this area will notice that NePhi is not among them, and the reason is worth stating rather than leaving as an omission for someone to find.

NePhi is a neural deformation field with a deliberately similar motivation to mine, an approximately diffeomorphic transformation carried by a functional representation with latent codes rather than by a voxel grid, so it is the single most informative comparison available and I would rather have it than not. The obstacle is practical. The paper reports results on lung CT and brain MRI, but the repository itself lists uploading model weights and committing evaluation scripts as outstanding tasks, and the training configuration it ships is for lung. Putting it in my table honestly would mean training it from scratch on IXI and then tuning it with the same budget and the same protocol every other method here received, because a baseline given less care than the proposed method is not evidence of anything.

That is a real piece of work rather than an afternoon, and doing it badly would be worse than not doing it. So it sits in the queue as a stated gap with a plan attached, not as a silence. Until it is run, the fair reading of my comparison is that it covers the published lattice based methods with public brain weights, which is a narrower claim than beating everything in the literature.