Most differentially private federated learning methods are still fundamentally first-order: clip per-example gradients, add Gaussian noise, aggregate, and take a step. DP-FedGD and DP-FedAvg follow this directly. DP-FedAdam, DP-FedYogi, and DP-SCAFFOLD improve the dynamics around the step, but they're still working off the same privatized gradient signal, not a better one.

The problem shows up under tight privacy budgets. Injected noise can overwhelm useful gradient information, so training needs more rounds or stalls outright, which is expensive when communication, not local computation, is the bottleneck.

Second-order approaches exist (DP-FedNew, DP-FedFC), but they get curvature from client-side feature covariance, which means O(d²) client memory and communication and a new source of sensitivity the privacy analysis now has to absorb. Each client ends up transmitting a full matrix instead of a gradient vector, and the mechanism has to account for that matrix, not just the vector.

Which raises a simpler question: can we get useful curvature into DP-FL without changing the private client release at all?

DP-FedSOFIM leaves the client side byte-identical to DP-FedGD and moves all curvature estimation to the server. The server keeps an EMA of the privatized aggregate and treats its regularized rank-one outer product as a Fisher proxy. Sherman-Morrison gives the preconditioned step in closed form:

M = beta * M_prev + (1 - beta) * G (momentum buffer) F = M M' + rho * I (rank-one Fisher proxy) HG = G / rho - M (M'G) / (rho^2 + rho |M|^2) (preconditioned step) 

The matrix is never formed. It's closer to inferring terrain slope from how a ball has been rolling than surveying the whole hillside at every step: you don't get the full Hessian, but a single running direction turns out to carry enough curvature signal to be useful.

Main results

• Because every server-side quantity is a deterministic function of an already-privatized aggregate, post-processing gives DP-FedSOFIM the same (eps, delta) guarantee as DP-FedGD under identical clipping, noise multiplier, participation, and accountant at O(d) client memory instead of O(d²). This is the standard DP post-processing immunity argument, but it's underused in FL specifically because most curvature work has stayed client-side by default.
• Round-10 margins over DP-FedGD reach +20.3 points (CIFAR-10/ResNet, eps=5), roughly 4-5x fewer rounds to reach 95% of DP-FedGD's final accuracy. Against the strongest adaptive baselines the advantage is concentrated early: at eps=5 on CIFAR-10/ResNet, DP-FedAdam and DP-FedYogi catch up, and round-70 differences aren't significant under McNemar.
• Preconditioning costs are under 2% wall-clock overhead per round relative to DP-FedGD, against roughly 6x for DP-SCAFFOLD at its tuned local-step counts.
• Ablating Sherman-Morrison against EMA alone: the curvature correction contributes about +7 points at eps=1, but at eps=0.5 on CIFAR-10, the DP noise dominates the curvature estimate and EMA alone does as well. The correction earns its place when the loss landscape is anisotropic enough for a rank-one proxy to capture something, which is why PathMNIST benefits at every budget.

Curvature adaptation in DP-FL is better treated as a server-side post-processing problem than a client-side estimation problem. Once the aggregate is privatized, anything the server does with it is free in privacy terms, the same reason you can compute arbitrarily many statistics off a released noisy histogram without paying again. That degree of freedom seems underused in DP-FL.

Happy to take questions on the analysis or accounting. If you work on DP-FL or second-order federated optimization, I'd be glad to hear from you, and I'm open to collaborating.

arXiv: https://arxiv.org/abs/2601.09166
OpenReview: https://openreview.net/forum?id=aDzj9DrwAR
GitHub: https://github.com/sid0nair/DP-FedSOFIM_V2