Right-to-be-Forgotten by Design in Adapter-Tuned Transformers

Enforcing the Right-to-be-Forgotten (RtBF) in transformer models remains challenging: full retraining is costly, while post-hoc unlearning often leaves residual signal without guarantees. We propose an RtBF-by-design protocol for adapter-tuned models that combines differential privacy (DP) applied to Low-Rank Adaptation (LoRA) adapters with a matched-control Deletion Sufficiency Certificate (DSC). A DP-trained model on the full dataset is evaluated against an identically configured and trained redacted-control model using three complementary criteria - prediction agreement, membership-inference separability, and calibrated confidence exposure - to assess deletion sufficiency. The DSC offers an operational go/no-go decision for RtBF without requiring full retraining. Experiments on a text classification task show that multiple privacy budgets (e.g., e {5, 6, 8}) preserve near-baseline utility while meeting all deletion sufficiency criteria, whereas post-hoc unlearning baselines either degrade utility or exhibit strong residual leakage. The protocol provides a lightweight, reproducible pathway to RtBF compliance in adapter-based fine-tuning.