If you fine-tune a model with weights stored in bf16 and let torch.optim.AdamW write to them directly, you get interesting behaviour. bf16 can only represent…