Replace AdamW with Muon as the optimizer #51
Description
Looks like PyTorch core is getting Muon as an optimizer option - from Soumith Chintala's X post
Might be useful to try (Muon is almost twice the training efficiency compared to Adam?) once it's there.