I second this. AdamW is so important these days as it correctly fixes weight decay. If quick fix is difficult, it'd be great if we don't fallback to legacy adamw. It's much better than just crashing.