You can also copy multiple symbols at once which is a nice touch.

For Keynote you can: right click on images in the SF Symbols app select Copy image then paste in to Keynote I believe by default it pastes in SVG format, but you also have the option to Copy image as Where you can choose SVG or PNG and some options.

This appears to have been fixed, there is now a SF Symbols 5.1 that I was able to download successfully today. Must have been an issue with adding the 5.1 update 🤷‍♂️

This appears to be resolved now, there is a SF Symbols 5.1 link that I was able to download today.