I'm not sure you would need to remove a view.
I don't know how SwiftUI uses memory in the background, but views are drawn according to the state of the data. I've never had any memory issues with my SwiftUI apps.
So, in your example, the Text view would not be rendered when isVisible is false, and you don't need to do anything to make that happen. You don't need to do the SwiftUI equivalent of removeFromSuperview() (if there is such a thing), so you don't need to do:
if(!isVisible) {
view.removeFromSuperview()
}