SwiftUI matchedGeometryEffect not working with NavigationView

I'm trying to used matchedGeometryEffect on a pair of views. It works well until you navigate to a child view and then back, in which case the matchedGeometryEffect seems broken briefly (the red rectangle is instantly visible when I try expanding my view)

Is there something I'm missing?

struct ContentView: View {
    @Namespace private var namespace
    @State private var expanded = false

    var body: some View {
        NavigationView {
            VStack {
                NavigationLink("Click Me") {
                    Text("Hello, world")
                }
                Group {
                    if expanded {
                        Rectangle()
                            .foregroundColor(.red)
                            .matchedGeometryEffect(id: "Rect", in: namespace)
                            .frame(width: 300, height: 300)
                    }
                    else {
                        Rectangle()
                            .foregroundColor(.blue)
                            .matchedGeometryEffect(id: "Rect", in: namespace)
                            .frame(width: 50, height: 50)
                    }
                }
                .onTapGesture {
                    withAnimation(.linear(duration: 2.0)) {
                        expanded.toggle()
                    }
                }
            }
        }
    }
}