Deleted Planes in ARKit Immediately Reappear

Hello,

In my app I'm trying to delete all but one chosen plane and do some raycasting on that plane. I noticed that, whenever I tried to delete other planes they would instantly reappear. Here is some sample code of the ARViewController I'm using that demonstrates the problem

class ARViewController: UIViewController {

    var arView: ARView!

*** Bunch of stuff ***

    func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
        // Iterate through the detected anchors
        for anchor in anchors {
            // Check if the detected anchor is an ARPlaneAnchor
            if let planeAnchor = anchor as? ARPlaneAnchor {
                plane_count += 1
                print("Plane added. Number of plane anchors = \(plane_count)")

            }
        }
    }

    func session(_ session: ARSession, didRemove anchors: [ARAnchor]) {
        for anchor in anchors {
            if let planeAnchor = anchor as? ARPlaneAnchor {
                plane_count -= 1
                print("SESSION CALLED: Plane Removed. Number of plane anchors = \(plane_count)")
            }
        }
    }

    func deletePlanes(){
        for anchor in arView.session.currentFrame?.anchors ?? [] {
            arView.session.remove(anchor: anchor)
        }
    }

When deletePlanes() is called, I'll see the following output populate instantly

SESSION CALLED: Plane Removed. Number of plane anchors = 2
SESSION CALLED: Plane Removed. Number of plane anchors = 1
SESSION CALLED: Plane Removed. Number of plane anchors = 0
Plane added. Number of plane anchors = 1
Plane added. Number of plane anchors = 2
Plane added. Number of plane anchors = 3

This even occurs when the phone is face down after detecting a few planes. It appears that the planes are not actually being removed from the session.

Please let me know if I'm doing anything wrong here! Thanks.