You can avoid it with a little trick. add an onChnage modifier and if editMode is not active empty the list. Add the following modifications: .onChange(of: selectedItemIDs, processSelection) func processSelection(_ oldValue: Set<Item.ID>, _ newValue: Set<Item.ID>) { if editMode == .inactive { self.selectedItemIDs = [] } }