Concurrent Processing of SKSpriteNode Copied Data in SpriteKit

I posted this question on stack overflow the other day but haven't received any replies as yet. I want to be able to access SpriteKit node characteristics on a background thread. I only require a copy of these characteristics but don't know how to do this. The full question was:

I’m working on my first Swift project. It uses SpriteKit and has quite a few vehicles moving around different tracks. Each vehicle must be aware of other vehicles to avoid collisions and for overtaking etc. The processing for this is very time consuming so I want to run most of the calculations concurrently on a number of background threads.

To do this, I copied the vehicles defined in the arrays track1Vehicles and track2Vehicles (arrays of SKSpriteNodes) to t1Vehicle and t2Vehicle. The idea was to perform all the calculations on these, then afterwards transfer the new positions etc back to the original arrays using SKActions on the main thread. The problem I’m having is that any reference to t1Vehicle and t2Vehicle parameters gives the error “Main actor-isolated property 'goalSpeed' can not be mutated from a non-isolated context” (goalSpeed being one of those parameters). Is there a way to allow t1Vehicle and t2Vehicle on a background thread, perhaps by disabling their physicsBodies or similar?

t1Vehicle & t2Vehicle are not intended to themselves alter the display. They are intended only as a means of accessing the data as it was when the ‘findObstacles’ function was called. If there’s another way to reference this data, that would also solve the problem.

The relevant parts of the code are shown below:

//this code is on the main thread
Task {
    let result = await findObstacles()
    let t1Vehicle = result.t1Vehicle
    let t2Vehicle = result.t2Vehicle
}
//Function called from the Task above
func findObstacles() async -> (t1Vehicle: Vehicle, t2Vehicle: Vehicle) {
//Create copy of vehicles for calculating proximity to other vehicles
//Do calculations on another thread
    var t1Vehicle = track1Vehicles.dropFirst()      //Straight Track Vehicles: Ignore element [0]
    t1Vehicle.sort(by: {$0.position.y < $1.position.y}) //Sort into positional order, 000 - 999.999
    var t2Vehicle = track2Vehicles.dropFirst()
    t2Vehicle.sort(by: {$0.position.y < $1.position.y}) //Sort into positional order, 000 - 999.999

//processing of data here..
    
    return (t1Vehicle, t2Vehicle)
}

ps. the 'Vehicle' type is an ObservableObject. A number of its parameters were also Published variables. I tried removing the '@Published' from the parameter 'goalSpeed' and it made no difference - the error still occurred on it.

I'm getting the impression that nodes can only be accessed from the main thread. With my code I am copying 2 arrays of nodes and use those copies in the background thread/s. The original arrays are not altered. They could even be copied prior to going into the background thread. I need access to things such as position and velocity for each node and there can potentially be 300 of them. A fast & simple way to copy these parameters from all nodes into a normal array would do. Currently my copy still contains nodes even though I don't display them and only require read access.

Answered by DTS Engineer in 730705022

I was beginning to think the only way I could achieve a result was by iterating over every SKNode, extracting information such as position, speed and size then writing all this into another array.

There are two ways you can approach APIs like SpriteKit:

  • You can keep all your state in the SpriteKit nodes. If you do that then, yeah, you will find yourself having to read data out of those nodes if you want to do other custom processing on it.

  • You can keep all of your state in your own model objects and then push any changes to the SpriteKit nodes.

Which approach to choose depends on how much SpriteKit is doing for you. If you can do the vast bulk of your processing in SpriteKit, then the first option is ideal. If not, the second option tends to be a better choice.

Coming back to this specific error:

The second line has the error Cannot assign value of type '[Any]' to type 'Array.SubSequence' (aka ‘ArraySlice’).

You’re getting this because Array and ArraySlice are different things. Consider this snippet:

let i = [1, 2, 3]
let s = i.dropFirst()

If you option click on i, it’ll display a type of [Int], that is, Array<Int>. If you option click on s, you get Array<Int>.SubSequence, aka ArraySlice<Int>. This is a common source of confusion in Swift.

The difference exists for performance reasons. Compare these two snippets:

Array(i.dropFirst().dropLast())
Array(Array(i.dropFirst()).dropLast())

In the first example, all the intermediate processing is done with slices and there’s a single conversion back to an array at the end. This can be a huge performance win.

In contrast, the second example shows what you get in other languages, ones without the concept of slices. Each processing operation creates a new array, which can be expensive.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

t1Vehicle & t2Vehicle are not intended to themselves alter the display. They are intended only as a means of accessing the data as it was when the findObstacles() function was called.

What I would do in this situation is create my own value that represents the state that I want to be Sendable, copy the vehicle info to that, send that between the isolation contexts, work on it there, send it back, and then apply it to the nodes on the main thread. That’ll avoid any questions about the sendabilility of SpriteKit nodes.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thanks for your feedback Quinn. I was beginning to think the only way I could achieve a result was by iterating over every SKNode, extracting information such as position, speed and size then writing all this into another array. Iterating through hundreds of nodes this way would be time consuming in itself and far from elegant.

I wasn’t aware of ‘Sendable’ before now but it does sound like it could do the job. A little research I did just now indicates that I should also make a ‘deep copy’ of the original. I’ve quickly tried this without success. If you could shed any light it would be appreciated.

I haven’t added the ‘Sendable’ part yet. Figured I need a deep copy first. The code I tried was:

    var t1Vehicle = sKLAllVehicles.dropFirst()
    t1Vehicle = t1Vehicle.map{ $0.copy() }		//<- error here
    Task {
        let result = await findObstacles()
        let t1Vehicle = result.t1Vehicle
        let t2Vehicle = result.t2Vehicle
    }

The second line has the error “Cannot assign value of type '[Any]' to type 'Array.SubSequence' (aka ‘ArraySlice’)”.

Any suggestions to solve this and include the Sendable option? Also, should I be returning values from the function as type ‘Vehicle’ or would this also have to change? (Sorry for being a pain but the only coding I’ve done before was assembler so I’m on a steep learning curve!)

Accepted Answer

I was beginning to think the only way I could achieve a result was by iterating over every SKNode, extracting information such as position, speed and size then writing all this into another array.

There are two ways you can approach APIs like SpriteKit:

  • You can keep all your state in the SpriteKit nodes. If you do that then, yeah, you will find yourself having to read data out of those nodes if you want to do other custom processing on it.

  • You can keep all of your state in your own model objects and then push any changes to the SpriteKit nodes.

Which approach to choose depends on how much SpriteKit is doing for you. If you can do the vast bulk of your processing in SpriteKit, then the first option is ideal. If not, the second option tends to be a better choice.

Coming back to this specific error:

The second line has the error Cannot assign value of type '[Any]' to type 'Array.SubSequence' (aka ‘ArraySlice’).

You’re getting this because Array and ArraySlice are different things. Consider this snippet:

let i = [1, 2, 3]
let s = i.dropFirst()

If you option click on i, it’ll display a type of [Int], that is, Array<Int>. If you option click on s, you get Array<Int>.SubSequence, aka ArraySlice<Int>. This is a common source of confusion in Swift.

The difference exists for performance reasons. Compare these two snippets:

Array(i.dropFirst().dropLast())
Array(Array(i.dropFirst()).dropLast())

In the first example, all the intermediate processing is done with slices and there’s a single conversion back to an array at the end. This can be a huge performance win.

In contrast, the second example shows what you get in other languages, ones without the concept of slices. Each processing operation creates a new array, which can be expensive.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Getting closer. I tried copying the data before accessing the background thread. The issue I have now is that Swift considers 't1Vehicle' a constant whereas I need it to be a variable. I used a struct here which I have not done before (perhaps there's a better way?). The error code when trying to sort t1Vehicle is "Cannot use mutating member on immutable value: 't1Vehicle' is a 'let' constant". I've tried a few things to get around this without success. It only allows me to make a method mutating.

In the code below I've only included the data from one track for readability.

//This code on Main Thread!
var temp1 = sKLAllVehicles.dropFirst()      //Straight Track Vehicles: Ignore 'All Vehicles'
var t1Vehicle: [NodeData] = []

for (index, veh1Node) in temp1.enumerated() {
    t1Vehicle[index].name = veh1Node.name!
    t1Vehicle[index].size = veh1Node.size
    t1Vehicle[index].position = veh1Node.position
    t1Vehicle[index].lane = veh1Node.lane
    t1Vehicle[index].laps = veh1Node.laps
    t1Vehicle[index].speed = veh1Node.physicsBody!.velocity.dy
}

Task {
    let result = await findObstacles(t1Vehicle: t1Vehicle)
    let t1Vehicle = result.t1Vehicle
    let t2Vehicle = result.t2Vehicle
}

...and I included the function as a method within a struct (I'd never done this before!)

struct NodeData {
    var name: String
    var position: CGPoint
    var size: CGSize
    var lane: CGFloat
    var speed: CGFloat
    var laps: CGFloat

    mutating func findObstacles(t1Vehicle: inout [NodeData]) async -> (t1Vehicle: [NodeData], t2Vehicle: [NodeData]) {
        //  Do calculations on background thread
        
        t1Vehicle.sort(by: {$0.position.y < $1.position.y}) //ERROR: Cannot use mutating member on immutable value: 't1Vehicle' is a 'let' constant
        
        //Rest of method here
       
    }
}

Any idea how to correct this? Thanks!

When you hit an error like this it’d help if you could highlight the specific line where you’re seeing it. In this case I suspect that it’s here:

Task {
    let result = await findObstacles(t1Vehicle: t1Vehicle)
    …

Is that right?

If so, it’s because the closure in your task has captured the t1Vehicle vehicle array as a constant. You can override this using a capture list but that would be a mistake in this case. If you’re going to work with the array in a separate task, you need the task to gets its own copy of the array to prevent you from accidentally sharing mutable state between tasks.

I tried adding the following line in front of the sort:

var t1Vehicle = t1Vehicle

The error then disappeared.

That's probably the right option here. The task captures the array as a constant and then you explicitly make a read/write copy of it.

Problem is I can't seem to access its properties such as position and size.

That should work. I suspect that the real issue is this:

var t1Vehicle: [NodeData] = []

for (index, veh1Node) in temp1.enumerated() {
    t1Vehicle[index].name = veh1Node.name!
    t1Vehicle[index].size = veh1Node.size
    t1Vehicle[index].position = veh1Node.position
    t1Vehicle[index].lane = veh1Node.lane
    t1Vehicle[index].laps = veh1Node.laps
    t1Vehicle[index].speed = veh1Node.physicsBody!.velocity.dy
}

There are a bunch of problems here. First, be very careful about using enumerated(), because the indexes it returns aren’t indexes in the source collection but rather offsets. This doesn’t matter in this case, but it’s something to watch out for.

However, the real problem is the way that you build t1Vehicle. On the first line you initialise it to the empty array and then you never extend it. So it remains empty. Honestly, I’m surprised that constructs like t1Vehicle[index].name aren’t trapping with an array out of bounds.

A more common way to do this would be to write:

var t1Vehicle: [NodeData] = []

for veh1Node in temp1 {
    let newNode = NodeData(…)
    t1Vehicle.append(newNode)
}

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn, thanks for the feedback, it helped a lot. Answers to your questions and revised code follows:

No, the error appeared on the line where the sort occurred on t1Vehicle. This was mentioned and I included the error description to the comment on that line of code to reaffirm.

In the Task I had “let result = “ as I never alter the values I get back. I only need to read them back on the main thread and use these within some SKActions. They are altered during the function though, so I’ll change it to var for now in case it makes any difference. The changes occur before they are returned.

I think both adding the “var t1Vehicle = t1Vehicle” and the “inout” work. The reason I didn’t see the properties was my fault. I’d added a print instruction at the start of the function just to check and I was looking for “t1Vehicle.position”. I should have been looking for “t1Vehicle[x].position” where x is the offset. With this even the code I had worked.

You’re correct about me not needing the ‘enumerated’ in this loop. It was a hangover from my initial code. This will still be done within the function as the index tells me in what order along the road each vehicle is - they are sorted accordingly. Not needed in the main thread when creating the new array.

I suspect you’re right about never extending the array. I’ve never used one this way before. Guess I assumed the loop would progressively fill the array. No errors including ‘out of bounds’ came up. I’m not real sure what you meant by “NodeData (…)” however I made a change I think should work. See below..

//Run on Main Thread!
var temp1 = sKLAllVehicles.dropFirst()
var t1Vehicle: [NodeData] = []
var nodeData: NodeData = NodeData()

for (index, veh1Node) in temp1.enumerated() {
    nodeData.name = veh1Node.name!
    nodeData.size = veh1Node.size
    nodeData.position = veh1Node.position
    nodeData.lane = veh1Node.lane
    nodeData.laps = veh1Node.laps
    nodeData.speed = veh1Node.physicsBody!.velocity.dy
    t1Vehicle.append(nodeData)
}

Although not shown here, I added an initialiser to the NodeData struct.

Concurrent Processing of SKSpriteNode Copied Data in SpriteKit
 
 
Q