Automated testing via session replay

Question

Created 3d

Replies 2

Boosts 0

Participants 2

This post is from the WWDC26 ARKit Q&A.

Is there a supported route to automated regression testing of ARKit apps? Reality Composer's "Record AR Session" plus Xcode's "ARKit Replay data" scheme option work well for manual debugging, but replay isn't wired into XCUITest, doesn't run in CI, and ARKit doesn't exist in the simulator — so every AR regression today needs to be run by a human holding a device. Even replay-driven ARSession in the simulator, or an XCTest hook for selecting replay files, would unlock some automated coverage.

Answered by Vision Pro Engineer in 891412022

ARKit replay via Reality Composer and Xcode is the supported path today.

If you don't need the CVPixelBuffer, you can also serialize the relevant properties of ARFrame out of the session delegate to a file and replay them into your own test harness. That unlocks a meaningful slice of regression coverage.

Could you share more detail on what exactly you're trying to cover? With that, please file a feature request in Feedback Assistant describing the concrete use case.

Answer 1

Vision Pro Engineer OP

Apple

3d

Recommended

ARKit replay via Reality Composer and Xcode is the supported path today.

If you don't need the CVPixelBuffer, you can also serialize the relevant properties of ARFrame out of the session delegate to a file and replay them into your own test harness. That unlocks a meaningful slice of regression coverage.

Could you share more detail on what exactly you're trying to cover? With that, please file a feature request in Feedback Assistant describing the concrete use case.

Answer 2

McDuffman OP

2d

Sure! My AR app's 3D interaction logic is regression-tested today only by device-tier XCUITests against the live camera (i.e. me holding the device while it adds a model, checks it can select it with a tap, and transform it as expected). In practice that caps how much AR coverage I can maintain, since every regression pass needs a human and a device.

I keep a library of recorded session .movs that reproduce specific scenarios; Xcode replays them fine under the Run action, but the scheme's replay setting (DeviceSensorReplayFileReference) doesn't apply to the Test action (and isn't valid inside one) so XCUITest always gets the live camera. Replay in tests would also unlock checks I can't do at all today: deterministically verifying that a saved AR scene relocalizes correctly on reload, which right now only works if I'm physically standing in the room where the scene was saved.

What I'd want, in order of preference:

a supported way to attach a replay file per test plan or per test, so AR XCUITests can run unattended (locally or in CI);
failing that, ARSession replay in the simulator.

I'll file Feedback with this use case, and I'll try the ARFrame-serialization harness you suggested for the unit tier in the meantime. That seems to cover the math/logic layer nicely, though not the render/hit-test path the XCUITests exercise, so the replay ask above still stands. Thanks!