ARKit problems with U shaped tables and hallways

Question

Created May ’22

Replies 4

Boosts 0

Participants 3

I've been working with ARKit plane detection and have noticed a few problems. The desk in my office is basically U shaped but when ARKit always turns it into one big plane with the center part closed off. Shouldn't it really detect this as 3 planes arranged in a U shape so the center part is empty?

Having similar problems with rooms that open into hallways. The walls of the room get recognized and turned into big planes which leaves the open doorway covered up by the plane representing the wall it's in.

This doesn't seem right, am I doing something wrong or is this the way it's supposed to work?

Boost

Answer 1

makabaka1880 OP

May ’22

Strange Bug?

0

Answer 2

Vision Pro Engineer OP

Apple

Jun ’22

ARKit merges adjacent planes into one, with a convex outline (see also https://developer.apple.com/documentation/arkit/arplanegeometry). So what you are seeing is the expected behavior. For a more detailed representation of the environment, you can take a look at the Scene Reconstruction API (requiring a LiDAR device).

0

Answer 3

Greg Corson OP

Jun ’22

I understand planes are merged, but this shouldn't happen if it produces wildly wrong results! See the photo below.

In the top one you can see the opening into the hallway has been partly closed off by merging the plane above the hole with the one to the right This should have been represented as 2 rectangular planes, leaving the hallway open.

In the bottom row you can see how the open part of the U shaped desk was completely filled in. These desks should have been represented as 4 planes with the open part left open.

I've seen similar behavior with L shaped tables being represented as triangular planes.

It seems ARKit is being overly aggressive at merging planes, to the point of generating very wrong results. If I use these planes for the placement of objects in an app it would result in things sitting on thin air.

I don't expect a perfect representation from planes, but it doesn't seem right to merge them in a way that extends them into large empty spaces.

0

Answer 4

Vision Pro Engineer OP

Apple

Jun ’22

Hi Greg! Plane representations in ARKit are limited to convex geometries, with the consequence that open parts will be filled in. While this is often the desired behavior, there are cases where the merged plane is not the intended result. Please feel free to file a feedback report on http://feedbackassistant.apple.com/.

What you can try for placing objects in the environment shown in the image is to turn off plane detection and use a raycast with target .estimatedPlane. This will disregard existing planes and just estimate a plane in the vicinity of the ray. If scene reconstruction is turned on (on LiDAR devices), and the target alignment is set to .any, the raycast will intersect the scene mesh for even more accurate results.

0