For the MaterialX shadergraph, the given example hard-codes two textures for blending at runtime ( https://developer.apple.com/documentation/visionos/designing-realitykit-content-with-reality-composer-pro#Build-materials-in-Shader-Graph ) Can I instead generate textures at runtime and set what those textures are as dynamic inputs for the material, or must all used textures be known when the material is created? If the procedural texture-setting is possible, how is it done, since the example shows a material with those hard-coded textures? EDIT: It looks like the answer is ”yes” since setParameter accepts textureResources https://developer.apple.com/documentation/realitykit/materialparameters/value/textureresource(_:)?changes=l_7 However, how do you turn a MTLTexture into a TextureResource?

Does Xcode 16 beta 4 include the SDK for iPadOS 18.1 beta, or should I stick with iPadOS 18.0 beta 4/18.0 betas? I want to make sure that I can continue building to my device if I update it to 18.1 beta.

Apparenrly, shadows aren’t generated for procedural geometry in RealityKit: https://codingxr.com/articles/shadows-lights-in-realitykit/ Has this been fixed? My projects tend to involve a lot of procedurally-generated meshes as opposed to importes-models. This will be even more important when VisionOS is out. On a similar note, it used to be that ground shadows were not per-entity. I’d like to enable or disable them per-entity. Is it possible? Since currently the only way to use passthrough AR in Vision OS will be to use RealityKit, more flexibility will be required. I can‘t simply apply my own preferences.

I wonder if an Apple engineer could confirm: will the Apple Pencil Pro squeeze functionality be detectable in the current API, or will this be a future iPadOS extension to gesture recognizers / UIKit? I’d like to start playing with the functionality if it’s detected behind an existing event though. (Long press?)

I am a spatial computing / XR and Human-Computer Interaction researcher from a private university. I am interested in using the vision pro's newly-exposed camera access to develop and evaluate new algorithms for computational perception. ( WWDC session here: https://developer.apple.com/wwdc24/10139 ) I understand this is targeted at large enterprises, but I would like to know if by some means as a researcher affiliated with an educational institution I could develop private for-development-only applications for the vision pro with the enterprise APIs enabled. The intent is not to publish apps, but rather to contribute to the research community through R&D. However, to my knowledge, I would be ineligible as a normal "business" as I do not employee 100+ employees. I am an independent researcher, and on occasion, I collaborate within small research groups within my university that focus on this kind of camera-based perception algorithm development. Could someone from Apple comment? Thank you.

It’s great that we’ll be able to use Metal custom renderers in passthrough mode on visionOS. https://developer.apple.com/wwdc24/10092 This is a lot of complicated set-up, however. It’s also unclear how occlusion and custom algorithms / raytracing will work in tandem with scene understanding. May we have a project template and/or sample? Preferably with the C api and not just swift. This would be much-appreciated and helpful to everyone who wants this set-up. I’d like to see the whole process. Thank you for introducing this feature!

I'm developing an application that needs smooth framerates within a wkwebview that interacts with native code. However, requestAnimationFrame by default is still throttled to 60hz even if all my target devices (the iPad Pro for example) have supported 120hz for a long time already. I noticed that the latest Safari in 18.3 beta supports unlocked framerates, but that's only under Safari feature flags. To my knowledge, these flags do not apply to WKWebView. Is there a way to enable unlocked framerate in WKWebView via requestAnimationFrame? (Calling JS at a faster rate from the native code side will not work, almost definitely, since WKWebView will still render at its own rate.) This is an experimental application for internal use and I'm okay if there are temporary beta solutions available.

This is related to the WWDC presentation, What's new in Metal rendering for immersive apps.. Specifically, the macOS spatial streaming to visionOS feature: For reference: the page in the docs. The presentation demonstrates it using a full immersive space and Metal rendering using compositor services. I'd like clarity on a few things: Is the remote device wireless, or must the visionOS device be connected via a wired connected? Is there a limit to the number of remote devices, and if not, could macOS render different things per remote device simultaneously? Can I also use mixed mode with passthrough enabled, instead of just a fully-immersive mode? Can I use RealityKit instead of Metal? If so, may I have an example, or would someone point to an example?

A lab I very much wanted to attend this Friday was assigned at the absolute worst time that conflicts with a critical business meeting. It was a gamble. Is there any possibility to reschedule within a more specific time range? Thank you for your time.

Xcode 13.4 only provides an SDK for macOS 12.3 according to the release notes. Can I build to macOS 12.4 using the lower point version SDK? I would not want to update the OS if I could not build to it yet. Thanks.

I wanted to try structured logging with os_log in C++, but I found that it fails to print anything given a format string and a variable: eg. void example(std::string& str) { os_log_info(OS_LOG_DEFAULT, "%s", str.c_str()); os_log_debug(OS_LOG_DEFAULT, "%s", str.c_str()); os_log_error(OS_LOG_DEFAULT, "%s", str.c_str()); } This prints a blank row in the console, but with no text. How is this meant to work with variables? It only works with literals and constants now as far as I can tell. I'm looking forward to getting this working.

On VisionOS, is a combination of full passthrough, unbounded volumes, and my own custom 3D rendering in Metal Possible? According to the RealityKit and Unity VisionOS talk, towards the end, it’s shown that an unbounded volume mode allows you to create full passthrough experiences with graphics rendering in 3D — essentially full 3D AR in which you fan move around the space. It’s also shown that you can get occlusion for the graphics. This is all great, however, I don’t want to use RealityKit or Unity in my case. I would like to be able to render to an unbounded volume using my own custom Metal renderer, and still get AR passthrough and the ability to walk around and composit virtual graphical content with the background. To reiterate, this is exactly what is shown in the video using Unity, but I’d like to use my own renderer instead of Unity or RealityKit. This doesn’t require access to the video camera texture, which I know is unavailable. Having the flexibility to create passthrough mode content in a custom renderwr is super important for making an AR experience in which I have control over rendering. One use case I have in mind is: Wizard’s Chess. You see the real world and can walk around a room-size chessboard with virtual chess pieces mixed with the real world, and you can see the other player through passthrough as well. I’d like to render graphics on my living room couches using scene reconstruction mesg anchors, for example, to change the atmosphere. The video already shows several nice use cases like being able to interact with a tabletop fantasy world with characters. Is what I’m describing possible with Metal? Thanks! EDIT: Also, if not volumes, then full spaces? I don’t need access to the camera images that are off-limits. I would just like passthrough + composition with 3D Metal content + full ARKit tracking and occlusion features.

Related to “what you can do in visionOS,” what are all of these camera-related functionalities for? (As of yet, not described in the documentation) https://developer.apple.com/documentation/realitykit/realityrenderer/cameraoutput/colortextures https://developer.apple.com/documentation/realitykit/realityrenderer/cameraoutput/relativeviewport What are the intended use cases? Is this the equivalent to render-to-texture? I also see some interop with raw Metal happening here.

I’m still a little unsure about the various spaces and capabilities. I’d like to make full use of hand tracking, joints and all. In the mode with passthrough and a single application present (not a shared space), is that available? (I am pretty sure that the answer is “yes,” but I’d like to confirm.) What is this mode called in the system? Mixed full-space?

I’d like to use ARKit world tracking and display both the back camera feed and the front camera feeds, using the front feed as as a PIP. This would work great for an internet streaming use case. However, it’s impossible. As soon as ARKit is told to use one mode, the camera for the other side freezes/doesn’t work. This page also says you have to pick one camera to show: https://developer.apple.com/documentation/arkit/arkit_in_ios/choosing_which_camera_feed_to_augment?language=objc A question to the developers: why is this limitation in-place? Are there any work-arounds for the use case of ARKit world tracking + displaying the back camera feed + displaying the front camera feed as an overlay? It’s possible to do this with plain camera initialization without ARKit. (There’s an official example.) With ARKit, it no longer works. It’s strange that I cannot access the front feed via one of the other frameworks, but I guess that ARKit blocks that.