Post

Replies

Boosts

Views

Activity

Reply to Is `.appEntityIdentifier` + `Transferable` the intended way to let Siri send an on-screen image to another app? (iOS 27)
Thanks, J0hn, that did it! Adopting the .files.file schema (@AppEntity(schema: .files.file)) with a FileEntityIdentifier and a FileRepresentation exporting a SentTransferredFile, then annotating the on-screen image with .appEntityIdentifier(...), I can now say "send this to " and Siri lifts the on-screen (filtered) image out to another app. Confirmed on device (iOS 27). But still, I wonder if this more generic approach shouldn’t also work. Before switching to .files.file, I had a plain custom AppEntity annotated on-screen with the same .appEntityIdentifier modifier, conforming to Transferable with image DataRepresentations (JPEG/PNG). That entity's EntityQuery.entities(for:) was essentially never called (the annotation wasn't consumed) whereas the otherwise-identical setup with a .files.file schema entity resolves and transfers fine. Is on-screen consumption intended to be limited to the predefined assistant schemas, or should a custom AppEntity + Transferable also be a supported way to expose arbitrary on-screen content, as "Making onscreen content available to Siri and Apple Intelligence" seems to imply? Related: .files.file needs a real file on disk at resolution time (the system builds the entity's display representation from it), so an in-memory image representation doesn't appear possible. I’d have expected a lighter-weight path for transient content, similar to or via IntentFile.
Jul ’26
Reply to Managed Asset Pack not found by TestFlight reviewer
So I just had a chat with App Review. It seems this is indeed a technical issue with the review environment, where the app doesn’t have access to the on-demand asset pack. While they are working on fixing this, I got the following recommendation: In the review notes, before submitting your app, explain this problem in detail to the reviewer. Explain the current issue with the review environment. Also explain what this asset pack is used for and what it would enable. Ideally, you would also provide a way to use the app without the asset pack, for instance, by having a demo mode the reviewer could access somehow. It’s important to put this into the review notes, as the reviewer will read this before testing the app, potentially avoiding a quick rejection. Let’s hope they will find the issue with the review environment soon. 🤞
Jun ’26
Reply to Apple-Hosted Asset Pack Support in App Review
We are facing the same problem right now. The iOS version of our app went through review just fine, but the macOS version (Catalyst) is stuck. We have an on-demand asset pack that we download at the start of the app. We first present the user with the download size and the option to download the asset. The reviewer reports an error that indicates that even the lookup of the asset pack (assetPack(withID:)) doesn't work. On all our test devices we don't have that issue. It seems the reviewer is running the app in an environment that is unable to load the asset pack information from TestFlight. We don't know what to do here. Any help would be appreciated!
May ’26
Reply to Core Image for depth maps & segmentation masks: numeric fidelity issues when rendering CIImage to CVPixelBuffer (looking for Architecture suggestions)
The problem might be this: Core Image uses 16-bit float RGBA as the default working format. That means that, whenever it needs an intermediate buffer for the rendering, it will create a 4-channel 16-bit float surface to render into. This also meant that your 1-channel unsigned integer values will automatically be mapped to float values in 0.0...1.0. That's probably where you lose precision. There are a few options to circumvent this: You could set the workingFormat context option to .L8 or .R8. However, this means all intermediate buffers will have that format. If you want to mix processing of the segmentation mask with other images, this won't work. If you only want to process the mask separately, you can set up a separate CIContext with this option. Note, however, that most built-in CIFilters assume a floating-point working format and might not perform well with this format. You can process your segmentation map with Metal (as you suggested) as part of your CIFilter pipeline using a CIImageProcessorKernel. For the kernel, you can set the formatForInput(...) and the outputFormat to .R8. This should tell CI that it doesn't need to convert the segmentation mask before passing it to your processor kernel. In the process method, you can access the input's Metal texture and perform custom Metal processing with it, rendering into the output's texture (which is then also R8 format). This way, you won't lose any precision. I think the second option is the best choice here as you get the best of both worlds (custom Metal processing + CI integration). Tip: You can always use CI_PRINT_TREE to check the format of the intermediate buffers CI is using during rendering.
Topic: Machine Learning & AI SubTopic: General Tags:
Feb ’26
Reply to BGContinuedProcessingTask GPU access — no iPhone support?
Good question! I haven't seen the new GPU background entitlements. Good to know! My guess is that they don't want to support iPhone here because they prioritize battery over expensive processing on the mobile devices. The iPads on the other hand are more targeted towards pro use cases (and have a bigger battery). What we are currently doing in our apps is to prevent the screen from locking during video export (using UIApplication.shared.isIdleTimerDisabled = true). We also pause the processing when the app is backgrounded and resume when it's active again.
Topic: Graphics & Games SubTopic: Metal Tags:
Feb ’26
Reply to Is `.appEntityIdentifier` + `Transferable` the intended way to let Siri send an on-screen image to another app? (iOS 27)
Thanks, J0hn, that did it! Adopting the .files.file schema (@AppEntity(schema: .files.file)) with a FileEntityIdentifier and a FileRepresentation exporting a SentTransferredFile, then annotating the on-screen image with .appEntityIdentifier(...), I can now say "send this to " and Siri lifts the on-screen (filtered) image out to another app. Confirmed on device (iOS 27). But still, I wonder if this more generic approach shouldn’t also work. Before switching to .files.file, I had a plain custom AppEntity annotated on-screen with the same .appEntityIdentifier modifier, conforming to Transferable with image DataRepresentations (JPEG/PNG). That entity's EntityQuery.entities(for:) was essentially never called (the annotation wasn't consumed) whereas the otherwise-identical setup with a .files.file schema entity resolves and transfers fine. Is on-screen consumption intended to be limited to the predefined assistant schemas, or should a custom AppEntity + Transferable also be a supported way to expose arbitrary on-screen content, as "Making onscreen content available to Siri and Apple Intelligence" seems to imply? Related: .files.file needs a real file on disk at resolution time (the system builds the entity's display representation from it), so an in-memory image representation doesn't appear possible. I’d have expected a lighter-weight path for transient content, similar to or via IntentFile.
Replies
Boosts
Views
Activity
Jul ’26
Reply to Managed Asset Pack not found by TestFlight reviewer
So I just had a chat with App Review. It seems this is indeed a technical issue with the review environment, where the app doesn’t have access to the on-demand asset pack. While they are working on fixing this, I got the following recommendation: In the review notes, before submitting your app, explain this problem in detail to the reviewer. Explain the current issue with the review environment. Also explain what this asset pack is used for and what it would enable. Ideally, you would also provide a way to use the app without the asset pack, for instance, by having a demo mode the reviewer could access somehow. It’s important to put this into the review notes, as the reviewer will read this before testing the app, potentially avoiding a quick rejection. Let’s hope they will find the issue with the review environment soon. 🤞
Replies
Boosts
Views
Activity
Jun ’26
Reply to Apple-Hosted Asset Pack Support in App Review
We are facing the same problem right now. The iOS version of our app went through review just fine, but the macOS version (Catalyst) is stuck. We have an on-demand asset pack that we download at the start of the app. We first present the user with the download size and the option to download the asset. The reviewer reports an error that indicates that even the lookup of the asset pack (assetPack(withID:)) doesn't work. On all our test devices we don't have that issue. It seems the reviewer is running the app in an environment that is unable to load the asset pack information from TestFlight. We don't know what to do here. Any help would be appreciated!
Replies
Boosts
Views
Activity
May ’26
Reply to Core Image for depth maps & segmentation masks: numeric fidelity issues when rendering CIImage to CVPixelBuffer (looking for Architecture suggestions)
The problem might be this: Core Image uses 16-bit float RGBA as the default working format. That means that, whenever it needs an intermediate buffer for the rendering, it will create a 4-channel 16-bit float surface to render into. This also meant that your 1-channel unsigned integer values will automatically be mapped to float values in 0.0...1.0. That's probably where you lose precision. There are a few options to circumvent this: You could set the workingFormat context option to .L8 or .R8. However, this means all intermediate buffers will have that format. If you want to mix processing of the segmentation mask with other images, this won't work. If you only want to process the mask separately, you can set up a separate CIContext with this option. Note, however, that most built-in CIFilters assume a floating-point working format and might not perform well with this format. You can process your segmentation map with Metal (as you suggested) as part of your CIFilter pipeline using a CIImageProcessorKernel. For the kernel, you can set the formatForInput(...) and the outputFormat to .R8. This should tell CI that it doesn't need to convert the segmentation mask before passing it to your processor kernel. In the process method, you can access the input's Metal texture and perform custom Metal processing with it, rendering into the output's texture (which is then also R8 format). This way, you won't lose any precision. I think the second option is the best choice here as you get the best of both worlds (custom Metal processing + CI integration). Tip: You can always use CI_PRINT_TREE to check the format of the intermediate buffers CI is using during rendering.
Topic: Machine Learning & AI SubTopic: General Tags:
Replies
Boosts
Views
Activity
Feb ’26
Reply to BGContinuedProcessingTask GPU access — no iPhone support?
Good question! I haven't seen the new GPU background entitlements. Good to know! My guess is that they don't want to support iPhone here because they prioritize battery over expensive processing on the mobile devices. The iPads on the other hand are more targeted towards pro use cases (and have a bigger battery). What we are currently doing in our apps is to prevent the screen from locking during video export (using UIApplication.shared.isIdleTimerDisabled = true). We also pause the processing when the app is backgrounded and resume when it's active again.
Topic: Graphics & Games SubTopic: Metal Tags:
Replies
Boosts
Views
Activity
Feb ’26