Post

Replies

Boosts

Views

Activity

CMIO Camera Extension user experience
I am developing a CMIO Camera Extension on macOS Ventura. Initially, I based this on the template camera extension (which creates its own frames). Later, I added a sink stream so that I could send the extension video from an app. That all works. Recently, I added the ability for the extension itself to initiate a capture session, so that it can augment the video from any available AVCaptureDevice without running its controlling app. That works, but I have to add the Camera capability to the extension's sandbox configuration, and add a camera usage string. This caused the OS to put up the user permission dialog, asking for permission to use the camera. However, the dialog uses the extension's bundle ID for its name, which is long and not user friendly. Furthermore, the extension isn't visible to the user (it is packaged inside the app which installs and controls it), so even a user-friendly name doesn't make that much sense to the end user. I tried adding a CFBundleDisplayName to the extension's plist, but the OS didn't use it in the permissions dialog. Is there a way to get the OS to present a more user-friendly name? Should I expect to see a permissions dialog pertaining to the extension at all? Where does the OS get the name from? After the changes (Camera access, adding a camera usage string), I noticed that the extension's icon (the generic extension icon) showed up in the dock, with its name equal to its bundle ID. Also, in Activity Monitor, the extension's process is displayed, using its CFBundleDisplayName (good). But about 30s after activation, the name is displayed in red, with " (not responding)" appended, although it is still working. The extension does respond to the requests I send it over the CMIO interface, and it continues to process video, but it isn't handling user events, while the OS thinks that it should, probably because of one or more of the changes to the plist that I have had to make. To get the icon out of the dock, I added LSUIElement=true to its plist. To get rid of the red "not responding", I changed the code in its main.swift from the template. It used to simply call CFRunLoopRun(). I commented out that call and instead make this call _ = NSApplicationMain(CommandLine.argc, CommandLine.unsafeArgv) That appears to work, but has the unfortunate side effect of increasing the CPU usage of the extension when it is idle from 0.3% to 1.0%. I do want the extension to be able to process Intents, so there is a price to be paid for that. But it doesn't need to do so until it is actively dealing with video. Is there a way to reduce the CPU usage of a background app, perhaps dynamically, making a tradeoff between CPU usage and response latency? Is it to be expected that a CMIOExtension shows up in the Dock, ever?
0
0
958
Jul ’23
CoreMediaIO object hierarchy and controls
How does one get the list of controls which a CMIOObject has to offer? How do the objects in the CMIO hierarchy map to CMIOExtension objects? I expected the hierarchy to be something like this: the system has owned objects of type: 'aplg' `(kCMIOPlugInClassID)` has owned objects of type 'adev' `(kCMIODeviceClassID,` which may have owned objects of type 'actl' `(kCMIOControlClassID)` and has at least one owned object of type 'astr' `(kCMIOStreamClassID),` each of which may have owned objects of type 'actl' `(kCMIOControlClassID)` Instead, when I recursively traverse the object hierarchy, I find the devices and the plug-ins at the same level (under the system object). Only some of the device in my system have owned streams, although they all have a kCMIODevicePropertyStreams ('stm#') property. None of the devices or streams appear to have any controls, and none of the streams have any owned objects. I'm not using the qualifier when searching for owned objects, because the documentation implies that it may be nil if I'm not interested in narrowing my search. Should I expect to find any devices or streams with controls? And if so, how do I get a list of them? CMIOHardwareObject.h says that "Wildcards... are especially useful ...for querying an CMIOObject's list of CMIOControls. ", but there's no example of how to do this. My own device (from my camera extension) has no owned objects of type stream. I don't see any API call to convey ownership of the stream I create by the device it belongs to. How does the OS decide that a stream is 'owned' by a device? I've tried various scopes and elements - kCMIOObjectPropertyScopeGlobal, kCMIOObjectPropertyScopeWildcard, kCMIOControlPropertyScope, and kCMIOObjectPropertyElementMain, kCMIOObjectPropertyElementWildcard and kCMIOControlPropertyElement. I can't get a list of controls using any of these. Ultimately, I'm trying to find my provider, my devices and my streams using the CMIO interface, so that I can set and query properties on them. Is it reasonable to assume that the CMIOObject of type 'aplg' is the one corresponding to a CMIOExtensionProviderSource? This is on Ventura 13.4.1 on M1.
0
0
878
Jul ’23
persist multiple window state in macOS app
I'd like to support multiple windows in my macOS app, which provides previews of cameras in the system, using the SwiftUI app life cycle, on macOS 13.5.2 and later. I can make multiple window without any problem, using the default behavior of WindowGroup and the File/New menu item. WindowGroup(id: "main-viewer", for: String.self) { $cameraUniqueID in ContentView(cameraUniqueID: cameraUniqueID) I can make a specific window on a camera using the .openWindow environment variable: .openWindow(id: "main-viewer", value:someSpecificCameraID) What I would like to be able to do is change the 'value' of my window at run time. When a user chooses "New Window", they get a window with a view of the first (or default) camera in it. They can then choose another camera to show in that window. I would like to be able to persist the chosen camera and the position and size of that window (originally opened with File/New Window). Windows opened with New Window are always opened with a nil value. Windows opened with .openWindow have their size and content saved, but I don't want to add UI to open specific windows. I want to open a generic window, then specify what camera it is looking at, move and resize it, and I'd like to save that window state. Is this possible, or am I holding SwiftUI wrong?
0
0
699
Sep ’23
how to inhibit -fprofile-instr-generate passed to linker
I'm struggling to build a driver for iPadOS in a particular project configuration. If I put the driver code and dext target into the same Xcode project which contains the iPad app, all is well. This is the way the Xcode driver template does it. However, I'd like to build and debug the dext on macOS, while eventually deploying on iPadOS. So I put the dext into a different project, which has a macOS target, a minimal iPadOS target and a DriverKit target. I made a workspace which contains both projects. I dragged the macOS project into the iPadOS project so that I can refer to the products of the macOS project (specifically, its driver target) as a dependency of the iPadOS target. Note that the main iPad app target depends on the driver target. So the workspace organization looks like this: Workspace iPad project main iPad app target (depends on driver) test project reference test project test macOS/iPad app target DriverKit dext target When I build the iPadOS target, it builds the dependent driver target in the macOS project, but it fails to link because Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/lib/darwin/libclang_rt.profile_driverkit.a is not found. If I just build the driver target directly in Xcode, there is no such complaint. I looked closely at the build logs, and I see for the failed link, there are these two linker flags set which are not set in the successful case -debug_variant -fprofile-instr-generate I can't seem to control the generation of this flag. I tried turning off the Profile switch in the Scheme editor for the driver, but is makes no difference. When I directly build the driver target, no -fprofile-instr-generate is set and it compiles and links. When i build the driver as a dependency of another target, -fprofile-instr-generate is passed to the linker, which fails. The obvious workaround is to put the driver source code into a separate driver target in the iPadOS project, but I'd rather have just one DriverKit driver for both platforms, with a few settings (such as bundle ID) controlled by a configuration file. Has anyone else encountered this problem, and know of a workaround?
1
0
1.1k
Oct ’23
USB DriverKit returning large asynchronous data
Can anyone advise, or give example of, communicating large (>128 byte) incoming buffers from a dext to a user-space app? My specific situation is interrupt reads from a USB device. These return reports which are too large to fit into the asyncData field of an AsyncCompletion call. Apple's CommunicatingBetweenADriverKitExtensionAndAClientApp sample shows examples of returning a "large" struct, but the example is synchronous. The asynchronous example returns data by copying into a IOUserClientAsyncArgumentsArray, which isn't very big. I can allocate a single buffer larger than 4K in user space, and communicate that buffer to my driver as an IOMemoryDescriptor when I set up my async callback. The driver retains the descriptor, maps it into its memory space and can thus write into it when the hardware returns interrupt data. The driver then calls AsyncCompletion, which will cause my user-side callback to be called, so the user side software knows that there's new data available in the previously allocated buffer. That's fine, it works, but there are data race problems - since USB interrupt reads complete whenever the hardware has provided data, incoming completions happen at unpredictable times, so the shared buffer contents could change while the user side code is examining them. Is there an example somewhere of how to deal with this? Can I allocate memory on the driver side on demand, create an IOMemoryDescriptor for it and return that descriptor packed inside the asyncData? If so, how does the driver know when it can relinquish that memory? I have a feeling there's something here I just don't understand...
1
0
886
Jan ’24
USB DriverKit returning large asynchronous data
this is a repost with more appropriate tags. The original is here: https://developer.apple.com/forums/thread/744268 Can anyone advise, or give example of, communicating large (>128 byte) incoming buffers from a dext to a user-space app? My specific situation is interrupt reads from a USB device. These return reports which are too large to fit into the asyncData field of an AsyncCompletion call. Apple's CommunicatingBetweenADriverKitExtensionAndAClientApp sample shows examples of returning a "large" struct, but the example is synchronous. The asynchronous example returns data by copying into a IOUserClientAsyncArgumentsArray, which isn't very big. I can allocate a single buffer larger than 4K in user space, and communicate that buffer to my driver as an IOMemoryDescriptor when I set up my async callback. The driver retains the descriptor, maps it into its memory space and can thus write into it when the hardware returns interrupt data. The driver then calls AsyncCompletion, which will cause my user-side callback to be called, so the user side software knows that there's new data available in the previously allocated buffer. That's fine, it works, but there are data race problems - since USB interrupt reads complete whenever the hardware has provided data, incoming completions happen at unpredictable times, so the shared buffer contents could change while the user side code is examining them. Is there an example somewhere of how to deal with this? Can I allocate memory on the driver side on demand, create an IOMemoryDescriptor for it and return that descriptor packed inside the asyncData? If so, how does the driver know when it can relinquish that memory? I have a feeling there's something here I just don't understand...
0
0
891
Jan ’24
unsatisfied entitlements macOS app
I recently built an update to one of our apps, which installs a driver extension. The new version won't launch on my Mac, Finder says it "can't be opened". I captured the logs, which say "no matching profile found": error 2024-01-10 14:36:03.306061 -0800 taskgated-helper <app-bundle-id>: Unsatisfied entitlements: com.apple.developer.system-extension.install, com.apple.developer.team-identifier info 2024-01-10 14:36:03.306279 -0800 amfid Requirements for restricted entitlements failed to validate, error -67671, requirements: '<private>' error 2024-01-10 14:36:03.306287 -0800 amfid Restricted entitlements not validated, bailing out. Error: Error Domain=AppleMobileFileIntegrityError Code=-413 "No matching profile found" UserInfo={NSURL=<private>, unsatisfiedEntitlements=<private>, NSLocalizedDescription=No matching profile found} default 2024-01-10 14:36:03.306432 -0800 amfid /Applications/<app-bundle-id>/Contents/MacOS/<app-name> not valid: Error Domain=AppleMobileFileIntegrityError Code=-413 "No matching profile found" UserInfo={NSURL=file:///Applications/C<escaped-app-name>/, unsatisfiedEntitlements=<CFArray 0x14f3041d0 [0x1dd7d39a0]>{type = immutable, count = 2, values = ( 0 : <CFString 0x14f3055a0 [0x1dd7d39a0]>{contents = "com.apple.developer.system-extension.install"} 1 : <CFString 0x14f304130 [0x1dd7d39a0]>{contents = "com.apple.developer.team-identifier"} )}, NSLocalizedDescription=No matching profile found} default 2024-01-10 14:36:03.306514 -0800 kernel AMFI: bailing out because of restricted entitlements. default 2024-01-10 14:36:03.306523 -0800 kernel mac_vnode_check_signature: /Applications/<app-bundle-id>/Contents/MacOS/<app-name>: code signature validation failed fatally: When validating /Applications/<app-bundle-id>/Contents/MacOS/<app-name>: Code has restricted entitlements, but the validation of its code signature failed. Unsatisfied Entitlements: com.apple.developer.system-extension.installcom.apple.developer.team-identifier The thing is, when I run this command codesign -v -vvv <path-to-app> the app is valid on disk and satisfies its Designated Requirement and these two commands: codesign --display --entitlements - security cms -D -i <path-to-app>/Contents/embedded.provisionprofile when run against the old app (which works) and the new app (which doesn't) have absolutely identical outputs. The certificates haven't expired yet. Where else should we be looking to figure out where we've messed up? We know we changed the signing and notarization flow; the working build was made by a person using Xcode, the new app was built, signed and notarized using the command line tools (xcodebuild and notarytool).
4
0
1.5k
Jan ’24
run command line tool with associated dylib
I've been given an Xcode project which produces a command line tool which links to a dylib. I have the dylib, but not its source code. I change the signing option for the command line tool target so it is signed automatically by my personal team. On an attempt to run the tool, it fails to load the dylib, because the dylib is signed with a different certificate. I manually codesign the dylib with the same certificate I am using for the command line app. Now, I can build the app, but not run it. If I try to do so, I see four dialogs telling me “libXXX.dylib” can’t be opened because Apple cannot check it for malicious software, then the console tells me "'/path/to/libXXX.dylib' not valid for use in process: library load disallowed by system policy)" I found an old document about Gatekeeper (https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html) which suggests that Gatekeeper just won't let me do this - I can't just put the dylib next to the executable, although the dynamic linker finds the dylib, Gatekeeper doesn't like it because the dylib isn't inside the app bundle (there is none), and isn't in one of the well-known places. I dealt with this by making a do-nothing app which I can sign with my personal certificate. Then I replace the signature on the dylib (and its dependent dylibs) with my own. I add the command line tool and all its dylib dependencies to the do-nothing app, then add those files into the Copy Bundle Resources phase of the do-nothing app. Now, the command line tool and its dylibs all live in do-nothing.app/Contents/Resources, and I can run the tool from there without Gatekeeper complaining. Is there an easier way (aside from asking my supplier for static libraries)? And if this is the only way, is Contents/Resources the right place to put command line tools and the dylibs they link to?
5
0
1.9k
May ’24
Xcode fails to provision target
I've alluded to this before in these posts and there are some posts from others about this, e.g. https://developer.apple.com/forums/thread/759845 and I've filed some bugs related to the behavior. FB20212935 FB19451832 FB19450508 FB19450162 FB19449747 Our company owns the USB vendor IDs X and Y . We've been granted a USB transport entitlement for both of those IDs. The crux of the problem is that I want to build a driver for USB vendor ID Y. Xcode's well-hidden auto-generated provisioning profile for my driver contains com.apple.developer.driverkit.transport.usb: { idVendor = X; } which is obviously not what I want. Xcode fails to provision the target. But I have another, much older project with an auto-generated provisioning profile containing com.apple.developer.driverkit.transport.usb: { idVendor = X; }, { idVendor = Y; } I can build a driver for idVendor Y without problems in this project. But that doesn't help me with my new project. What can I do to fix this? Do I need to request our entitlements again? I fear if I do so, something will get lost in the process. Is there a way to inspect what we have already been granted? - I can't see a "managed entitlements" section on the account portal. I can go through the motions of making a new App ID, then I can see that some Capability Request have been "Assigned", but I don't see what their values are. A second question I have is about the userclient-access entitlement. Are these tied to the bundle ID of the app which claims the access? In other words, if I have two drivers com.mycompany.app1.driver1 com.mycompany.app2.driver2 and I would like to have com.mycompany.app1 communicate with com.mycompany.app1.driver1, I would ask for the com.apple.developer.driverkit.userclient-access capability for com.mycompany.app1.driver1. But must I request that access for each specific app bundle ID that will talk to that driver, or once the entitlement is granted, can I use com.apple.developer.driverkit.userclient-access = { com.mycompany.app1.driver1 } in any of my apps?
1
0
183
Sep ’25