I've spent way too long today trying to convert an Object Detection TensorFlow2 model to a CoreML object classifier (with bounding boxes, labels and probability score) The 'SSD MobileNet v2 320x320' is here: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md And I've been following all sorts of posts and ChatGPT https://apple.github.io/coremltools/docs-guides/source/tensorflow-2.html#convert-a-tensorflow-concrete-function https://developer.apple.com/videos/play/wwdc2020/10153/?time=402 To convert it. I keep hitting the same errors though, mostly around: NotImplementedError: Expected model format: [SavedModel | concrete_function | tf.keras.Model | .h5 | GraphDef], got <ConcreteFunction signature_wrapper(input_tensor) at 0x366B87790> I've had varying success including missing output labels/predictions. But I simply want to create the CoreML model with all the right inputs and outputs (including correct names) as detailed in the docs here: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tf2.md It goes without saying I don't have much (any) experience with this stuff including Python so the whole thing's been a bit of a headache. If anyone is able to help that would be great. FWIW I'm not attached to any one specific model, but what I do need at minimum is a CoreML model that can detect objects (has to at least include lights and lamps) within a live video image, detecting where in the image the object is. The simplest script I have looks like this: import coremltools as ct import tensorflow as tf model = tf.saved_model.load("~/tf_models/ssd_mobilenet_v2_320x320_coco17_tpu-8/saved_model") concrete_func = model.signatures[tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY] mlmodel = ct.convert( concrete_func, source="tensorflow", inputs=[ct.TensorType(shape=(1, 320, 320, 3))] ) mlmodel.save("YourModel.mlpackage", save_format="mlpackage")

I'm using a custom create ML model to classify the movement of a user's hand in a game, The classifier has 3 different spell movements, but my code constantly predicts all of them at an equal 1/3 probability regardless of movement which leads me to believe my code isn't correct (as opposed to the model) which in CreateML at least gives me a heavily weighted prediction My code is below. On adding debug prints everywhere all the data looks good to me and matches similar to my test CSV data So I'm thinking my issue must be in the setup of my model code? /// Feeds samples into the model and keeps a sliding window of the last N frames. final class WandGestureStreamer { static let shared = WandGestureStreamer() private let model: SpellActivityClassifier private var samples: [Transform] = [] private let windowSize = 100 // number of frames the model expects /// RNN hidden state passed between inferences private var stateIn: MLMultiArray /// Last transform dropped from the window for continuity private var lastDropped: Transform? private init() { let config = MLModelConfiguration() self.model = try! SpellActivityClassifier(configuration: config) // Initialize stateIn to the model’s required shape let constraint = self.model.model.modelDescription .inputDescriptionsByName["stateIn"]! .multiArrayConstraint! self.stateIn = try! MLMultiArray(shape: constraint.shape, dataType: .double) } /// Call once per frame with the latest wand position (or any feature vector). func appendSample(_ sample: Transform) { samples.append(sample) // drop oldest frame if over capacity, retaining it for delta at window start if samples.count > windowSize { lastDropped = samples.removeFirst() } } func classifyIfReady(threshold: Double = 0.6) -> (label: String, confidence: Double)? { guard samples.count == windowSize else { return nil } do { let input = try makeInput(initialState: stateIn) let output = try model.prediction(input: input) // Save state for continuity stateIn = output.stateOut let best = output.label let conf = output.labelProbability[best] ?? 0 // If you’ve recognized a gesture with high confidence: if conf > threshold { return (best, conf) } else { return nil } } catch { print("Error", error.localizedDescription, error) return nil } } /// Constructs a SpellActivityClassifierInput from recorded wand transforms. func makeInput(initialState: MLMultiArray) throws -> SpellActivityClassifierInput { let count = samples.count as NSNumber let shape = [count] let timeArr = try MLMultiArray(shape: shape, dataType: .double) let dxArr = try MLMultiArray(shape: shape, dataType: .double) let dyArr = try MLMultiArray(shape: shape, dataType: .double) let dzArr = try MLMultiArray(shape: shape, dataType: .double) let rwArr = try MLMultiArray(shape: shape, dataType: .double) let rxArr = try MLMultiArray(shape: shape, dataType: .double) let ryArr = try MLMultiArray(shape: shape, dataType: .double) let rzArr = try MLMultiArray(shape: shape, dataType: .double) for (i, sample) in samples.enumerated() { let previousSample = i > 0 ? samples[i - 1] : lastDropped let model = WandMovementRecording.DataModel(transform: sample, previous: previousSample) // print("model", model) timeArr[i] = NSNumber(value: model.timestamp) dxArr[i] = NSNumber(value: model.dx) dyArr[i] = NSNumber(value: model.dy) dzArr[i] = NSNumber(value: model.dz) let rot = model.rotation rwArr[i] = NSNumber(value: rot.w) rxArr[i] = NSNumber(value: rot.x) ryArr[i] = NSNumber(value: rot.y) rzArr[i] = NSNumber(value: rot.z) } return SpellActivityClassifierInput( dx: dxArr, dy: dyArr, dz: dzArr, rotation_w: rwArr, rotation_x: rxArr, rotation_y: ryArr, rotation_z: rzArr, timestamp: timeArr, stateIn: initialState ) } }

I'm playing about with the hand tracking systems in reality kit / Vision Pro I thought it would be interesting if I could attach a virtual object to a hand when the hand is gripping (thought it would be fun to attach a basic cylinder to mimic a wand from Harry Potter) I'm able to detect when the user is gripping but having trouble placing an object as though it's within the hand. The simplest version of this is using an AnchorEntity pointing to the user's palm which kind of works, but quickly breaks the illusion when you rotate the wrist or hand. It seems as though I will have to roll my own anchor entity using the various points of the user's hand and I thought calculating some median point between the thumb and little finger tips would be a good start but it's proven a little difficult as we need both rotation and position. I'm already out of my depth with reality kit and matrices (and thanks to ChatGPT) I have some code, but as soon as I apply the position manually (as opposed to a hand anchor entity) it fails to render on the user's hand. It feels like this should already have been something someone has looked in to, any ideas on what might be the issue here? Note: HandTrackingSystem.handTracking is a HandTrackingProvider() guard let anchors = HandTrackingSystem.handTracking.latestAnchors.leftHand else { return } if let thumb = anchors.handSkeleton?.joint(.thumbTip), let little = anchors.handSkeleton?.joint(.littleFingerTip) { let thumbPos = simd_make_float3(thumb.anchorFromJointTransform.columns.3) let littlePos = simd_make_float3(little.anchorFromJointTransform.columns.3) let midPos = (thumbPos + littlePos) / 2 let direction = normalize(littlePos - thumbPos) let rotation = simd_quatf(from: [0, 1, 0], to: direction) wandEntity.transform.translation = midPos wandEntity.transform.rotation = rotation content.add(wandEntity) }

Introduced in the Keynote was the 3D Lock Screen images with the kangaroo: https://9to5mac.com/wp-content/uploads/sites/6/2025/06/3d-lock-screen-2.gif I can't see any mention on if this effect is available for developers with an API to convert flat 2D photos in to the same 3D feeling image. Does anyone know if there is an API?

Not sure if this is common with releases but I've been doing some CI work recently so it's the first time I've seen this myself, When I list the runtimes installed on my machine: xcrun simctl list runtimes I notice the iOS 18.3.1 release has the below info: == Runtimes == iOS 18.3 (18.3.1 - 22D8075) - com.apple.CoreSimulator.SimRuntime.iOS-18-3 Meanwhile the other runtimes are listed as: == Runtimes == iOS 17.5 (17.5 - 21F79) - com.apple.CoreSimulator.SimRuntime.iOS-17-5 iOS 18.4 (18.4 - 22E5216h) - com.apple.CoreSimulator.SimRuntime.iOS-18-4 watchOS 11.2 (11.2 - 22S99) - com.apple.CoreSimulator.SimRuntime.watchOS-11-2 visionOS 2.3 (2.3 - 22N895) - com.apple.CoreSimulator.SimRuntime.xrOS-2-3 (Apologies for the weird formatting above, using code blocks and quote markdown condenses things down to one line for some reason) This is causing some funkiness in my CI code which I've managed to workaround, but wondered if this was a common thing, specifically the mismatch between the iOS name and the runtime version. iOS 18.3 and com.apple.CoreSimulator.SimRuntime.iOS-18-3 vs 18.3.1 - 22D8075 where the .1 has been dropped for the runtime names?

On my local machine I'm running Xcode 16.2. I currently have: iOS SDK 18.2 (22C146) Simulator 18.3.1 (22D8075) I'm setting up a CI machine to match my local setup, but I can't seem to get the 18.3.1 simulator on there. I'm running: xcodebuild -downloadPlatform iOS -exportPath ~/Downloads -buildVersion 16.2 The -buildVersion is 16.2 (I'm pretty sure to match the 16.2 of Xcode, as oppose to 18.2 for iOS) but this doesn't fetch the 18.3.1 simulator. Is someone able to explain to me what's going on here? I've not seen this mismatch before between Simulator and iOS but according to another post this seems like a valid setup, but I could be misinterpreting it.

I have a UIKit app with a UIHostingController embedded as a child controller. In this UIHostingController there's a SwiftUI view which expands and collapses with an animation to show/hide content within it. The hosting controller uses .intrinsicContentSize sizing option. This all works fine, and the animation of the expand/collapse looks good so far as the SwiftUI view, in a preview for example. But running the app the hosting controller doesn't animate its view's size alongside the SwiftUI view animating its size. Instead the hosting controller jumps from the correct start/end sizes without any animation. So technically although it has the right size when the SwiftUI view is expanded/collapsed, this not a nice UX as the hosting controller jumps immediately from its small size to its larger one on expanding and vice versa for collapsing while the SwiftUI contents animates nicely. I'm assuming there's somewhere I should be calling (in some form or another) a: UIView.animate(withDuration: 0.4) { hostingController.view.layoutIfNeeded() } alongside the SwiftUI animations - but I can't see any hook between my SwiftUI view .animation(value:) function and somewhere that hosting controller could jump in alongside this animation and animate its view's frame.

I'm currently writing a macro which outputs Swift Testing code: // Macro code ... @MainActor @SnapshotSuite struct MySuite { func makeView() -&gt; some View { Text("a view") } } which expands to... // Expanded macro code ... @MainActor @Suite struct _GeneratedSnapshotSuite { @MainActor @Test(.tags(.snapshots)) func assertSnapshotMakeView() async throws { let generator = SnapshotGenerator( testName: "makeView", traits: [ .theme(.all), .sizes(devices: .iPhoneX, fitting: .widthAndHeight), .record(false), ], configuration: .none, makeValue: { MySuite().makeView() }, fileID: #fileID, filePath: #filePath, line: 108, column: 5 ) await __assertSnapshot(generator: generator) } } In short, this macro creates snapshot tests from a function. This all works but I'm finding a couple of limitations, potentially with how Macros are expanded in Swift. Xcode diamonds are not visible The snapshots tag isn't discovered There are a couple of things worth noting though: The suites and tests are discovered in Xcode's Test Navigator each time Xcode runs tests (cmd + u) - although they need to rerun to be updated. I manually add a @Suite in my code as well as my @SnapshotSuite to all of the suites. (The tags never seem to be available though) Couple of questions on this: Is this a known issue? Are there any workarounds? I do wonder if there's a workaround for showing the diamonds with XCTest APIs such as: https://github.com/swiftlang/swift-corelibs-xctest/tree/main https://developer.apple.com/documentation/xctest

Instead of implementing a textview from scratch (UITextInput it a lot of work/boilerplate) It makes sense for me to subclass UITextView. However, when subclassing it seems this is limited to TextKit 1 only, I get an assertion failure: *** Assertion failure in -[_UITextKit1LayoutController initWithTextView:textContainer:], _UITextKit1LayoutController.m:72 I thought I would just need to call the super init: super.init(usingTextLayoutManager: true) But this isn't a designated initialiser: Must call a designated initializer of the superclass 'UITextView' Is there a way to do this and override the layout manager so that it uses TextKit 2 in the subclass? (My aim is to then draw the fragments manually using TextKit2 to get a custom layout while ultimately using all of the UITextView implementation as 99% of it is what I want - other than custom drawing of text fragments). My code is below: class DocumentTextView: UITextView { private let _textLayoutManager = NSTextLayoutManager() private var textContentStorage: NSTextContentStorage { textLayoutManager!.textContentManager as! NSTextContentStorage } override var textLayoutManager: NSTextLayoutManager? { _textLayoutManager } init() { let textContainer = NSTextContainer(size: .zero) super.init(frame: .zero, textContainer: textContainer) _textLayoutManager.textContainer = textContainer textContentStorage.attributedString = NSAttributedString(string: text, attributes: [ .foregroundColor: UIColor.label, ]) textContentStorage.addTextLayoutManager(_textLayoutManager) } required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } }

I'm trying to unpack this sentence Note that after adding a second text container to the layout manager, the text views become uneditable and unselectable. Found on this example code: https://developer.apple.com/documentation/uikit/textkit/display_text_with_a_custom_layout I'm implementing my own UITextInput which makes use of the example code setup to replicate a multi-page text editor (kind of like a basic version of Pages/Word) Each page has its own NSTextContainer which gets added to one NSLayoutManager. The code works and my views are rendering text across each page, however, only the first text view is editable. It appears this is "by design" (?) How do we get around this? Is there another level we have to drop down to enable multiple editing containers? The sample code is wayback in iOS 13, so it would seem very odd if the capability of multiple editable text fields/containers was impossible to achieve.

Hellooo, I’m looking to implement an OpenAI assistant using APIs but I want to do this locally on a group of files. I want to be able to train a GPT on the contents of a folder for example. Does anyone have any experience in this? It seems OpenAI needs a lot of uploading on each request if I were to do this with their API after playing around (but this feels like I’m missing something). It’s also quite costly to use. I was hoping to use local machine learning and models but this is quite limited in what it can do (eg Lumachain)

Originally asked on Swift Forums: https://forums.swift.org/t/using-bindable-with-a-observable-type/70993 I'm using SwiftUI environments in my app to hold a preferences object which is an @Observable object But I want to be able to inject different instances of the preferences object for previews vs the production code so I've abstracted my production object in to a Preferences protocol and updated my Environment key's type to: protocol Preferences { } @Observable final class MyPreferencesObject: Preferences { } @Observable final class MyPreviewsObject: Preferences { } // Environment key private struct PreferencesKey: EnvironmentKey { static let defaultValue : Preferences & Observable = MyPreferencesObject() } extension EnvironmentValues { var preferences: Preferences & Observable { get { self[PreferencesKey.self] } set { self[PreferencesKey.self] = newValue } } } The compiler is happy with this until I go to use @Bindable in my code where the compiler explodes with a generic error, eg: @Environment(\.preferences) private var preferences // ... code @Bindable var preferences = preferences If I change the environment object back to a conforming type eg: @Observable final class MyPreferencesObject() { } private struct PreferencesKey: EnvironmentKey { static let defaultValue : MyPreferencesObject = MyPreferencesObject() } extension EnvironmentValues { var preferences: MyPreferencesObject { get { self[PreferencesKey.self] } set { self[PreferencesKey.self] = newValue } } } Then @Bindable is happy again and things compile. Is this a known issue/limitation? Or am I missing something here?

When running a SwiftUI Multiplatform app in Xcode Cloud I'm getting the following error: CIWatcher encountered an error in CIWatcherTests failed with: Failed to install or launch the test runner. (Underlying Error: Could not launch “CIWatcherTests”. Runningboard has returned error 5. Please check the system logs for the underlying cause of the error. (Underlying Error: The operation couldn’t be completed. Launch failed. (Underlying Error: Launchd job spawn failed))) This runs fine on iOS and all those tests pass. The SwiftUI app on Mac is a menu bar app so I'm not sure if this could be the issue as it's not launching for some reason on the Xcode Cloud system? Maybe I need to add some run option to make this work? When run locally both iOS and macOS pass all the tests successfully

Sometimes we just want to use modifiers but don't have a view to draw to the screen, for example an .overlay {} modifier on a view needs a view inside it before you can add modifiers on to that overlay view. The current approach for this type of thing seems to be Color.clear and while I've seen this almost everywhere I'm not sure if this is a) the officially recommended way to do this in SwiftUI and b) the only way to do this in SwiftUI? I have the following custom view in my project just to avoid the explicit use of Color.clear import SwiftUI struct ClearView: View { var body: some View { Color.clear } } Are there any other ways to do this? Note - this is of course not the same as EmptyView() - where Color.clear renders a view with a transparent colour and EmptyView which doesn't render anything at all.

When: Embedding a Image with a symbol effect in a stack view in SwiftUI, and putting this in a List Actual: The symbols animate when scrolling that item in and out of the view Expected: The symbol doesn't animate until the value changes. I have the following code which is boiled down from my actual code where the animating value is a @Model object property (other animations work fine when their animation's value is watching the same property.) I've boiled it down to the below examples: The top example works as expected (no symbol animations on scroll) The bottom example animates the symbols every time they scroll in to view #Preview { VStack { List { ForEach(1...100, id: \.self) { index in Image(systemName: "square.3.layers.3d") .symbolEffect(.bounce, value: index) } } List { ForEach(1...100, id: \.self) { index in HStack { Image(systemName: "square.3.layers.3d") .symbolEffect(.bounce, value: index) } } } } } This happens on Mac and iOS. I'm running Xcode 15.3 RC2 but it also happens in Xcode 15.2 I've found this will happen whether embedding in a HStack, VStack or ZStack but seems fine in a Group. I've tried adding a id() modifier to the HStack to give it an explicit identifier but this doesn't seem to work. Is there something I'm missing here?