Hello,
I am not a developer, and I am programming a macOS application in Xcode with the help of Claude (Anthropic). Claude writes the code for me, but I need your help regarding the technical specifications.
I am not trying to understand every technical detail, nor am I looking for someone to write the code for me. My goal is to obtain the most accurate technical information possible so that I can pass it directly to Claude, who will then generate the corresponding code in Xcode.
This application is a personal project. It is not intended to be published on the App Store. Its sole purpose is to automatically process my own videos recorded with two DJI Action 6 cameras. It requires no user settings : the optimal parameters are defined by default within the application, and it directly outputs the final video file.
My objective is to create an application that takes as input the two videos from the DJI Action 6 cameras mounted on a rig, and then produces a file that delivers a 180° immersive video with the highest possible quality.
Claude has already conducted research in Apple's documentation as well as various online resources to determine the correct output format, encoding parameters, and necessary constraints. Despite this, the current result is not satisfactory.
Currently, my application does not directly generate a file in the Apple Immersive format. Instead, it produces a stereoscopic video file containing both the left and right views side-by-side. I then use Moonlink on my Mac Studio and on my Apple Vision Pro, to play this video (the video is on the Mac and sent to the vision pro, wich works perfectly fine on some videos I have found on internet).
The file is correctly recognized and can be opened, but the result presents two major issues:
-
Playback is extremely choppy (about one second of playback followed by several seconds of freezing);
-
The image is pinched at the top and bottom, which likely indicates an issue with the format, projection, or how the player interprets the file.
I have no preference regarding the final format. If the best solution is to directly generate a file in the Apple Immersive format compatible with the Apple Vision Pro, that works for me.
If, on the other hand, the best approach is to produce a high-quality stereoscopic video with the left and right views side-by-side, which can then be interpreted by a player application on the Apple Vision Pro or potentially other headsets, that works for me as well.
My sole objective is to obtain:
-
The best possible image quality (even though I am aware it will not match the quality produced by a Blackmagic URSA Cine Immersive camera);
-
Perfectly smooth playback, without any stuttering or freezing;
-
A correctly displayed 180° immersive video.
Here are the specifications of my source videos:
-
Two DJI Action 6 cameras.
-
Interpupillary/Inter-axial distance (between optical centers): 6.5 cm.
-
FOV Boost lens mounted on each camera (182° coverage).
-
Video mode: 4K Custom 3840 \times 3840 at 60 fps.
-
Color profile: D-Log M.
-
Color conversion planned in the application via a DJI D-Log M to Rec.709 LUT in .cube format (.cube provided to the app).
-
In FOV Boost mode, DJI enforces Electronic Image Stabilization (EIS) and removes gyroscope data from the MP4 files. Stereoscopic alignment is therefore performed via image analysis within the application (using ORB feature detection). With no FOV Boost I do not reach 180° with is the target.
-
Fixed aperture.
-
Both cameras are triggered simultaneously using the DJI GPS remote control.
Fine synchronization is subsequently performed in the application via audio cross-correlation.
My question is therefore the following : What exactly are the output specifications that my application should produce (format, codec, resolution, projection, metadata, encoding parameters, and any other technical constraints) in order to achieve the best possible result for a 180° immersive video on the Apple Vision Pro?
I will pass all this information directly to Claude so that he can generate the corresponding code in Xcode. Therefore, I am not looking for a code example, but rather the technical specifications to follow.
I would also welcome any advice or recommendations that a specialist in Vision Pro, immersive video, or Apple technologies might deem useful to improve the quality of the result or simplify my approach.
Thank you in advance for your help.
Best regards,
Georges
attached is a screen capture of the pinched image in the Vision Pro, this happens both at top and bottom, only displayed bottom.