Reality Composer Pro 3 Assistant appears to have useful project-aware tool categories for working with scenes, assets, materials, animation, behavior trees, navigation, skeletons, audio, planning, documentation, and ShaderGraph material creation.
It would be very helpful if a supported subset of these capabilities could be exposed to external developer tools through an Apple-provided MCP server, command-line tool, or similar local automation interface.
Similar to what's possible with Xcode: https://developer.apple.com/documentation/Xcode/giving-external-agents-access-to-xcode
Possible exposed capabilities could include:
- Querying and modifying scene hierarchy, entities, and components.
- Inspecting and managing project assets and structure.
- Working with materials and ShaderGraph/SSL material creation.
- Reading and editing animation libraries, timelines, skeletons, and rigs.
- Inspecting behavior trees and navigation data.
- Reading audio components and playback settings.
- Querying Reality Composer Pro documentation through the Assistant.
- Returning structured JSON responses for external tools.
- Letting external coding agents request project-aware plans or safe editor actions.
Expected Result: Reality Composer Pro provides a documented, permission-aware MCP server, CLI, or similar interface that allows external developer tools to safely use supported Assistant/project capabilities.
Actual Result: Reality Composer Pro Assistant is available only inside the app UI. External coding agents such as Codex or Claude Code cannot reliably inspect or automate Reality Composer Pro projects except through unsupported package/file manipulation.
Why This Matters: Many developers now use coding agents alongside Xcode and Reality Composer Pro. A supported interface would make app development smoother by allowing those tools to coordinate Swift code, RealityKit assets, scene state, materials, animations, and Assistant workflows through an official integration point.
Feedback ticket: FB23276072