Hello!
My name is Mason Prather. I'm a graduate student at Kennesaw State University and a Research Engineer working in XR environments through my Graduate Research Assistant role. I’m currently building a research prototype that connects a mobile companion application to a VR headset.
The mobile application is built in Unity and deployed on iOS, and it streams video frames to a remote Unity client using WebRTC.
Environment
Device: iPhone 15
OS: iOS 26.3 (tested on physical device, not Simulator)
Engine: Unity 2022.3.57f1
Graphics API: Metal
Streaming Technology: WebRTC (Unity WebRTC package)
Architecture: Mobile Unity app streaming video frames to a remote Unity client
Receiver Device: Meta Quest Pro headset (Unity application)
Networking: LAN (UDP discovery + TCP signaling)
Video Source: Unity RenderTexture
Goal
The goal of the system is to allow a VR user to view media stored on their phone inside a VR environment.
The iOS app:
renders or captures media content
converts frames into a WebRTC video track
streams the video to the headset
Current Status
Connection setup works correctly.
Observed behavior:
Signaling connection successful
ICE candidate exchange successful
PeerConnection state becomes Connected
Video track created successfully
However, the receiving application displays black frames.
iOS App Details
The video source originates from a Unity RenderTexture.
Inside the phone application:
RenderTexture displays correctly
Frames appear correct locally
But the receiving peer does not display the frames.
Relevant Components
Unity WebRTC package
iOS Metal rendering pipeline
Custom TCP signaling
LAN discovery via UDP
Expected Behavior
Rendered frames should transmit via WebRTC and appear on the remote device.
Actual Behavior
The remote video track is active, but the rendered frames appear black on the receiving client.
Questions
Are there known issues involving Unity WebRTC + iOS Metal texture capture?
Are there specific pixel format requirements when streaming textures from Unity on iOS?
Could the issue relate to texture readback limitations or GPU synchronization?
I am more than happy to provide screenshots and console logs upon request.
If anyone has experience streaming Unity video frames via WebRTC on iOS, I would greatly appreciate any guidance.
Topic:
Media Technologies
SubTopic:
Streaming