The view space would be limited, no 360 degress, because there would be no tracking of head movements, but it would still be very good experience with that 4K video playing optically far in front of me like in movie theater, yet I could still occasioanlly peak at other apps for updates.

Not necessarily, the device could just display content in fixed position in the center in front of the viewer, regardless of where the viewer turns his head. Then no tracking of the user nor the vehicle is needed. First app started, e.g. the video playback would be opened in front of the user's eyes, second app, e.g. email also in the front and user would just move it to one side, then weather app could on the other side of playing video, messages above the video, playback controls below video.

Simple solution would be if user could disable accelerometer tracking. When watching movies or do passive tasks, like checking emails, you don't need tracking of head movements and change of physical location of the person wearing the headset. Even image sensor data might not be needed for certain applicatins, like those 2 I mentioned. But for augmented reality, processing the surroundings might still be a good idea. Tourists on a bus might want to see info dispayed next to the object they see.