Okay I was able to figure it out. Basically I have to make use of AVAssetCache.

@Polyphonic first thanks for you answer. What I am trying to achieve is implement a video quality picker in our app where the user can pick specific quality of the stream available from the manifest (something like youtube has) so for example even if the user doesn't have a great network connection but he wants/needs to see the video in high quality and don't care about stalling make this possible for him and prevent the AVPlayer switching the video back to lower quality and keep the one that our user selects. Other use case may be when you are watching video using mobile data. Some users have unlimited data plans and don't care about data consumption but others may have limited data plans and want to switch to lower qualities.