Also, for the currently playing AVPlayer, I set player.preferredForwardBufferDuration = 0. This tells the system to use the default logic for how far ahead to load while playing. For the other players, I set player.preferredForwardBufferDuration = 1. This tells the players to just load one HLS chunk (generally 6-10 seconds).

I initialize an AVPlayer for the user's current scroll position, the position above the user's current position, and the 3 positions below the user's current scroll position. Initializing the AVPlayer automatically loads the first HLS chunk. I just keep references to those 5 AVPlayers. So if the user scrolls down one position then scrolls back, I don't need to re-initialize the AVPlayer. But if the user scrolls far away and then scrolls back, I do need to re-load the video.