It turns out, they use a deep CNN for action classification and a classifier is trained on top of the body key points. 

Update and Answer: The activeFormat is just the currently active format. Printing .formats can provide you all possible formats. If you want to set your frames per second to 60, do so by: swift // Instantiate the video device: wide angle camera, back position         let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera,                                                   for: .video,                                                   position: .back)                           // Set the frame rate to 60, as expected by the machine learning model         try! videoDevice?.lockForConfiguration()                  videoDevice?.activeFormat = (videoDevice?.formats[30])!         videoDevice?.activeVideoMinFrameDuration = CMTimeMake(value: 1, timescale: 60)         videoDevice?.activeVideoMaxFrameDuration = CMTimeMake(value: 1, timescale: 60)                  videoDevice?.unlockForConfiguration()                  // print(videoDevice?.activeFormat) Thanks to Rhythmic Fistman for helping me on this question. https://stackoverflow.com/users/22147/rhythmic-fistman

For anyone interested in this: I came to the conclusion that the best way to setting these properties ist trial and error. Just record the action and adjust the properties from "both sides" (min and max). I am not done doing this, but i am tracking a golf ball and setting the max size to 0.2 is still "too big". Hope this helps everyone else who doesn't know the unit of these properties.

I have already found the answer. The method is called periodically and basically "automatically" whenever a new video frame is received via the delegate.