resolved. Apple's crawler was blocked when trying to access the apple-site-association-file