I debugged a bit deeper by subclassing the UICollectionViewCompositionalLayout.
The main difference I saw between a working and a failing version is the callback for func invalidateLayout(with context: UICollectionViewLayoutInvalidationContext), for which the working version's parameter contains actual values for invalidatedItemIndexPaths in its context, meanwhile the failing version does not.
Again, same code and same customization. In both, the printed modified NSCollectionLayoutVisibleItem reads the same, only difference between the layout property printed.