Post

Replies

Boosts

Views

Activity

Controlling simdgroup structure in planar compute dispatches
In the "Discover advances Metal for A15 Bionic" Tech Talk right around the 20:00 mark, the presenter (Katelyn Hinson) says: The output image is split into a set of SIMD groups, where each SIMD group is a 4-by-8 chunk, [with] each thread writing to a single output. Supposing that we know the simdgroup will contain 32 threads (which they mention in the talk is true for Apple Silicon), is the only way to ensure that the threads in each simdgroup will be arranged into a 4 x 8 chunk to perform a dispatch with threadgroups that have a width dividing the number of threads per simdgroup? I can't think of another way to control the shape of a simdgroup directly within threadgroups since there is no explicit API to do so. For example, if we perform a dispatchThreadgroups(_:threadsPerThreadgroup:) with a threadgroup size of 8 x 8 to attempt to recreate the visuals in the presentation, wouldn't the resulting simdgroup shape be an 8 x 4 region and not a 4 x 8 region? The assumptions made in the video about where to sample the source texture and which shuffle functions to use are heavily influenced by the shape of the simdgroup. I'm trying to implement a similar reduction but I'm currently figuring out how to shape each simdgroup. If we don't know whether the simdgroup is 32 threads (I believe it's possible simdgroups have 64 threads?). What would be a reliable way to control the structure of the simdgroups? I believe if we always ensure that the width of the threadgroup divides the number of threads in the simdgroup we should get the behavior that we want, but I'm looking to confirm this logic. IIRC, simdgroups will always have a multiple of 8 threads (maybe it was only 4?), so perhaps a width of 8 (or 4) would always suffice for the threadgroup and you could specify a height of computePipelineState.maxTotalThreadsPerThreadgroup / 4 for example. Finally, must we only use uniform threadgroups (viz. we couldn't use dispatchThreads(_:threadsPerThreadgroup:)) for reliable results? I'm thinking that non-uniform threadgroups would again violate our assumptions about the simdgroup shape
0
1
708
Jun ’22
Difference between `thread_execution_width` and `threads_per_simdgroup`
I have a compute kernel that makes use of simdgroup operations such as simd_shuffle_up, simd_or, etc, and I'm looking to rewrite the kernel to support older hardware. One such computation requires that I know the index of the thread in the simdgroup (thread_index_in_simdgroup). I was hoping to derive it from the thread's position in its threadgroup (thread_position_in_threadgroup) and the thread execution width (thread_execution_width), along with other knowledge about the size of the threadgroup when I noticed there was also the threads_per_simdgroup attribute. The spec describes both respectively as thread_execution_width: The execution width of the compute unit. threads_per_simdgroup: The thread execution width of a SIMD-group. Under what conditions, if any, could these two values differ? If they do differ, is there a way to determine a thread's position in the simdgroup on hardware that doesn't support Metal 2.2?
1
1
923
Jun ’22
UITextInput autocorrection: where do we receive text replaced by the system?
I have an implementation of UITextInput that is used to implement note taking with text. We have a custom UIMenuItem that lists suggested text replacements for a misspelled word that a user can interact with to fix the word. This works well on iPhone and iPad where the only path for changing text is via this menu. On Mac Catalyst, however, the system also presents text replacement options with the best replacement; and when users attempt to replace text with the menu options provided by the system, our UITextInput handler seems to only receive a call to setSelectedTextRange: (the code is in Objective-C). I would expect a call to, for example, replaceRange:WithText: after an autocorrection is made Any ideas what could possibly be incorrectly implementing? I.e., how can we receive the text that the system attempts to replace?
0
0
662
Apr ’22
Unexpected behavior for shared MTLBuffer during CPU work
I have an image processing pipeline that performs some work on the CPU after the GPU processes a texture and then writes its result into a shared buffer (i.e. storageMode = .shared) used by the CPU for its computation. After the CPU does its work, it similarly writes at a different offset into the same shared MTLBuffer object. The buffer is arranged as so: uint | uint | .... | uint | float offsets (contiguous): 0 | ... where the floating point slot is written into by the CPU and later used by the GPU in subsequent compute passes. I haven't been able to explain or find documentation on the following strange behavior. The compute pipeline with the above buffer (call it buffer A) is as follows (without the force unwraps): let device = MTLCreateSystemDefaultDevice()! let commandQueue = device.makeCommandQueue()! let commandBuffer = commandQueue.makeCommandBuffer()! let sharedEvent = device.makeSharedEvent()! let sharedEventQueue = DispatchQueue(label: "my-queue") let sharedEventListener = MTLSharedEventListener(dispatchQueue: sharedEventQueue) // Compute pipeline kernelA.encode(commandBuffer: commandBuffer, sourceTexture: sourceTexture, destinationBuffer: bufferA)         commandBuffer.encodeCPUExecution(for: sharedEventObject, listener: sharedEventListener) { [self] in var value = Float(0.0) bufferA.unsafelyWrite(&value, offset: Self.targetBufferOffset) } kernelB.setTargetBuffer(histogramBuffer, offset: Self.targetBufferOffset) kernelB.encode(commandBuffer: commandBuffer, sourceTexture: sourceTexture, destinationTexture: destinationTexture) Note that commandBuffer.encodeCPUExecution simply is a convenience function around the shared event object (encodeSignalEvent and encodeWaitEvent) that signals and waits on event.signaledValue + 1 and event.signaledValue + 2 respectively. In the example above, kernel B does not see the writes made during the CPU execution. It can however see the values written into the buffer from kernelA. The strange part: if you write to that same location in the buffer before the GPU schedules this work (e.g. during the encoding instead of in the middle of the GPU execution or whenever before), kernelB does see the value of the writes by the CPU. This is odd behavior that to me suggests there is undefined behavior. If the buffer were .managed I could understand the behavior since changes on each side must be made explicit; but with a .shared buffer this behavior seems quite unexpected, especially considering that the CPU can read the values made by the preceding kernel (viz. kernelA) What explains this strange behavior with Metal? Note: This behavior occurs on an M1 Mac running MacCatalyst and an iPad Pro (5th generation) running iOS 15.3
4
0
1.6k
Mar ’22
vImage vs CoreImage vs MetalPerformaceShaders strengths and weaknesses
While the above three frameworks (viz. vImage, CoreImage, and MetalPerformaceShaders) serve different overall purposes, what are the strengths and weaknesses of the each of the three frameworks in terms of performance with respect to image processing? It seems that any of the three frameworks is highly performant; but where does each framework shine?
1
3
1.4k
Mar ’22
DocC Documentation for targets other than static and dynamic libraries
Is it possible to add DocC documentation to a target that does not result in either a static library or a framework? It doesn't yet appear to be a feature of DocC. If not, will there be support in the future to add documentation to "regular" Xcode projects that don't result in a static library or framework? I think it could be useful to have documentation for larger apps that may use multiple frameworks in complex ways
2
0
1.7k
Feb ’22
Metal Quadgroups Example Usage
I've started working with simdgroups and as I was looking through the MSL documentation I noticed that there exists, in addition to simdgroups, quadgroups. The shading language documentation merely states that A quad-group function is a SIMD-group function (see section 6.9.2) with an execution width of 4. However, it doesn't appear there's a clear reason for using quadgroups over simdgroups, and I have yet to find demonstrations of using quadgroups within a compute kernel. What are quadgroups and how are they used in conjunction with/replacement of simdgroups?
1
2
992
Jan ’22
Will TSAN or the Swift compiler identify possible Swift async-await race conditions?
I haven't followed the swift forums very closely, so perhaps there is news buried deep somewhere mentioning this. Will the swift compiler and/or TSAN at runtime in the future be able to identify possible race conditions associated with Swift async-await (excluding data races that are "erased" by async-await)? I suppose this could equate to proving a function is reentrant in some scenarios (from a compiler's perspective, though I'm not knowledgeable about compilers)? Consider, e.g. the scenario described in "Protect Mutable State with Swift Actors" around 9:15, where Dario talks about actor reentrancy, with the cache for the image URL
0
0
771
Jan ’22
MTLSharedEvent scheduled block called before command buffer scheduling and not in-flight
I am using a MTLSharedEvent to occasionally relay new information from the CPU to the GPU by writing into a MTLBuffer with storage mode .storageModeManaged within a block registered by the shared event (using the notify(_:atValue:block:) method of MTLSharedEvent, with a MTLSharedEventListener configured to be notified on a background dispatch queue). The process looks something like this: let device = MTLCreateSystemDefaultDevice()! 	let synchronizationQueue = DispatchQueue(label: "com.myproject.synchronization") 		 		let sharedEvent = device.makeSharedEvent()! 		let sharedEventListener = MTLSharedEventListener(dispatchQueue: synchronizationQueue) 		 		// Updated only occasionally on the CPU (on user interaction). Mostly written to 		// on the GPU 		let managedBuffer = device.makeBuffer(length: 10, options: .storageModeManaged)! 		 		var doExtra = true func computeSomething(commandBuffer: MTLCommandBuffer) { 	 	 // Do work on the GPU every frame 	 // After writing to the buffer on the GPU, synchronize the buffer (required) 	 let blitToSynchronize = commandBuffer.makeBlitCommandEncoder()! 				blitToSynchronize.synchronize(resource: managedBuffer) 				blitToSynchronize.endEncoding() 				 	 // Occassionally, add extra information on the GPU 	 if doExtraWork { 					 			 // Register a block to write into the buffer 			sharedEvent.notify(sharedEventListener, atValue: 1) { event, value in 								 						 // Safely write into the buffer. Make sure we call `didModifyRange(_:)` after 								 						// Update the counter 						event.signaledValue = 2 			} 		 commandBuffer.encodeSignalEvent(sharedEvent, value: 1) 		 commandBuffer.encodeWaitForEvent(sharedEvent, value: 2) 	 } 				 				// Commit the work 			 commandBuffer.commit() } The expected behavior is as follows: The GPU does some work with the managed buffer Occasionally, the information needs to be updated with new information on the CPU. In this frame, we register a block of work to be executed. We do so in a dedicated block because we cannot guarantee that by the time execution on the main thread reaches this point the GPU is not simultaneously reading from or writing to the managed buffer. Hence, it is unsafe to simply write to it currently and must make sure the GPU is not doing anything with this data When the GPU schedules this command buffer to be executed, commands executed before the encodeSignalEvent(_:value:) call are executed and then execution on the GPU stops until the block increments the signaledValue property of the event passed into the block When execution reaches the block, we can safely write into the managed buffer because we know the CPU has exclusive access to the resource. Once we've done so, we resume execution of the GPU The issue is that it seems Metal is not calling the block when the GPU is executing the command, but rather *before* the command buffer is even scheduled. Worse, the system seems to "work" with the initial command buffer (the very first command buffer, before any other are scheduled). I first noticed this issue when I looked at a GPU frame capture after my scene would vanish after a CPU update, which is where I saw that the GPU had NaNs all over the place. I then ran into this strange situation when I purposely waited on the background dispatch queue with a sleep(:_) call. Quite correctly, my shared resource semaphore (not shown, signaled in a completion block of the command buffer and waited on in the main thread) reached a value of -1 after committing three command buffers to the command queue (three being the number of recycled shared MTLBuffers holding scene uniform data etc.). This suggests that the first command buffer has not finished executing by then time the CPU is more than three frames ahead, which is consistent with the sleep(_:) behavior. Again, what isn't consistent is the ordering: Metal seems to call the block before even scheduling the buffer. Further, in subsequent frames, it doesn't seem that Metal cares that the sharedEventListener block is taking so long and schedules the command buffer for execution even while the block is running, which finishes dozens of frames later. This behavior is completely inconsistent with what I expect. What is going on here? P.S. There is probably a better way to periodically update a managed buffer whose contents are mostly modified on the GPU, but I have not yet found a way to do so. Any advice on this subject is appreciated as well. Of course, a triple buffer system *could* work, but it would waste a lot of memory as the managed buffer is quite large (whereas the shared buffers managed by the semaphore are quite small)
2
0
1.3k
Jan ’22
Controlling simdgroup structure in planar compute dispatches
In the "Discover advances Metal for A15 Bionic" Tech Talk right around the 20:00 mark, the presenter (Katelyn Hinson) says: The output image is split into a set of SIMD groups, where each SIMD group is a 4-by-8 chunk, [with] each thread writing to a single output. Supposing that we know the simdgroup will contain 32 threads (which they mention in the talk is true for Apple Silicon), is the only way to ensure that the threads in each simdgroup will be arranged into a 4 x 8 chunk to perform a dispatch with threadgroups that have a width dividing the number of threads per simdgroup? I can't think of another way to control the shape of a simdgroup directly within threadgroups since there is no explicit API to do so. For example, if we perform a dispatchThreadgroups(_:threadsPerThreadgroup:) with a threadgroup size of 8 x 8 to attempt to recreate the visuals in the presentation, wouldn't the resulting simdgroup shape be an 8 x 4 region and not a 4 x 8 region? The assumptions made in the video about where to sample the source texture and which shuffle functions to use are heavily influenced by the shape of the simdgroup. I'm trying to implement a similar reduction but I'm currently figuring out how to shape each simdgroup. If we don't know whether the simdgroup is 32 threads (I believe it's possible simdgroups have 64 threads?). What would be a reliable way to control the structure of the simdgroups? I believe if we always ensure that the width of the threadgroup divides the number of threads in the simdgroup we should get the behavior that we want, but I'm looking to confirm this logic. IIRC, simdgroups will always have a multiple of 8 threads (maybe it was only 4?), so perhaps a width of 8 (or 4) would always suffice for the threadgroup and you could specify a height of computePipelineState.maxTotalThreadsPerThreadgroup / 4 for example. Finally, must we only use uniform threadgroups (viz. we couldn't use dispatchThreads(_:threadsPerThreadgroup:)) for reliable results? I'm thinking that non-uniform threadgroups would again violate our assumptions about the simdgroup shape
Replies
0
Boosts
1
Views
708
Activity
Jun ’22
Difference between `thread_execution_width` and `threads_per_simdgroup`
I have a compute kernel that makes use of simdgroup operations such as simd_shuffle_up, simd_or, etc, and I'm looking to rewrite the kernel to support older hardware. One such computation requires that I know the index of the thread in the simdgroup (thread_index_in_simdgroup). I was hoping to derive it from the thread's position in its threadgroup (thread_position_in_threadgroup) and the thread execution width (thread_execution_width), along with other knowledge about the size of the threadgroup when I noticed there was also the threads_per_simdgroup attribute. The spec describes both respectively as thread_execution_width: The execution width of the compute unit. threads_per_simdgroup: The thread execution width of a SIMD-group. Under what conditions, if any, could these two values differ? If they do differ, is there a way to determine a thread's position in the simdgroup on hardware that doesn't support Metal 2.2?
Replies
1
Boosts
1
Views
923
Activity
Jun ’22
UITextInput autocorrection: where do we receive text replaced by the system?
I have an implementation of UITextInput that is used to implement note taking with text. We have a custom UIMenuItem that lists suggested text replacements for a misspelled word that a user can interact with to fix the word. This works well on iPhone and iPad where the only path for changing text is via this menu. On Mac Catalyst, however, the system also presents text replacement options with the best replacement; and when users attempt to replace text with the menu options provided by the system, our UITextInput handler seems to only receive a call to setSelectedTextRange: (the code is in Objective-C). I would expect a call to, for example, replaceRange:WithText: after an autocorrection is made Any ideas what could possibly be incorrectly implementing? I.e., how can we receive the text that the system attempts to replace?
Replies
0
Boosts
0
Views
662
Activity
Apr ’22
Unexpected behavior for shared MTLBuffer during CPU work
I have an image processing pipeline that performs some work on the CPU after the GPU processes a texture and then writes its result into a shared buffer (i.e. storageMode = .shared) used by the CPU for its computation. After the CPU does its work, it similarly writes at a different offset into the same shared MTLBuffer object. The buffer is arranged as so: uint | uint | .... | uint | float offsets (contiguous): 0 | ... where the floating point slot is written into by the CPU and later used by the GPU in subsequent compute passes. I haven't been able to explain or find documentation on the following strange behavior. The compute pipeline with the above buffer (call it buffer A) is as follows (without the force unwraps): let device = MTLCreateSystemDefaultDevice()! let commandQueue = device.makeCommandQueue()! let commandBuffer = commandQueue.makeCommandBuffer()! let sharedEvent = device.makeSharedEvent()! let sharedEventQueue = DispatchQueue(label: "my-queue") let sharedEventListener = MTLSharedEventListener(dispatchQueue: sharedEventQueue) // Compute pipeline kernelA.encode(commandBuffer: commandBuffer, sourceTexture: sourceTexture, destinationBuffer: bufferA)         commandBuffer.encodeCPUExecution(for: sharedEventObject, listener: sharedEventListener) { [self] in var value = Float(0.0) bufferA.unsafelyWrite(&value, offset: Self.targetBufferOffset) } kernelB.setTargetBuffer(histogramBuffer, offset: Self.targetBufferOffset) kernelB.encode(commandBuffer: commandBuffer, sourceTexture: sourceTexture, destinationTexture: destinationTexture) Note that commandBuffer.encodeCPUExecution simply is a convenience function around the shared event object (encodeSignalEvent and encodeWaitEvent) that signals and waits on event.signaledValue + 1 and event.signaledValue + 2 respectively. In the example above, kernel B does not see the writes made during the CPU execution. It can however see the values written into the buffer from kernelA. The strange part: if you write to that same location in the buffer before the GPU schedules this work (e.g. during the encoding instead of in the middle of the GPU execution or whenever before), kernelB does see the value of the writes by the CPU. This is odd behavior that to me suggests there is undefined behavior. If the buffer were .managed I could understand the behavior since changes on each side must be made explicit; but with a .shared buffer this behavior seems quite unexpected, especially considering that the CPU can read the values made by the preceding kernel (viz. kernelA) What explains this strange behavior with Metal? Note: This behavior occurs on an M1 Mac running MacCatalyst and an iPad Pro (5th generation) running iOS 15.3
Replies
4
Boosts
0
Views
1.6k
Activity
Mar ’22
vImage vs CoreImage vs MetalPerformaceShaders strengths and weaknesses
While the above three frameworks (viz. vImage, CoreImage, and MetalPerformaceShaders) serve different overall purposes, what are the strengths and weaknesses of the each of the three frameworks in terms of performance with respect to image processing? It seems that any of the three frameworks is highly performant; but where does each framework shine?
Replies
1
Boosts
3
Views
1.4k
Activity
Mar ’22
DocC Documentation for targets other than static and dynamic libraries
Is it possible to add DocC documentation to a target that does not result in either a static library or a framework? It doesn't yet appear to be a feature of DocC. If not, will there be support in the future to add documentation to "regular" Xcode projects that don't result in a static library or framework? I think it could be useful to have documentation for larger apps that may use multiple frameworks in complex ways
Replies
2
Boosts
0
Views
1.7k
Activity
Feb ’22
Metal Quadgroups Example Usage
I've started working with simdgroups and as I was looking through the MSL documentation I noticed that there exists, in addition to simdgroups, quadgroups. The shading language documentation merely states that A quad-group function is a SIMD-group function (see section 6.9.2) with an execution width of 4. However, it doesn't appear there's a clear reason for using quadgroups over simdgroups, and I have yet to find demonstrations of using quadgroups within a compute kernel. What are quadgroups and how are they used in conjunction with/replacement of simdgroups?
Replies
1
Boosts
2
Views
992
Activity
Jan ’22
Will TSAN or the Swift compiler identify possible Swift async-await race conditions?
I haven't followed the swift forums very closely, so perhaps there is news buried deep somewhere mentioning this. Will the swift compiler and/or TSAN at runtime in the future be able to identify possible race conditions associated with Swift async-await (excluding data races that are "erased" by async-await)? I suppose this could equate to proving a function is reentrant in some scenarios (from a compiler's perspective, though I'm not knowledgeable about compilers)? Consider, e.g. the scenario described in "Protect Mutable State with Swift Actors" around 9:15, where Dario talks about actor reentrancy, with the cache for the image URL
Replies
0
Boosts
0
Views
771
Activity
Jan ’22
MTLSharedEvent scheduled block called before command buffer scheduling and not in-flight
I am using a MTLSharedEvent to occasionally relay new information from the CPU to the GPU by writing into a MTLBuffer with storage mode .storageModeManaged within a block registered by the shared event (using the notify(_:atValue:block:) method of MTLSharedEvent, with a MTLSharedEventListener configured to be notified on a background dispatch queue). The process looks something like this: let device = MTLCreateSystemDefaultDevice()! 	let synchronizationQueue = DispatchQueue(label: "com.myproject.synchronization") 		 		let sharedEvent = device.makeSharedEvent()! 		let sharedEventListener = MTLSharedEventListener(dispatchQueue: synchronizationQueue) 		 		// Updated only occasionally on the CPU (on user interaction). Mostly written to 		// on the GPU 		let managedBuffer = device.makeBuffer(length: 10, options: .storageModeManaged)! 		 		var doExtra = true func computeSomething(commandBuffer: MTLCommandBuffer) { 	 	 // Do work on the GPU every frame 	 // After writing to the buffer on the GPU, synchronize the buffer (required) 	 let blitToSynchronize = commandBuffer.makeBlitCommandEncoder()! 				blitToSynchronize.synchronize(resource: managedBuffer) 				blitToSynchronize.endEncoding() 				 	 // Occassionally, add extra information on the GPU 	 if doExtraWork { 					 			 // Register a block to write into the buffer 			sharedEvent.notify(sharedEventListener, atValue: 1) { event, value in 								 						 // Safely write into the buffer. Make sure we call `didModifyRange(_:)` after 								 						// Update the counter 						event.signaledValue = 2 			} 		 commandBuffer.encodeSignalEvent(sharedEvent, value: 1) 		 commandBuffer.encodeWaitForEvent(sharedEvent, value: 2) 	 } 				 				// Commit the work 			 commandBuffer.commit() } The expected behavior is as follows: The GPU does some work with the managed buffer Occasionally, the information needs to be updated with new information on the CPU. In this frame, we register a block of work to be executed. We do so in a dedicated block because we cannot guarantee that by the time execution on the main thread reaches this point the GPU is not simultaneously reading from or writing to the managed buffer. Hence, it is unsafe to simply write to it currently and must make sure the GPU is not doing anything with this data When the GPU schedules this command buffer to be executed, commands executed before the encodeSignalEvent(_:value:) call are executed and then execution on the GPU stops until the block increments the signaledValue property of the event passed into the block When execution reaches the block, we can safely write into the managed buffer because we know the CPU has exclusive access to the resource. Once we've done so, we resume execution of the GPU The issue is that it seems Metal is not calling the block when the GPU is executing the command, but rather *before* the command buffer is even scheduled. Worse, the system seems to "work" with the initial command buffer (the very first command buffer, before any other are scheduled). I first noticed this issue when I looked at a GPU frame capture after my scene would vanish after a CPU update, which is where I saw that the GPU had NaNs all over the place. I then ran into this strange situation when I purposely waited on the background dispatch queue with a sleep(:_) call. Quite correctly, my shared resource semaphore (not shown, signaled in a completion block of the command buffer and waited on in the main thread) reached a value of -1 after committing three command buffers to the command queue (three being the number of recycled shared MTLBuffers holding scene uniform data etc.). This suggests that the first command buffer has not finished executing by then time the CPU is more than three frames ahead, which is consistent with the sleep(_:) behavior. Again, what isn't consistent is the ordering: Metal seems to call the block before even scheduling the buffer. Further, in subsequent frames, it doesn't seem that Metal cares that the sharedEventListener block is taking so long and schedules the command buffer for execution even while the block is running, which finishes dozens of frames later. This behavior is completely inconsistent with what I expect. What is going on here? P.S. There is probably a better way to periodically update a managed buffer whose contents are mostly modified on the GPU, but I have not yet found a way to do so. Any advice on this subject is appreciated as well. Of course, a triple buffer system *could* work, but it would waste a lot of memory as the managed buffer is quite large (whereas the shared buffers managed by the semaphore are quite small)
Replies
2
Boosts
0
Views
1.3k
Activity
Jan ’22