Minimal working example for TCTouchController + SpriteKit SKRenderer + Metal?

Hi,

I’m trying to use the new TouchController framework together with a custom Metal rendering pipeline using SpriteKit’s SKRenderer.

Right now there seems to be almost no real documentation or end-to-end example showing how the framework is intended to work. Or is it just my own inexperience that I cant get it to work?

My setup is:

  • MTKView
  • custom Renderer
  • SKRenderer
  • SKScene
  • rendering SpriteKit manually through: skRenderer.render(withViewport:commandBuffer:renderPassDescriptor:)

I managed to get:

  • Metal rendering working
  • SpriteKit rendering through SKRenderer
  • TCTouchController connecting successfully

But I still struggle to get the actual controls to render reliably on screen.

Is there any minimal working example showing:

  1. MTKView
  2. SKRenderer
  3. TCTouchController
  4. rendering controls with render(using:)

all together in one pipeline?

Even a very small sample project would help a lot.

Thanks!

Answered by DTS Engineer in 887954022

This isn't inexperience on your part — you've already done the hard work of getting each piece running individually. There isn't a public Apple sample covering your exact stack (MTKView + custom Metal renderer + SKRenderer + TCTouchController). The Touch Controller framework shipped at WWDC 2025, its public video coverage is limited to a brief mention in session 209 ("Level up your games"), and the framework documentation doesn't list any sample code project.

That said, the pattern you're describing — individual pieces working, but the controls not drawing reliably when composed with a custom Metal + SpriteKit pipeline — is typically one of two things: a pipeline-state mismatch between TCTouchControllerDescriptor and the MTLRenderCommandEncoder you pass to render(using:), or an unset drawableSize on the touch controller. Both are worth ruling out first.

Looking at the framework source, TCTouchController builds its internal Metal render pipeline state once at initialization, from the descriptor's colorPixelFormat, depthAttachmentPixelFormat, stencilAttachmentPixelFormat, and sampleCount fields. It then binds that pre-built pipeline to whatever encoder you pass to render(using:), without checking whether the encoder's render pass attachments match. A mismatch either raises a Metal validation error or fails silently, depending on whether Metal API Validation is enabled.

There's a convenience initializer that reads the pipeline-state properties directly from the MTKView:

let descriptor = TCTouchControllerDescriptor(mtkView: yourMTKView)
let touchController = TCTouchController(descriptor: descriptor)
touchController.drawableSize = yourMTKView.drawableSize

If you're currently using TCTouchControllerDescriptor() and setting properties manually, try switching to init(mtkView:) and see if the behavior changes. Note the explicit drawableSize assignment after creation — the convenience initializer doesn't set it, and you'll also need to keep touchController.drawableSize updated in your mtkView(_:drawableSizeWillChange:) implementation. If drawableSize is zero, the framework's internal orthographic projection becomes degenerate and controls render to invalid coordinates — which would produce exactly the kind of unreliable rendering you're seeing.

When interleaving with SKRenderer, also check:

  • Create a new render pass for render(using:) after the SKRenderer pass, with loadAction = .load on the color attachment, to preserve the SpriteKit output.
  • Make sure that render pass's attachments match the pixel formats in the descriptor.
  • Call touchController.connect() once before your first render(using:) call.

If the composition looks right and controls still aren't drawing reliably, four silent-failure scenarios are worth eliminating before assuming a deeper rendering bug:

  • Enable Metal API Validation in your Xcode scheme (Edit Scheme → Run → Diagnostics → Metal API Validation). With it on, any mismatch between the pipeline state's attachment formats and your encoder's render pass produces a specific console message naming the conflict. Without it, a mismatch fails silently.
  • Confirm touchController.drawableSize is non-zero and matches your MTKView's current drawable size. If it's zero (the default after init(mtkView:) or plain init()), the framework's internal projection math fails silently.
  • Confirm touchController.isConnected is true at the point of each render(using:) call. The method returns early without drawing if connect() hasn't taken effect yet.
  • Confirm touchController.controls.count > 0. If your control creation code isn't populating the internal list, the render method also no-ops silently.

A few questions to help narrow this down further:

  1. What does "not rendering reliably" look like — invisible, flickering, wrong position or size, drawn behind Metal content, or something else?
  2. How is the descriptor configured — using init(mtkView:) or manually?
  3. Where is render(using:) called in your draw loop relative to your Metal draws and skRenderer.render(...)?
  4. What are your MTKView.colorPixelFormat and MTKView.sampleCount values?

If none of the above resolves it, there are two ways to take this further:

  • Code-Level Support — I can look at your project directly.
  • A Feedback report with a minimal reproducing Xcode project attached — this reaches the framework team directly, which — for a framework with limited sample code — is often the fastest way to get concrete engineering input.

Either one benefits from having a small standalone sample ready, so preparing one is worth doing regardless.

Accepted Answer

This isn't inexperience on your part — you've already done the hard work of getting each piece running individually. There isn't a public Apple sample covering your exact stack (MTKView + custom Metal renderer + SKRenderer + TCTouchController). The Touch Controller framework shipped at WWDC 2025, its public video coverage is limited to a brief mention in session 209 ("Level up your games"), and the framework documentation doesn't list any sample code project.

That said, the pattern you're describing — individual pieces working, but the controls not drawing reliably when composed with a custom Metal + SpriteKit pipeline — is typically one of two things: a pipeline-state mismatch between TCTouchControllerDescriptor and the MTLRenderCommandEncoder you pass to render(using:), or an unset drawableSize on the touch controller. Both are worth ruling out first.

Looking at the framework source, TCTouchController builds its internal Metal render pipeline state once at initialization, from the descriptor's colorPixelFormat, depthAttachmentPixelFormat, stencilAttachmentPixelFormat, and sampleCount fields. It then binds that pre-built pipeline to whatever encoder you pass to render(using:), without checking whether the encoder's render pass attachments match. A mismatch either raises a Metal validation error or fails silently, depending on whether Metal API Validation is enabled.

There's a convenience initializer that reads the pipeline-state properties directly from the MTKView:

let descriptor = TCTouchControllerDescriptor(mtkView: yourMTKView)
let touchController = TCTouchController(descriptor: descriptor)
touchController.drawableSize = yourMTKView.drawableSize

If you're currently using TCTouchControllerDescriptor() and setting properties manually, try switching to init(mtkView:) and see if the behavior changes. Note the explicit drawableSize assignment after creation — the convenience initializer doesn't set it, and you'll also need to keep touchController.drawableSize updated in your mtkView(_:drawableSizeWillChange:) implementation. If drawableSize is zero, the framework's internal orthographic projection becomes degenerate and controls render to invalid coordinates — which would produce exactly the kind of unreliable rendering you're seeing.

When interleaving with SKRenderer, also check:

  • Create a new render pass for render(using:) after the SKRenderer pass, with loadAction = .load on the color attachment, to preserve the SpriteKit output.
  • Make sure that render pass's attachments match the pixel formats in the descriptor.
  • Call touchController.connect() once before your first render(using:) call.

If the composition looks right and controls still aren't drawing reliably, four silent-failure scenarios are worth eliminating before assuming a deeper rendering bug:

  • Enable Metal API Validation in your Xcode scheme (Edit Scheme → Run → Diagnostics → Metal API Validation). With it on, any mismatch between the pipeline state's attachment formats and your encoder's render pass produces a specific console message naming the conflict. Without it, a mismatch fails silently.
  • Confirm touchController.drawableSize is non-zero and matches your MTKView's current drawable size. If it's zero (the default after init(mtkView:) or plain init()), the framework's internal projection math fails silently.
  • Confirm touchController.isConnected is true at the point of each render(using:) call. The method returns early without drawing if connect() hasn't taken effect yet.
  • Confirm touchController.controls.count > 0. If your control creation code isn't populating the internal list, the render method also no-ops silently.

A few questions to help narrow this down further:

  1. What does "not rendering reliably" look like — invisible, flickering, wrong position or size, drawn behind Metal content, or something else?
  2. How is the descriptor configured — using init(mtkView:) or manually?
  3. Where is render(using:) called in your draw loop relative to your Metal draws and skRenderer.render(...)?
  4. What are your MTKView.colorPixelFormat and MTKView.sampleCount values?

If none of the above resolves it, there are two ways to take this further:

  • Code-Level Support — I can look at your project directly.
  • A Feedback report with a minimal reproducing Xcode project attached — this reaches the framework team directly, which — for a framework with limited sample code — is often the fastest way to get concrete engineering input.

Either one benefits from having a small standalone sample ready, so preparing one is worth doing regardless.

Thank you very much! This helped me in achieving to get the TouchControllers to render :)

To get this to work, the following steps helped me:

  1. Use init(mtkView:)
  2. Set touchController.drawableSize = yourMTKView.drawableSize
  3. In mtkView(_:drawableSizeWillChange:) do this:
    func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
      touchController.drawableSize = size
    }
    
  4. In func draw(in view: MTKView) create a second render pass descriptor using the previous descriptors colorAttachments, depthAttachment and stencilAttachment.

Kind Regards Patrick Müller

Minimal working example for TCTouchController + SpriteKit SKRenderer + Metal?
 
 
Q