Dear Apple engineers and DriverKit developers,
We have developed a DriverKit (DEXT) driver for an HBA RAID controller.
The RAID controller is connected to hosts through Thunderbolt (PCIe port of the Thunderbolt controller).
We use an IO script to verify the developed driver. The test fails after a few (10-12) hours of running with an error:
“BUG IN CLIENT OF LIBDISPATCH: dispatch_sync called on queue already owned by current thread”.
We inspected the stack trace of the crash report. This error happens in the interrupt handler.
Thread 5 Crashed:
0 libdispatch.dylib 0x19671aa8c __DISPATCH_WAIT_FOR_QUEUE__ + 484
1 libdispatch.dylib 0x19671a5d0 _dispatch_sync_f_slow + 152
2 DriverKit 0x195d3fc1c IODispatchQueue::DispatchSync_f(void*, void (*)(void*)) + 296
3 DriverKit 0x195d40860 IOInterruptDispatchSourceThread(void*) + 380
4 libsystem_pthread.dylib 0x1968a3738 _pthread_start + 140
5 libsystem_pthread.dylib 0x1968ac6c8 thread_start + 8
On our side we created 5 DispatchQueue(s) for the interrupt processing and configured 5 interrupt handlers using ConfigureInterrupts(kIOInterruptTypePCIMessagedX, 1, 5). It gives 5 interrupts, as requested and it is not clear what is the reason for the issue.
Our code samples are below
uint32_t configureInterrupts(uint32_t requested, uint32_t required)
{
const kern_return_t ret = ioPCIdevice->ConfigureInterrupts(kIOInterruptTypePCIMessagedX, required, requested);
if (ret != kIOReturnSuccess)
return 0;
uint64_t interruptType = 0;
uint32_t interruptsCount = 0;
uint32_t interruptIndex = 0;
for ( ;; ++interruptIndex) {
if (IOInterruptDispatchSource::GetInterruptType(ioPCIdevice, interruptIndex, &interruptType) != kIOReturnSuccess)
break;
if ((interruptType & kIOInterruptTypePCIMessagedX) == 0)
continue;
++interruptsCount;
}
return interruptsCount;
}
.....
// Create DQs
for(int i = 0; i < maxInterrupts; ++i) {
ret = IODispatchQueue::Create(INTERRUPT_DQ_NAME, 0, 0, &ivars->interruptQueue[i]);
if (kIOReturnSuccess != ret || nullptr == ivars->interruptQueue[i]) {
GH_PRINT_ERR("Interrupts queue %d creation failed with error %d", i, ret);
return false;
}
}
.....
// Link DQ with interrupt
for(int index = 0; index < maxInterrupts; ++ index) {
kern_return_t ret = CreateActionHandleInterruptRequest(size, &ivars->interruptActions[index]);
if (kIOReturnSuccess != ret) {
GH_PRINT_ERR("Create action for interrupt handler %u failed.", index);
return ;
}
ret = IOInterruptDispatchSource::Create(ivars->PCI_io.dev, index, ivars->interruptQueue[index], &ivars->interruptSources[index]);
if (kIOReturnSuccess != ret || nullptr == ivars->interruptSources[index]) {
GH_PRINT_ERR("Creating interrupt source %u failed for interrupt index %u.", index, index);
return ;
}
ret = ivars->interruptSources[index]->SetHandler(ivars->interruptActions[index]);
if (kIOReturnSuccess != ret) {
GH_PRINT_ERR("Setting the handler for interrupt source %u failed.", index);
return ;
}
ivars->contexts[index] = ivars->interruptActions[index]->GetReference();
}
.....
// definition for interrupt handler
void HandleInterruptRequest ( OSAction *action, uint64_t count, uint64_t time) TYPE (IOInterruptDispatchSource::InterruptOccurred);
Do you have any clue how we can fix this error? Or directions and ways for investigation?
Please let us know if you need more details.
