I’m working with apple dispatch queue in C with the following design: multiple dispatch queues enqueue tasks into a shared context, and a dedicated dispatch queue (let’s call it dispatch queue A) processes these tasks. However, it seems this design has a memory visibility issue.
Here’s a simplified version of my setup:
I have a shared_context struct that holds:
task_lis: a list that stores tasks to be prioritized and run — this list is only modified/processed by dispatch queue A (a serially dispatch queue), so I don't lock around it.
cross_thread_tasks: a list that other queues push tasks into, protected by a lock.
Other dispatch queues call a function schedule_task that
locks and appends a new task to cross_thread_tasks
call dispatch_after_f() to schedule a process_task() on dispatch queue A
process_task() that processes the task_list and is repeatedly scheduled on dispatch queue A :
Swaps cross_thread_tasks into a local list (with locking).
Pushes the tasks into task_list.
Runs tasks from task_list.
Reschedules itself via dispatch_after_f().
Problem:
Sometimes the tasks pushed from other threads don’t seem to show up in task_list when process_task() runs. The task_list appears to be missing them, as if the cross-thread tasks aren’t visible. However, if the process_task() is dispatched from the same thread the tasks originate, everything works fine.
It seems to be a memory visibility or synchronization issue. Since I only lock around cross_thread_tasks, could it be that changes to task_list (even though modified on dispatch queue A only) are not being properly synchronized or visible across threads?
My questions
What’s the best practice to ensure shared context is consistently visible across threads when using dispatch queues? Is it mandatory to lock around all tasks? I would love to minimize/avoid lock if possible.
Any guidance, debugging tips, or architectural suggestions would be appreciated!
===============================
And here is pseudocode of my setup if it helps:
struct shared_data {
struct linked_list* task_list;
}
struct shared_context {
struct shared_data *data;
struct linked_list* cross_thread_tasks;
struct thread_mutex* lock; // lock is used to protect cross_thread_tasks
}
static void s_process_task(void* shared_context){
struct linked_list* local_tasks;
// lock and swap the cross_thread_tasks into a local linked list
lock(shared_context->lock)
swap(shared_context->cross_thread_tasks, local_tasks)
unlock(shared_context->lock)
// I didnt use lock to protect `shared_context->data` as they are only touched within dispatch queue A in this function.
for (task : local_tasks) {
linked_list_push(shared_context->data->task_list)
}
// If the `process_task()` block is dispatched from `schedule_task()` where the task is created, the `shared_context` will be able to access the task properly otherwise not.
for (task : shared_context->data->task_list) {
run_task_if_timestamp_is_now(task)
}
timestamp = get_next_timestamp(shared_context->data->task_list)
dispatch_after_f(timestamp, dispatch_queueA, shared_context, process_task);
}
// On dispatch queue B
static void schedule_task(struct task* task, void* shared_context) {
lock(shared_context->lock)
push(shared_context->cross_thread_tasks, task)
unlock(shared_context->lock)
timestamp = get_timestamp(task)
// we only dispatch the task if the timestamp < 1 second. We did this to avoid the dispatch queue schedule the task too far ahead and prevent the shutdown process. Therefore, not all task will be dispatched from the thread it created.
if(timestamp < 1 second)
dispatch_after_f(timestamp, dispatch_queueA, shared_context, process_task);
}
