CKSyncEngine initial sync on a new device

Question

Created 5d

Replies 2

Boosts 0

Participants 2

I have implemented CKSyncEngine synchronization, and it works well. I can update data on one device and see the changes propagate to another device quickly. However, the initial sync when a user downloads the app on a new device is a significant issue for both me and my users.

One problem is that the sync engine fetches deletion events from the server. On a new device, the local database is empty, so these deletions are essentially no-ops. This would not be a big problem if there were only a few records or if it was fast. I measured the initial sync and found that there are 150 modified records and 62,168 deletions. Counting these alone takes over five minutes, even without processing them. The deletions do nothing because the local database has nothing to delete, yet they still add a significant delay.

I understand that the sync engine ensures consistency across all devices, but five minutes of waiting with the app open just to insert a small number of records is excessive. The problem would be worse if there were tens of thousands of new records to insert, since downloading and saving the data would take even longer.

This leads to a poor user experience. Users open the app and see data being populated for several minutes, or they are stuck on a screen that says the data is being synchronized with iCloud.

I am wondering if there is a way to make the sync engine ignore deletion events when the state serialization is nil. Alternatively, is there a recommended method for handling initial synchronization more efficiently?

One idea I considered is storing all the data as a backup in iCloud Documents, along with the state serialization at that point in time. When a user opens the app for the first time, I could download the file, extract the data, and set the state serialization to the saved value. I am not sure if this would work. I do not know if state serialization is tied to the device or if it only represents the point where the sync engine left off. My guess is that it might reference some local device storage.

I am not sure what else to try. I could fetch all data using CloudKit, create the sync engine with an empty state serialization, and let it fetch everything again, but that would still take a long time.

My records are very small, mostly a date when something happened and an ID referencing the parent. Since the app tracks watched episodes, I only store the date the user watched the episode and the ID of that episode.

Boost

Answer 1

DTS Engineer OP

Apple

2d

It's probably that, when doing an initial synchronization, CKSyncEngine can't tell whether you have built a local cache or not, and so to ensure that you can maintain the synchronization between your local cache, if any, and the server, CKSyncEngine returns you the deleted record IDs if:

The deleted records were created on the device that is doing the fetch. A device that is not the "creator" won't get the deleted record IDs.
The records was concurrently deleted during the fetching of changes.

This is similar to the behavior I mentioned in this post.

With this background, I am curious how you observed the slowness, if you don't mind to share the detailed steps.

Also, is it that CKSyncEngine takes long time to fetch and deliver the deletions (events) to your app, or is it that your app needs long time to handle the deletions? In the former case, I'd suggest that you file a feedback report because it doesn't seem that anything from the app side can fix the issue – If you do so, please share your report ID here.

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

0

Answer 2

Tabonx OP

1d

@DTS Engineer

Hi,

Getting deleted records on the device that made the edits makes sense. It makes less sense to get them even after I fully deleted the app, but this would not be a problem if receiving the delete events and any other events were reasonably fast.

As I mentioned in the post, I have removed all of the processing logic from my app and only logged the number of items per batch using Logger. I found that on average a batch contains about 200 items. In very few batches, it can reach 600 items, but then it stabilizes at around 200 per batch, with roughly one second between batches. This is true for batches containing modifications or deletions. With that, simple math matches my observations in the post: 62,000 items need to be synced, which is about 310 batches of 200 items each. 310 divided by 60 is approximately 5.17 minutes.

If there is no way to increase the batch size or reduce the time between events, I believe there is nothing I can do to reduce the time it takes to sync everything on the device. Please do not think only in terms of deletions, as this is also true for modified items. Even with 62,000 items in CloudKit and zero deleted records, it would take roughly the same time to fetch all changes.

0