Missing entitlement for API HomeKit error

During a test run of my HomeKit application in Xcode, after some time the app did run smoothly, I've started for one characteristic to get a pretty weird error in the readValueWithCompletionHandler: callback block:

Let me start with the messages here:

"Handler does not support background access"
...
"Missing entitlement for API."

That error and the entitlement it references are a leftover artifact from the Apple Watch's initial release, not a "real" entitlement we actually grant/use. Very early in watchOS 2.0’s development, an entitlement WAS created and integrated into the framework so that watchOS extensions (which run "in the background") running on the iPhone could control HomeKit. However, very early in the pre-release development testing process, it became clear that there were so many other "foreground only" edge cases that it was better to simply "lie" to the extension (so our frameworks considered it "foreground") instead of trying to sort out all of those edge cases.

That made the HomeKit entitlement unnecessary, but there wasn't any reason to remove the existing code, so it was left in place, referencing an entitlement that we've never granted to any developer and probably never will.

That leads back to the error itself. Is our app in the background when it happens? If so, then the answer is "don't do that". HomeKit background access isn't supported [1].

Note please that immediately after launch (and for some time after that) the value was read in through the same code without any problem.

SO, there are three cases here:

1. If you're in the foreground, the API works fine.

2. Immediately after an app enters the background, the system continues to treat it as "foreground" for "awhile.” That might seem strange, but the problem here is that an API like Metal isn't something an app can just "stop" using. Preventing an app from freeing up Metal resources because it's "in the background" is counterproductive, so the system continues treating the app as "foreground" despite the fact it’s in the background.

3. If you're running in the debugger, the system’s behavior can get "weird.” The debugger disables normal app suspensions as well as other things like the main thread watchdog, but that can create unexpected side effects in other places. For example, many daemon's rely on process assertion expiration to trigger behaviors you're normally associated with "entering the background,” but expiration only occurs if/when your app is actually eligible to suspend, so they don't fire. Further complicating things, most of those cases aren't even bugs, as things like the "audio" background category rely on them.

Strangely enough, when I've switched the device through the Home application,

The background restriction is tied to your ability to send commands "into" HomeKit, NOT HomeKit's ability to interact with your app. Things like notifications that you've set up before entering the background may continue to work, particularly if you're dealing with #3.

[1] Why that is the case is a question I'm happy to answer if you're curious, but it's almost dinner time and I should finish up.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks a lot, interesting!

I do apologise for stupidity: I've completely forgot there is the foreground-only limitation. Sigh.

As for [1] (I hope you enjoyed the dinner :), and), correct me please if I am wrong, but I believe it is so for privacy concerns, so that no 3rd party app can analyse what I do with my accessories for evil goals.

Nevertheless this is an application I wrote myself to be run by myself for my own completely benign (and even if not, it's my problem) goals in my own home (still the same one of https://developer.apple.com/forums/thread/849339), thus of course I don't mind it analyses my behaviours (it is its very purpose in fact). Compare please also closely related comment https://developer.apple.com/forums/thread/849288?answerId=907772022#907772022. I've added a request for the background support to my feedback ticket FB25066661.

If you can see any way to overcome these limits (both the background and the ones caused by the profile, detailed in the thread linked above) in this particular case (my own private app run in my own home only), I'd be grateful for any advice.

As for [1] (I hope you enjoyed the dinner :)

I did actually!

and), correct me please if I am wrong, but I believe it is so for privacy concerns, so that no 3rd party app can analyse what I do with my accessories for evil goals.

Actually, no, not at all. The actual reason is to try to minimize/reduce the likelihood that you'll end up with a house where crazy things happen that you can't understand, explain, or debug.

The issue here is that, from the beginning, HomeKit's been built around the idea that your home's configuration isn't something that’s tied to any particular device or "setup". You don't have a home server that you manage and configure; you have home configuration that’s visible and editable on all of your devices, and things "just work". In its original design, the idea was that the home database was simply shared to all of your devices, any one of which could individually connect to an accessory and control it.

Every device had access to the full home state and could independently manage "everything". This had all sorts of effects on HomeKit's design. For example, notice that basic automation only allows you to change an accessory to a specific state and can only be scheduled down to ~1 min. That combination means that if multiple controllers happen to execute exactly the same automation, nothing "bad" actually happens— whichever command arrives "second" is ignored by the accessory (since it's already changed state) and the relatively large timing window means that the devices don't really need to be synced to the same time.

In practice, that did have issues (particularly as the number of controllers in the home increased), which eventually led to the introduction of "Home Hubs", which simplified things by letting us route all commands through one controller. That centralization also opened up other opportunities, most notably embedding shortcuts in automations. Multiple controllers running the same shortcut at the same time could cause huge problems, but the home hub architecture prevents that from happening by routing all commands through the same controller.

In any case, the key idea this is all built on is that ALL of your home’s configuration (including automations) is captured in a single data store, which is then shared among devices. Any device can modify the data store. One device (the Home Hub) may be responsible for executing the data store, but that role isn't particularly "special" or complicated. If the Home Hub goes offline, then we can pick another without anything really being visible to the user.

Now, switching to the user perspective, any time an accessory does something, there are only two reasons it can happen:

1. A user on a device directly "did something" in an app they were actually using.

2. It happened "on its own" as part of an automation they configured.

Similarly, if something "weird" happens, the investigation is straightforward— "ask your family" (#1) or open Home.app on any device you have at hand and start looking through your automations.

...and the words "any device" are why this:

I've added a request for the background support to my feedback ticket FB25066661.

...isn't going to happen. More specifically, allowing background HomeKit access means that investigating "mysterious" behavior could require systematically looking at EVERY device you own as you try to determine which of those devices originated the unexpected command. Even worse, once multiple apps are involved, the ultimate problem could be a side effect of the interactions between different apps on different devices owned by different users. Frankly, this worst case here is bad enough that I think it would make HomeKit effectively unusable.

Advanced users don't think about this because their perspective is that they want to set up some kind of "home management server" that does "interesting things," and they're frustrated by the extra friction the foreground requirement creates. The problem is that the "home management server" concept ONLY works when there is only one of them, and the friction the foreground requirement creates IS what's preventing more than one of them.

Shifting topics:

Nevertheless, this is an application I wrote myself to be run by myself for my own completely benign (and even if not, it's my problem) goals in my own home.

In terms of the code signing issues, the path forward here is to either pay for a developer membership or use the "nuclear option.” One other comment about the nuclear option, based on one of Quinn's notes:

IMPORTANT Do not disable SIP on a machine you care about. I do all of my EndpointSecurity testing in a VM, so I don’t have to disable SIP on my main work machine.

Using a VM as the "container" you run THIS particular app in isn't a terrible idea. You can restrict its network access to reduce exposure, and you can keep all of "your" data off that system to limit risk. If I were doing this myself, I think what I'd do is create a new iCloud user that was JUST for this machine, then share my home to that user to give it access. There are things you could do to harden it further, but the best protection is simply not caring about the contents of the machine.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks again for all the detailed and very interesting insights!

Nevertheless I am afraid I do not quite get it. Seems to me (probably due to my misunderstanding) that none of these concerns should apply to my application, which never ever tries to set or change anything in HK.

All it does is reading the characteristic values; and even that just since waiting for the update notifications is unreliable, as detailed in https://developer.apple.com/forums/thread/849598.

Based on this and my (probably mistaken) understanding of what you wrote, there should be absolutely no reason not to allow my app to work seamlessly in the background. What do I miss here?

P.S. Just for the record, in future I shall need to change a couple of things, namely

• the setting of the thermostat which drives the main heating;
• switching a couple of power outlets which control some auxiliary heating units on/off.

At this moment though, it is a far future and not my pressing concern; that is the observing and analysing the current state, effectively, far as HK is concerned, read-only.