Semantic Voice Search and Natural Language Parameter Resolution

How does next-gen Siri handle semantic or semantic-adjacent parameter resolution when using EntityQuery? For example, if a user says 'Siri, show memory tags related to auth errors' and the exact tag is auth-handler-failure:

Does the App Intents framework support passing the raw query string (from natural language) directly to our search/recall logic within the intent, or does it try to map it strictly to pre-registered AppEntity lists? How do we configure an AppEntity to support synonyms or regex-like string match resolution directly within Siri's offline/online parsing?

Answered by Frameworks Engineer in 891950022

If you are implementing a schematized app intent that takes in an entity, Siri will look at the available options (from Spotlight indexing) and pick one through semantic matching, or ask the user to pick one if it’s not confident. In some cases, Siri may also run a StringQuery if you have one on the entity and send you the text it wants to search for based on the user’s request. You then you have an opportunity to implement the search and return the entity.

If you want to get open-ended text that starts a search in your app, implement the system.search schema and you will get open-ended text passed into the criteria in that intent.

If you want to get open-ended text from Siri for for a different kind of action (one outside of schemas, or one that needs to run in the background for example), use an AppShortcut with an un-filled open-ended string parameter, and Siri will use a second turn to ask for that value and send you the whole string.

If you are implementing a schematized app intent that takes in an entity, Siri will look at the available options (from Spotlight indexing) and pick one through semantic matching, or ask the user to pick one if it’s not confident. In some cases, Siri may also run a StringQuery if you have one on the entity and send you the text it wants to search for based on the user’s request. You then you have an opportunity to implement the search and return the entity.

If you want to get open-ended text that starts a search in your app, implement the system.search schema and you will get open-ended text passed into the criteria in that intent.

If you want to get open-ended text from Siri for for a different kind of action (one outside of schemas, or one that needs to run in the background for example), use an AppShortcut with an un-filled open-ended string parameter, and Siri will use a second turn to ask for that value and send you the whole string.

Semantic Voice Search and Natural Language Parameter Resolution
 
 
Q