Natural Language Understanding

Shared content

The FANTASIA data structures are designed to match the Azure component for Natural Language Understanding (NLU(, which is the only one that is currently available for this task, but will be used to represent data coming from other solutions in the future.

Data types

NLUIntentStruct

This structure holds information about a single intent recognized by the NLU service. Its fields are:

• Intent: a string representing the recognized intent;
• Confidence: A float representing the confidence of the intent recognition model towards the recognized intent.

NLUEntityStruct

This structure holds information about a single entity recognized by the NLU service. Its fields are:

• Entity: a string representing the entity as it appears in the sentence;
• Type: a string representing the entity type;
• StartIndex: an integer indicating the character number from which the annotation starts;
• EndIndex: an integer indicating the character number from which the annotation ends;
• Score: a float indicating the confidence of the entity recognition model.

NLUResponseStruct

This structure holds an NLU interpretation in terms of recognized intents and entities. Its fields are:

• Query: the original sentence;
• Entities: a list of NLUEntities;
• Intents: a list of NLUIntents;

Azure

This component allows to call the Azure Conversational Language Understanding (CLU) service, which is already integrated with the ASR service. This way, starting the NLU service also starts the ASR service using the main microphone, reducing latency. Check out the video for an example.

Configuration

The configuration section of the component has the following fields:

• Speech key: the secret key to access the Azure ASR service;
• Region: the availability region of the service;
• Language: the language code of the service (e.g. en-US, it-IT, ect.);
• Language key: the secret key to access the CLU service;
• Endpoint: the endpoint of the CLU service;
• Project name: the name of the CLU project;
• Deployment name: the name of the deployment under the CLU project.

Functions

NLUStart

This function will start streaming audio from the main microphone to the NLU service for transription and intent/entity recognition.

Events

IncomingMessage

This event fires when the NLU interpretation of the sentence is ready.

Data:

NLU Response: an NLUResponse struct containing the interpretation of the detected utterance.