Version 2.9.1 alexa-smart-screen-sdk

*** Bug Fixes ***
* Fixed hold-to-talk event handling logic in the native layer to keep it in sync with GUI

*** Enhancements ***
* Upgraded dependency to APL Client Library 1.8.3
* Upgraded nanoid version to 3.2.0, minimist version to 1.2.6 and refreshed the package-lock.json with new dependencies. Note that users upgrading to this version will need to remove the existing node modules from modules/GUI/js/node_modules before the build.

Author: Vishnu Karpuram

CHANGELOG.md

@@ -1,5 +1,12 @@
-### v2.9.0 released 12/16/2021:
+### v2.9.1 released 04/05/2022:
+#### Bug fixes
+* Fixed hold-to-talk event handling logic in the native layer to keep it in sync with GUI
+
+#### Enhancements
+* Upgraded dependency to APL Client Library 1.8.3
+* Upgraded nanoid version to 3.2.0, minimist version to 1.2.6 and refreshed the package-lock.json with new dependencies. Note that users upgrading to this version will need to remove the existing node modules from modules/GUI/js/node_modules before the build.
 
+### v2.9.0 released 12/16/2021:
 #### Breaking Changes
 * Decoupled APL Client Library and APL Core Engine build from Smart Screen SDK. Please follow new build instructions here [https://developer.amazon.com/en-US/docs/alexa/alexa-smart-screen-sdk/overview.html]
 
@@ -82,7 +89,6 @@
 * Fixed a regression introduced in 2.4 wherein keyboard events no longer interrupted TTS and APL activity
 
 ### v2.4.0 released 11/09/2020:
- 
 #### Enhancements
 * Upgraded dependency to Device SDK 1.21.0 [https://github.com/alexa/avs-device-sdk/tree/v1.21.0]
 * Added DND Support
@@ -92,7 +98,6 @@
 * Fixed an issue where the sample app automatically goes into the listening mode when prompted by a skill
 
 ### v2.3.0 released 09/24/2020:
- 
 #### Enhancements
 * Upgraded dependency to Device SDK 1.20.1 [https://github.com/alexa/avs-device-sdk/tree/v1.20.1]
 * Support for APL 1.4 [https://developer.amazon.com/en-US/docs/alexa/alexa-presentation-language/apl-changes-1-4.html]
@@ -107,7 +112,6 @@
 * Fixed import content failure cache update
 
 ### v2.2.0 released 07/13/2020:
-
 #### Enhancements
 * Upgraded dependency to Device SDK 1.20.0
 * Optimization for APL package import latency
@@ -131,7 +135,6 @@
 * The AVS Device SDK must be built with an additional flag: `-DRAPIDJSON_MEM_OPTIMIZATION=OFF`
 
 ### v2.1.0 released 06/02/2020:
-
 #### Enhancements
 * Upgraded dependency to Device SDK 1.19.1
 * Provided a reference implementation for rendering of captions
@@ -156,7 +159,6 @@
 * Progress bar on Now Playing display card does not reflect the actual audio offset when such information is not provided in RenderPlayerInfo directive 
 
 ### v2.0.2 released 04/14/2020:
-
 #### Bug fixes
 * Added support for Raspbian Buster.
 * Upgrade dependency on Device SDK 1.19
@@ -174,7 +176,6 @@
 * The build option ENABLE_CAPTIONS does not enable captions. Enabling it will cause undefined behaviour during a build.
 
 ### v2.0.1 released 11/26/2019:
-
 #### Bug Fixes
 * Fixed segfault is situation when bad Misc DB path was provided in configuration.
 * Fixed situation where video playback controls were unresponsive on some skills.
@@ -185,7 +186,6 @@
 * In rare cases button presses could trigger repeat speech.
 
 ### v2.0 released 11/16/2019:
-
 Initial public release
 
 #### Known Issues

CMakeLists.txt

@@ -1,6 +1,6 @@
 cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
 
-project(AlexaSmartScreenSDK VERSION 2.9.0 LANGUAGES CXX)
+project(AlexaSmartScreenSDK VERSION 2.9.1 LANGUAGES CXX)
 set(PROJECT_BRIEF "A cross-platform, modular SDK for multi modal interaction with the Alexa Voice Service")
 configure_file (
         "${PROJECT_SOURCE_DIR}/modules/Alexa/Utils/include/Utils/SmartScreenSDKVersion.h.in"

README.md

@@ -1,12 +1,12 @@
 # Alexa Smart Screen SDK
 
 <p>
-    <a href="https://github.com/alexa/alexa-smart-screen-sdk/tree/v2.9.0" alt="version">
-        <img src="https://img.shields.io/badge/stable%20version-2.9.0-brightgreen" /></a>
+    <a href="https://github.com/alexa/alexa-smart-screen-sdk/tree/v2.9.1" alt="version">
+        <img src="https://img.shields.io/badge/stable%20version-2.9.1-brightgreen" /></a>
     <a href="https://github.com/alexa/avs-device-sdk/tree/v1.26.0" alt="DeviceSDK">
         <img src="https://img.shields.io/badge/avs%20device%20sdk-1.26.0-blueviolet" /></a>
-    <a href="https://github.com/alexa/apl-client-library/tree/v1.8.2" alt="APLClientLibrary">
-        <img src="https://img.shields.io/badge/apl%20client%20library-1.8.2-blue" /></a>
+    <a href="https://github.com/alexa/apl-client-library/tree/v1.8.3" alt="APLClientLibrary">
+        <img src="https://img.shields.io/badge/apl%20client%20library-1.8.3-blue" /></a>
     <a href="https://github.com/alexa/alexa-smart-screen-sdk/issues" alt="issues">
         <img src="https://img.shields.io/github/issues/alexa/alexa-smart-screen-sdk" /></a>
 </p>

modules/Alexa/ApplicationUtilities/Communication/src/WebSocketServer.cpp

@@ -99,25 +99,30 @@ bool WebSocketServer::start() {
     }
 #endif
 
-    websocketpp::lib::error_code errorCode;
+    try {
+        websocketpp::lib::error_code errorCode;
 
-    m_webSocketServer.start_accept(errorCode);
-    if (errorCode) {
-        ACSDK_ERROR(LX("server::start_accept")
-                        .d("errorCode", errorCode.value())
-                        .d("errorCategory", errorCode.category().name()));
-        return false;
-    }
+        m_webSocketServer.start_accept(errorCode);
+        if (errorCode) {
+            ACSDK_ERROR(LX("server::start_accept")
+                            .d("errorCode", errorCode.value())
+                            .d("errorCategory", errorCode.category().name()));
+            return false;
+        }
 
-    auto endpoint = m_webSocketServer.get_local_endpoint(errorCode);
-    if (errorCode) {
-        ACSDK_ERROR(LX("server::get_local_endpoint")
-                        .d("errorCode", errorCode.value())
-                        .d("errorCategory", errorCode.category().name()));
-        return false;
-    }
+        auto endpoint = m_webSocketServer.get_local_endpoint(errorCode);
+        if (errorCode) {
+            ACSDK_ERROR(LX("server::get_local_endpoint")
+                            .d("errorCode", errorCode.value())
+                            .d("errorCategory", errorCode.category().name()));
+            return false;
+        }
 
-    ACSDK_INFO(LX("Listening for websocket connections").d("interface", endpoint.address()).d("port", endpoint.port()));
+        ACSDK_INFO(
+            LX("Listening for websocket connections").d("interface", endpoint.address()).d("port", endpoint.port()));
+    } catch (const std::bad_weak_ptr& e) {
+        ACSDK_ERROR(LX("std::bad_weak_ptr").d("reason", e.what()));
+    }
 
     m_webSocketServer.run();
 

modules/Alexa/ApplicationUtilities/SmartScreenClient/include/SmartScreenClient/SmartScreenClient.h

@@ -1137,11 +1137,11 @@ AudioInputProcessor.
 
     /**
      * Handle execute command result.
-     * @param token The token.
-     * @param result The execute command result.
-     * @param error The error.
+     * @param token the execute command result token.
+     * @param event the command execution event.
+     * @param message the execute command completion message.
      */
-    void handleExecuteCommandsResult(const std::string& token, bool result, std::string error);
+    void handleExecuteCommandsResult(const std::string& token, const std::string& event, const std::string& message);
 
     /**
      * Handle activity event.

modules/Alexa/ApplicationUtilities/SmartScreenClient/src/SmartScreenClient.cpp

@@ -58,6 +58,7 @@
 #include <acsdkBluetooth/BluetoothMediaInputTransformer.h>
 #endif
 
+#include <AlexaPresentation/AplCommandsExecutionEvent.h>
 #include <SDKComponent/SDKComponent.h>
 
 #include "SmartScreenClient/DefaultClientComponent.h"
@@ -1942,8 +1943,14 @@ void SmartScreenClient::handleRenderDocumentResult(const std::string& token, boo
     m_alexaPresentation->processRenderDocumentResult(token, result, error);
 }
 
-void SmartScreenClient::handleExecuteCommandsResult(const std::string& token, bool result, std::string error) {
-    m_alexaPresentation->processExecuteCommandsResult(token, result, error);
+void SmartScreenClient::handleExecuteCommandsResult(
+    const std::string& token,
+    const std::string& event,
+    const std::string& message) {
+    m_alexaPresentation->processExecuteCommandsResult(
+        token,
+        alexaSmartScreenSDK::smartScreenCapabilityAgents::alexaPresentation::stringToCommandExecutionEvent(event),
+        message);
 }
 
 void SmartScreenClient::handleActivityEvent(

modules/Alexa/SampleApp/include/SampleApp/AplClientBridge.h

@@ -82,7 +82,10 @@ class AplClientBridge
 
     void onSendEvent(const std::string& token, const std::string& event) override;
 
-    void onCommandExecutionComplete(const std::string& token, bool result) override;
+    void onCommandExecutionComplete(
+        const std::string& token,
+        APLClient::AplCommandExecutionEvent event,
+        const std::string& message) override;
 
     void onRenderDocumentComplete(const std::string& token, bool result, const std::string& error) override;
 

modules/Alexa/SampleApp/include/SampleApp/GUI/GUIClient.h

@@ -374,11 +374,18 @@ class GUIClient
     void executeHandleTapToTalk(rapidjson::Document& message);
 
     /**
-     * Handle HoldToTalk event.
+     * Handle HoldToTalkStart event.
      *
      * @param message A complete message object with header and payload.
      */
-    void executeHandleHoldToTalk(rapidjson::Document& message);
+    void executeHandleHoldToTalkStart(rapidjson::Document& message);
+
+    /**
+     * Handle HoldToTalkEnd event.
+     *
+     * @param message A complete message object with header and payload.
+     */
+    void executeHandleHoldToTalkEnd(rapidjson::Document& message);
 
     /**
      * Handle focus acquire requests.

modules/Alexa/SampleApp/include/SampleApp/GUI/GUIManager.h

@@ -178,7 +178,7 @@ class GUIManager
     /// @{
     void handleTapToTalk() override;
 
-    void handleHoldToTalk() override;
+    void handleHoldToTalk(bool start) override;
 
     void handleMicrophoneToggle() override;
 
@@ -219,7 +219,8 @@ class GUIManager
 
     void handleRenderDocumentResult(std::string token, bool result, std::string error) override;
 
-    void handleExecuteCommandsResult(std::string token, bool result, std::string error) override;
+    void handleExecuteCommandsResult(const std::string& token, const std::string& event, const std::string& message)
+        override;
 
     void handleActivityEvent(
         alexaSmartScreenSDK::smartScreenSDKInterfaces::ActivityEvent event,
@@ -469,9 +470,6 @@ class GUIManager
     /// The wake word audio provider.
     alexaClientSDK::capabilityAgents::aip::AudioProvider m_wakeWordAudioProvider;
 
-    /// Whether a hold is currently occurring.
-    bool m_isHoldOccurring;
-
     /// Whether a tap is currently occurring.
     bool m_isTapOccurring;
 

modules/Alexa/SampleApp/src/AplClientBridge.cpp

@@ -150,9 +150,14 @@ void AplClientBridge::onSendEvent(const std::string& token, const std::string& e
     m_executor.submit([this, token, event] { m_guiManager->handleUserEvent(token, event); });
 }
 
-void AplClientBridge::onCommandExecutionComplete(const std::string& token, bool result) {
+void AplClientBridge::onCommandExecutionComplete(
+    const std::string& token,
+    APLClient::AplCommandExecutionEvent event,
+    const std::string& message) {
     ACSDK_DEBUG9(LX(__func__));
-    m_executor.submit([this, token, result] { m_guiManager->handleExecuteCommandsResult(token, result, ""); });
+    m_executor.submit([this, token, event, message] {
+        m_guiManager->handleExecuteCommandsResult(token, APLClient::commandExecutionEventToString(event), message);
+    });
 }
 
 void AplClientBridge::onRenderDocumentComplete(const std::string& token, bool result, const std::string& error) {
@@ -362,6 +367,11 @@ void AplClientBridge::executeCommands(const std::string& jsonPayload, const std:
         auto aplClientRenderer = getAplClientRendererFromAplToken(token);
         if (aplClientRenderer) {
             aplClientRenderer->executeCommands(jsonPayload, token);
+        } else {
+            m_guiManager->handleExecuteCommandsResult(
+                token,
+                APLClient::commandExecutionEventToString(APLClient::AplCommandExecutionEvent::FAILED),
+                "No document renderer instance for token.");
         }
     });
 }

modules/Alexa/SampleApp/src/GUI/GUIClient.cpp

@@ -35,8 +35,11 @@ static const std::string MESSAGE_TYPE_INIT_RESPONSE("initResponse");
 /// The message type for Tap To Talk.
 static const std::string MESSAGE_TYPE_TAP_TO_TALK("tapToTalk");
 
-/// The message type for Hold To Talk.
-static const std::string MESSAGE_TYPE_HOLD_TO_TALK("holdToTalk");
+/// The message type for Hold To Talk start.
+static const std::string MESSAGE_TYPE_HOLD_TO_TALK_START("holdToTalkStart");
+
+/// The message type for Hold To Talk end.
+static const std::string MESSAGE_TYPE_HOLD_TO_TALK_END("holdToTalkEnd");
 
 /// The message type for Focus acquire request.
 static const std::string MESSAGE_TYPE_FOCUS_ACQUIRE_REQUEST("focusAcquireRequest");
@@ -331,8 +334,11 @@ GUIClient::GUIClient(
         m_captionManager{SmartScreenCaptionStateManager(miscStorage)} {
     m_messageHandlers.emplace(
         MESSAGE_TYPE_TAP_TO_TALK, [this](rapidjson::Document& payload) { executeHandleTapToTalk(payload); });
+    m_messageHandlers.emplace(MESSAGE_TYPE_HOLD_TO_TALK_START, [this](rapidjson::Document& payload) {
+        executeHandleHoldToTalkStart(payload);
+    });
     m_messageHandlers.emplace(
-        MESSAGE_TYPE_HOLD_TO_TALK, [this](rapidjson::Document& payload) { executeHandleHoldToTalk(payload); });
+        MESSAGE_TYPE_HOLD_TO_TALK_END, [this](rapidjson::Document& payload) { executeHandleHoldToTalkEnd(payload); });
     m_messageHandlers.emplace(MESSAGE_TYPE_FOCUS_ACQUIRE_REQUEST, [this](rapidjson::Document& payload) {
         executeHandleFocusAcquireRequest(payload);
     });
@@ -608,8 +614,12 @@ void GUIClient::executeHandleTapToTalk(rapidjson::Document& message) {
     m_guiManager->handleTapToTalk();
 }
 
-void GUIClient::executeHandleHoldToTalk(rapidjson::Document& message) {
-    m_guiManager->handleHoldToTalk();
+void GUIClient::executeHandleHoldToTalkStart(rapidjson::Document& message) {
+    m_guiManager->handleHoldToTalk(true);
+}
+
+void GUIClient::executeHandleHoldToTalkEnd(rapidjson::Document& message) {
+    m_guiManager->handleHoldToTalk(false);
 }
 
 void GUIClient::executeHandleAcceptCall(rapidjson::Document& message) {

modules/Alexa/SampleApp/src/GUI/GUIManager.cpp

@@ -127,7 +127,6 @@ GUIManager::GUIManager(
         m_playerActivityState{PlayerActivity::FINISHED} {
     m_guiClient = guiClient;
     m_isMicOn = true;
-    m_isHoldOccurring = false;
     m_isTapOccurring = false;
     m_isSpeakingOrListening = false;
     m_clearAlertChannelOnForegrounded = false;
@@ -240,15 +239,20 @@ void GUIManager::handleTapToTalk() {
         ;
 }
 
-void GUIManager::handleHoldToTalk() {
-    ACSDK_DEBUG9(LX("handleHoldToTalk"));
-    m_executor.submit([this]() {
+void GUIManager::handleHoldToTalk(bool start) {
+    ACSDK_DEBUG9(LX(__func__).d("start", start));
+    m_executor.submit([this, start]() {
         handleASRProfileChanged(m_holdToTalkAudioProvider.profile);
         if (!m_isMicOn) {
             return;
         }
         bool activeCameraWithMicrophone = false;
 
+        // Set the value of @c isHoldOccurring to what UI thinks it should be.
+        // There could be circumstances where the user applications might fall out
+        // of sync with SDK about the status of hold-to-talk.
+        bool isHoldOccurring = !start;
+
 #ifdef ENABLE_RTCSC
         // Mic input is fully routed to active camera and not Alexa AIP when:
         // - ASR Profile is CLOSE_TALK (physical remote mic input)
@@ -264,12 +268,12 @@ void GUIManager::handleHoldToTalk() {
               smartScreenSDKInterfaces::AudioState::UNKNOWN != m_cameraMicrophoneAudioState) &&
              smartScreenSDKInterfaces::ConcurrentTwoWayTalk::UNKNOWN != m_cameraConcurrentTwoWayTalk);
 #endif
-        if (!m_isHoldOccurring) {
+        if (!isHoldOccurring) {
             // If we have no active 2-way talk camera, route mic input to Alexa AIP provider as usual.
-            m_isHoldOccurring =
+            isHoldOccurring =
                 activeCameraWithMicrophone || m_ssClient->notifyOfHoldToTalkStart(m_holdToTalkAudioProvider).get();
         } else {
-            m_isHoldOccurring = false;
+            isHoldOccurring = false;
             if (!activeCameraWithMicrophone) {
                 m_ssClient->notifyOfHoldToTalkEnd();
             }
@@ -279,9 +283,9 @@ void GUIManager::handleHoldToTalk() {
         // If we have an active 2-way camera, enable/disable its microphone.
         if (activeCameraWithMicrophone) {
             // Set camera mic state
-            handleSetCameraMicrophoneState(m_isHoldOccurring);
+            handleSetCameraMicrophoneState(isHoldOccurring);
             // Inform GUI of camera mic state
-            m_guiClient->handleCameraMicrophoneStateChanged(m_isHoldOccurring);
+            m_guiClient->handleCameraMicrophoneStateChanged(isHoldOccurring);
         }
 #endif
     });
@@ -580,9 +584,12 @@ void GUIManager::handleRenderDocumentResult(std::string token, bool result, std:
     m_executor.submit([this, result, token, error]() { m_ssClient->handleRenderDocumentResult(token, result, error); });
 }
 
-void GUIManager::handleExecuteCommandsResult(std::string token, bool result, std::string error) {
+void GUIManager::handleExecuteCommandsResult(
+    const std::string& token,
+    const std::string& event,
+    const std::string& message) {
     m_executor.submit(
-        [this, token, result, error]() { m_ssClient->handleExecuteCommandsResult(token, result, error); });
+        [this, token, event, message]() { m_ssClient->handleExecuteCommandsResult(token, event, message); });
 }
 
 void GUIManager::handleActivityEvent(smartScreenSDKInterfaces::ActivityEvent event, const std::string& source) {