Our interactive app, Vocate, uses photos to help seniors remember and cherish past memories. The user interacts with Vocate by having a conversation with it. In other words, Vocate uses complete text-to-speech and speech-to-text to make the user simply speak and have the app speak back. With an easy-to-use interface, the app talks to the user about various photos uploaded on the device using the information provided with the pictures. The information given to the app includes details like the people in the picture, and a special question to the user that can be inserted by a caretaker (e.g. "which restaurant did we celebrate your birthday at?"). Vocate uses this information to either “ask” or “tell” about a photo to make the conversation more fluid. This detailed information and switching between “ask” and “tell” modes makes interactions with the app more engaging, dynamic, and improves the overall user experience.
Vocate has a "small talk" feature where it initially asks how the user is doing and engages in other simple dialogue making the app more amiable.
It also asks follow-up questions during conversation. For example, when talking about people in the photo, it will ask if they are family; when asking about something on where the photo was taken, it will ask if the user had been there before.
Furthermore, Vocate has support for a loved one or a “caretaker” to help the user interact with the app; they can upload pictures (with information) and look at statistics on how the user is interacting with the app. These statistics are able to display how the user is able to recollect and what memories they have a difficult time recollecting.
To prevent frequent repetition of photos, the app uses an algorithm that brings up memories in an intelligent order.
Due to the lockdown enforcement and unsafe conditions (especially for older people) caused by COVID-19, almost all seniors are having to stay home without socializing, leading to severe loneliness. In addition to improving their memory, Vocate tries to improve their quality of life by putting a smile on their faces and making their day just a little bit brighter. It helps them look at happy memories and happy photos assuring them that one day they will be back out and be making more cheerful and joyful memories.