Google MeetSense

A demo application showing a post-meeting summary, action items, and other metrics.
View Demo »

Table of Contents

1. About The Project
   • Demo
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. License
4. Contact

About The Project

Google MeetSense was created to answer the following question:

How might we ensure that meeting organizers have a clear understanding of how to move forward after virtual medium-sized meetings?

Over the course of five months our team has worked to create a prototype application that demonstrates the vision for a tool that could provide a post-meeting summary, action items, and other metrics to meeting organizers. During this time, we have received feedback from a team in Google's Corporate Engineering division, our BigCo studio instructors, peers, and others. We have conducted numerous interviews to guide our thinking toward this demo application. The result is a combination of a Figma mockup, an API that processes an audio file of a real meeting to produce a real meeting summary and statistics, and a front-end application that displays this information.

[Back to Top]

Demo

A demo video of the ideal usage of the product is shown below. It shows what our product might look like if integrated with Google Meet.

[Back to Top]

Built With

• 
• 
• 
• 
• 

[Back to Top]

Getting Started

There are a couple components to the project and the files have been broken down as follows:

1. colab notebooks: These are the Machine Learning (ML) research and prototyping code that was used to finetune the summarization model. Mainly contains attempts to fine-tune summarization (and a few transcription) models.
2. react-app: This is the front-end application that contains all the user-facing pages.
3. studio-api: This is the Google Cloud api that holds the code needed for ML models, metrics, and synthetic data.

Prerequisites

• npm

  npm install npm@latest -g

• gcloud CLI (see here)
• Google Cloud project + linked billing account

Installation

General Steps:

1. Clone the repo

   git clone https://github.com/jainr3/Google-MeetSense.git
   

2. Install required pre-requisites. See Prerequisites.

3. Setup a Google Cloud project for the API and Firebase. Generate a key.json file for the API to access Firebase (more instructions here).

Steps for deploying the studio-api on Google Cloud Run platform:

1. Download the finetuned meeting summarization model from Huggingface Model hub here. Place the t5-model-v3.pth file in the studio-api/model folder.

2. Change directory in your terminal to the studio-api folder. Run gcloud run deploy. Note that to test the project locally, one may run gcloud beta run services proxy studio-api --project insert-gcloud-project-name (may require installation of some other packages).

3. Monitor the build progress and API logs on the Google Cloud console. Note: may need to increase the API timeout to 3600sec and allocate 2 CPUs with 8GiB memory. Also need to check box for Allow unauthenticated invocations under the security tab.

Steps for setting up Google Firebase:

1. Setup a firestore database collection titled meetings under the same Google Cloud project.

Running the react-app:

1. Ensure that the pre-requisites are installed and run npm install (may need to delete the package-lock.json file).

2. Change directory into the react-app folder and run npm start to view the page.

[Back to Top]

License

Distributed under the Cornell Standard Project Agreement (Version 2.0, May 17, 2021). See here for the latest information.

[Back to Top]

Contact

This project was created by Rahul Jain, James Chu, Jenny Zhu, Kıvanç Komesli, and Katherine Lui during the BigCo Studio course (Spring 2023) at Cornell Tech.

Project Link: https://github.com/jainr3/Google-MeetSense

[Back to Top]