Meeting Summarization Use Case

[manisnesan]

          [From rasbt post](https://x.com/rasbt/status/1754516687896887449?s=46&t=aOEVGBVv9ICQLUYL4fQHlQ) - Flan T5 is a great go to model for text classification. 

Tiny titans - Can smaller LLM models punch above their weight for meeting summarization

Originally posted by @manisnesan in #47 (comment)

Questions

• What are the datasets available
• What are the key constituents involved in an effective meeting summary
• What are the challenges involved in creating effective meeting summarization
• What are the most recent advancements in meeting summarization tech
• How is this different from other diverse summarization involved in news, science, technology, medical
• How meeting summarization is related customer service call summarization?
  • meeting (multi party interactions - more than two speakers) where as customer service call or medical appoints l is a biparty interaction ie only two speakers are involved.

[manisnesan]

Meeting Summarization

Meeting summarization is the process of creating a concise overview of the key points, decisions, and action items discussed during a meeting[1]. It serves to keep stakeholders informed, facilitate decision-making, encourage accountability, and enhance communication[1].

There are several proven ways to summarize a meeting effectively:

1. Take concise notes during the meeting, focusing on the most important information[1].

2. Use a clear and organized format in the summary, such as including the date, time, location, attendees, agenda items, discussion points, decisions, action items, and next steps[1].

3. Follow and fill out the meeting agenda when creating the summary notes[1].

4. Summarize the meeting over email to all participants after the fact[1].

5. Use AI tools to automatically generate meeting summaries from transcripts[1][2].

Challenges in meeting summarization include the difficulty of collecting confidential meeting data, the labor-intensive process of annotating summaries, and the need to capture key issues while excluding irrelevant discussions[4][5]. Recent research has focused on creating benchmark datasets[3][4][5] and developing advanced summarization models[2][3].

In summary, meeting summarization is a crucial skill for keeping teams aligned and productive, with various manual and automated techniques available to create high-quality summaries efficiently.

Citations:
[1] https://fireflies.ai/blog/summarize-a-meeting
[2] https://github.com/topics/meeting-summarization
[3] https://paperswithcode.com/task/meeting-summarization
[4] https://arxiv.org/abs/2305.17529
[5] https://aclanthology.org/2023.acl-long.906.pdf

[manisnesan]

Diverse Summarization Dataset

From Pegasus - Paper

[manisnesan]

From Abstractive Meeting Summarization

• A system that could reliably identify and sum up the most important points of a conversation would be valuable in a wide variety of real-world contexts, from business meetings to medical consultations to customer service calls.

Customer Service Calls could be multi-party conversation but only two party speak in a given time span. Also the format of the meeting in customer service is problem solving in nature.

Eg: Customer Rep - Agent 1 ---> Customer Rep - Agent 2 ----> Customer Rep -- Agent 3

Related: Abstractive Dialogue summarization, Abstractive Text Summarization, Meeting Summariziation, text Generation

Stages in abstractive

• Selection of important points that are worthy enough. This is same as extractive summarization.

• Synthesis

• language generation
  

• Figure 1 shows excerpts of the human-made extractive (left column) and abstractive (right col- umn) summaries of meeting ES2011c.

• The col- ored lines relate each abstractive sentence to the set of extractive sentences—the abstractive com- munity—that annotators judged as supporting it.

[manisnesan]

Differences from traditional summarization

• linguistic interactions involved in the meetings
• multiparty conversations

[manisnesan]

From Call Summarization: why it is important and what it is possible today and in a near future

• It aims to automatically generate concise, fluent summaries capturing the key points of a conversation, which can help improve customer experience and reduce agent workload

"AUTOMATIC SUMMARIZATION OF CALL-CENTER CONVERSATION" by E. Stepanov, B. Favre, F. Alam, S. Chowdhury, K. Singla, J. Trione, F. Be ́chet, G. Riccardi. offers a hybrid approach using both extractive/abstractive.

See

• "State of the Art Summarisation Techniques", Information Systems Seminar 19/20 by Anna Franziska Bothe, Alex Truesdale, LukasKolbe
• https://humboldtwi.github.io/blog/research/information_systems_1920/nlp_text_summarization_techniques/

[manisnesan]

From Generating Abstractive Summaries from Meeting Transcripts

[manisnesan]

Challenges involved

Nature of meeting-style speech :

• leads to low information density & high noise
• significantly longer eg: AMI transcript tokens 4, 757 & its summary 322
• constrasting to two speaker conversations - multiparty conversations has challenges to speaker & addressee identification

Preference for abstractive summarization

• LEAD-3 baseline - extractive methods - first 3 sentences of a doc
• selection of important material

Heterogeneous meeting formats

• sharing info or brainstorming ideas
• depending on meeting formats require a variety of automatic systems

Subjectivity

• reformulating the same content in different words & style
• what is counted as summary-worth

[manisnesan]

See the example case study from Orca paper on Meeting Transcript processing

Example from the paper

System

You are a teacher. Given a task, you explain in simple steps what the task is asking, any guidelines it provides, and how to use those guidelines it provides to find the answer.

User

You will read a meeting transcript, then extract the relevant segments to answer the following question

Question: How does Steven feel about selling?

$Meeting_Transcript

Please answer the following question
Question: How does Steven feel about selling?

Extract from transcript the most relevant segments for the answer, then answer the question.

[manisnesan]

https://www.reddit.com/r/LocalLLaMA/s/xeSFTXwa5q

[manisnesan]

https://community.openai.com/t/how-to-summarize-large-research-articles/142730

[manisnesan]

Five levels of summarizing Youtube

• langchain map reduce is an interesting idea showcased
• Topic modelling using language models is also another interesting approach here

Usecase

YouTube Videos - Auto Chapter Generation
Podcasts - Extract structured information
Meeting Notes - Send topic summaries to participants
Town Hall Meetings - Structured information
Earnings Report Calls - Sell structured data to investment groups
Legal Documents - Quickly summarize by topic
Movie Scripts - Quick bullet points for production recaps
Books - Auto generate table of contents