Multi Stream in SDK 3.5 Using Categories

What is Multi-stream?

With multi-stream, you'll see the video of the most active participants in your meetings.

NOTE: The below new APIs are available from SDK version 3.5.0 onwards.

• Stream order will be allocated in the order of joining the meeting.
• 2 Categories are present - Category-A, Category-B.
• Category-A: It will contain 1 stream and it will be the active speaker participant in the meeting.
• Category-B: It will contain all other streams in the meeting. It can also contain the duplicate active speaker stream.
• Category-B can be restricted to contain the number of streams. If more participant joins the meeting above the limit they will go into the queue.
• If someone from the queue will speak he will get into the allocated stream and will replace the stream who is less active in the meeting.
• If someone from the allocated stream leaves the meeting, the random participant from the queue will replace the stream.
• The Active Speaker API can be used to check if the participant is an active speaker.
• The ActiveSpeakerChangedEvent is triggered if active speaker changes.

Typical Scenario

In a meeting with more than two participants, if you want to see the active speaker along with other joined participants, you can use multi-stream to achieve it.

How to Use

To implement multi-stream, the client should implement MediaStreamAvailabilityEvent which is part of CallObserver.onMediaChanged. The following are the main steps you should follow.

override fun onMediaChanged(event: CallObserver.MediaChangedEvent?) {
     event?.let { _event ->
            val call = _event.getCall()
            when (_event) {
                is CallObserver.MediaStreamAvailabilityEvent -> {
                    // event is CallObserver.MediaStreamAvailabilityEvent
                    //if (event.isAvailable()) {
                    //     new stream available
                    //} else {
                    //     stream unavailable
                    //}
                }
                else -> {}
            }
}

1. Provide a view for rendering the stream

When there is a newly available stream, such as a new participant joining the meeting, the above stream availability event will be fired and the client should provide the view to SDK for rendering the video of the stream.

// event is CallObserver.MediaStreamAvailabilityEvent
event.getStream()?.setRenderView(view)

2. OnMediaStreamInfoChanged Listener

We support different event types - MediaStreamChangeEventType. If any of the types changes concerning the stream, the event will get triggered.

enum class MediaStreamChangeEventType {
    Size,
    Membership,
    Video,
    Audio
}

The client can register the info change listener to receive the event change.

// event is CallObserver.MediaStreamAvailabilityEvent
event.getStream()?.setOnMediaStreamInfoChanged { type, info ->
                    //type: MediaStreamChangeEventType
                    //info: MediaStreamChangeEventInfo
                }

3. setMediaStreamCategoryA

We can add the active speaker stream with the specified parameters if it does not already exist otherwise, update the active speaker stream with the specified parameters.

Category-A will contain only 1 stream and it will always be an active speaker stream.

// duplicate if true, the same active speaker will also be included in Category-B streams.
    fun setMediaStreamCategoryA(duplicate: Boolean, quality: MediaStreamQuality)

enum class MediaStreamQuality {
    Undefined,
    LD,  // 180p
    SD,  // 360p
    HD,  // 720p
    FHD // 1080p
}

4. setMediaStreamsCategoryB

Set all Category-B streams to the specified params.

• If the number of existing B streams is less than numStreams, it will add B streams as necessary.
• If the number of existing B streams is more than numStreams, it will remove the extra B streams.

    fun setMediaStreamsCategoryB(numStreams: Int, quality: MediaStreamQuality)

5. removeMediaStreamCategoryA

Remove the Active Speaker stream. To add the active stream back, use setMediaStreamCategoryA().

• After removal of Category-A, if Category-B exists, all streams will be of the same priority and streams will not change based on the active speaker.

    fun removeMediaStreamCategoryA()

5. removeMediaStreamsCategoryB

Remove all Category-B streams. To change the number of B streams, use setMediaStreamsCategoryB().

• After removal of Category-B, if Category-A exists then only the active speaker stream will be visible and the participant who speaks will be visible in the Category-A stream.

    fun removeMediaStreamsCategoryB()