Multi Stream in SDK 2.8 Using Aux Streams

What is Multi-stream?

With multi-stream, you'll see the video of the most active participants in your meetings. Each stream is referred to as an auxiliary stream. You can open a certain number of auxiliary videos to display active participants. The most active speaker who is talking now will display on the main remote video view, and the other active speakers will display on the auxiliary video views.

Typical Scenario

In a meeting with more than two participants, if you want to see the active speaker along with other joined participants, you can use multi-stream to achieve it.

Limitations

• You can only open 4 auxiliary video streams at most.
• You cannot specify a stream showing a participant, and which is only determined by the activity of the participants.

How to Use

In order to implement multi-stream, client should inherit from the MultiStreamObserver interface and implement it. The following is the main steps you should follow.

public interface MultiStreamObserver {
    View onAuxStreamAvailable();
    View onAuxStreamUnavailable();
    void onAuxStreamChanged(AuxStreamChangedEvent event);
}

1. Provide a view to open an auxiliary stream

When there is a new available auxiliary stream, such as a new participant join in the meeting, SDK will callback OnAuxStreamAvailable and the client should give SDK a view for rendering, and the AuxStreamOpenedEvent would be triggered indicating whether the stream is successfully opened.

@Override
public View onAuxStreamAvailable() {
    // SDK will open this auxiliary stream with this view and the result will be notified by AuxStreamOpenedEvent.
    View auxStreamView = LayoutInflater.from(getActivity()).inflate(R.layout.remote_video_view, null);
    AuxStreamViewHolder auxStreamViewHolder = new AuxStreamViewHolder(auxStreamView);
    mAuxStreamViewMap.put(auxStreamViewHolder.mediaRenderView, auxStreamViewHolder);
    return auxStreamViewHolder.mediaRenderView;
}

2. Provide a view to close the auxiliary stream

When there is an auxiliary stream unavailable, such as a participant left the meeting and the number of joined participants is smaller than the number of opened streams, SDK will callback OnAuxStreamUnavailable and the client should give SDK a view handle which will be closed or if the given view is null, SDK will automatically close the last opened stream if needed.

@Override
public View onAuxStreamUnavailable() {
    // You can indicate to close which view or let SDK automatically close the last opened view.
    // The result will be notified by AuxStreamClosedEvent.
    return null;
}

3. Display the view of the auxiliary stream

When an auxiliary stream is opened successfully or not, AuxStreamOpenedEvent will be triggered. On this event, the client can display the view of the auxiliary stream if the result is successful.

4. Hide the view of the auxiliary stream

When an auxiliary stream is closed successfully or not, AuxStreamClosedEvent will be triggered. On this event, the client can hide the view of the auxiliary stream if the result is successful.

@Override
public void onAuxStreamChanged(AuxStreamChangedEvent event) {
    if (event instanceof MultiStreamObserver.AuxStreamOpenedEvent) {
        if ((AuxStreamOpenedEvent)event.isSuccessful()){
            Ln.d("AuxStreamOpenedEvent successful");
            viewAuxVideos.addView(mAuxStreamViewMap.get((AuxStreamOpenedEvent)event.getRenderView()).item);
        }else{
            Ln.d("AuxStreamOpenedEvent failed: " + (AuxStreamOpenedEvent)event.getError());
            mAuxStreamViewMap.remove((AuxStreamOpenedEvent)event.getRenderView());
        }
    } else if (event instanceof MultiStreamObserver.AuxStreamClosedEvent) {
        if ((AuxStreamClosedEvent)event.isSuccessful()){
            Ln.d("AuxStreamClosedEvent successful");
            AuxStreamViewHolder auxStreamViewHolder = mAuxStreamViewMap.get((AuxStreamClosedEvent)event.getRenderView());
            mAuxStreamViewMap.remove((AuxStreamClosedEvent)event.getRenderView());
            viewAuxVideos.removeView(auxStreamViewHolder.item);
        }else{
            Ln.d("AuxStreamClosedEvent failed: " + (AuxStreamClosedEvent)event.getError());
        }
    } 
}