Skip to content

Commit

Permalink
Documentation updates
Browse files Browse the repository at this point in the history
  • Loading branch information
Cartisim committed Mar 22, 2023
1 parent e770d37 commit 97ed5de
Show file tree
Hide file tree
Showing 4 changed files with 31 additions and 39 deletions.
16 changes: 8 additions & 8 deletions Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ FCSDKiOS offers Picture in Picture support. In order to use this feature you wil

The Picture in Picture feature is set up in the consumer of the **SDK**, however **FCSDKiOS** gives you all of the needed pieces in order to build a PiP app. 2 things are required to use Picture in Picture mode:

1. Your application must be using **FCSDKiOS**'s Preview Buffer View
2. Your application must be using **FCSDKiOS**'s Sample Buffer View
1. Your application must be using **FCSDKiOS**'s Local Buffer View
2. Your application must be using **FCSDKiOS**'s Remote Buffer View

If you have conformed to these 2 **UIView**s please see how to set up Picture in Picture below.

### AVCaptureSession
In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your Devices is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise, you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView.
In order to check if your devices support Multitasking Camera Access, when a **LocalBufferView** is created we can get the **AVCaptureSession** from the **LocalBufferView**. If your devices are supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise, you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **LocalBufferView** for your Local Video UIView.
```swift
@available(iOS 15, *)
@objc final public func captureSession() async -> AVCaptureSession?
Expand All @@ -28,7 +28,7 @@ An example in checking for the method is as follows:
```

### Set PiP Controller
Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture.
Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **RemoteBufferView** accordingly when the app activates Picture in Picture.
```swift
@available(iOS 15, *)
@MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async
Expand All @@ -54,17 +54,17 @@ if show {
}

func setUpPip() async {
guard let sampleBufferView = sampleBufferView else { return }
guard let remoteBufferView = remoteBufferView else { return }

if #available(iOS 16.0, *) {
guard let captureSession = captureSession else { return }
if captureSession.isMultitaskingCameraAccessSupported {
let pipVideoCallViewController = AVPictureInPictureVideoCallViewController()
pipVideoCallViewController.preferredContentSize = CGSize(width: 1080, height: 1920)
pipVideoCallViewController.view.addSubview(sampleBufferView)
pipVideoCallViewController.view.addSubview(remoteBufferView)

let source = AVPictureInPictureController.ContentSource(
activeVideoCallSourceView: sampleBufferView,
activeVideoCallSourceView: remoteBufferView,
contentViewController: pipVideoCallViewController)

pipController = AVPictureInPictureController(contentSource: source)
Expand All @@ -81,7 +81,7 @@ if #available(iOS 16.0, *) {
}

func aChipLogic() async {
let layer = sampleBufferView.layer as? AVSampleBufferDisplayLayer
let layer = remoteBufferView.layer as? AVSampleBufferDisplayLayer
guard let sourceLayer = layer else { return }

let source = AVPictureInPictureController.ContentSource(sampleBufferDisplayLayer: sourceLayer, playbackDelegate: self)
Expand Down
14 changes: 3 additions & 11 deletions Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,16 @@ Please see <doc:VideoCalls> for an explanation of how to use ACBClientCall
@objc final public var remoteView: UIView?

/// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications
@available(*, deprecated, message: "use sampleBufferView() async -> UIView? instead, This method will be removed in future versions of FCSDKiOS")
@objc final public func remoteBufferView() async -> UIView?

/// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications
@MainActor @objc final public func sampleBufferView() async -> UIView?

/// The client should always remove the `remoteBufferView()` when finished using the bufferView.
@available(*, deprecated, message: "use removeSampleView() async instead, This method will be removed in future versions of FCSDKiOS")
@objc final public func removeBufferView() async

/// The client should always remove the `remoteBufferView()` when finished using the bufferView.
@objc final public func removeSampleView() async

/// The call's ``UIView`` used for displaying local buffer views intended for use in picture in picture/Virtual Background enabled applications
@MainActor @objc final public func previewBufferView() async -> UIView?
@MainActor @objc final public func localBufferView() async -> UIView?

/// The client should always remove the `removePreviewView()` when finished using the previewView.
@objc final public func removePreviewView() async
@objc final public func removeLocalBufferView() async

/// The AVCaptureSession that the PreviewBufferView provides
@objc final public func captureSession() async -> AVCaptureSession?
Expand All @@ -44,7 +36,7 @@ Please see <doc:VideoCalls> for an explanation of how to use ACBClientCall
@MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async

/// Feeds the UIImage from the consuming app into the SDK for Video Processing
@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async
@objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async

/// Removes the Background Image from the Video Processsing flow
@objc final public func removeBackgroundImage() async
Expand Down
28 changes: 14 additions & 14 deletions Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,34 +5,34 @@ This article describes changes in version 4.2.0 of FCSDKiOS
## Overview
Version 4.2.0 reaches a new milestone in terms of Video Calls. We now support Picture in Picture and Virtual Background. The following Topics are the changes that we have made.

### Preview Buffer View
### Local Buffer View
This method is designed to create a metal rendered `UIView` for your local video during a video call. This method must be used if you desire to use our **Virtual Background** feature.
When a call is finished the **SDK** consumer is required to clean up the call by calling the `removePreviewView` method.
When a call is finished the **SDK** consumer is required to clean up the call by calling the `removeLocalBufferView` method.
```swift
@available(iOS 15, *)
@MainActor @objc final public func previewBufferView() async -> UIView?
@MainActor @objc final public func localBufferView() async -> UIView?
```

```swift
@available(iOS 15, *)
@objc final public func removePreviewView() async
@objc final public func removeLocalBufferView() async
```

### Sample Buffer View
### Remote Buffer View
This method is designed to create a metal rendered `UIView` for your remote video during a video call. This method must be used if you desire to use our **Picture in Picture** feature.
When a call is finished the **SDK** consumer is required to clean up the call by calling the `removeBufferView` method.
```swift
@available(iOS 15, *)
@MainActor @objc final public func sampleBufferView() async -> UIView?
@MainActor @objc final public func remoteBufferView() async -> UIView?
```

```swift
@available(iOS 15, *)
@objc final public func removeSampleView() async
@objc final public func removeBufferView() async
```

### AVCaptureSession
In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your devices are supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView.
In order to check if your devices support Multitasking Camera Access, when a **LocalBufferView** is created we can get the **AVCaptureSession** from the **LocalBufferView**. If your device is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **LocalBufferView** for your Local Video UIView.
```swift
@available(iOS 15, *)
@objc final public func captureSession() async -> AVCaptureSession?
Expand All @@ -48,18 +48,18 @@ if captureSession.isMultitaskingCameraAccessSupported {
```

### Set PiP Controller
Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture.
Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **RemoteBufferView** accordingly when tha app activates Picture in Picture.
```swift
@available(iOS 15, *)
@MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async
```

### Feed Background Image
In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. This will save your application from excessive memory and cpu usage. If you fail to feed the SDK a smaller **UIImage** we will automatically adjust the size to an acceptable limit. If you wish to blur the background instead of using an image, you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`.
### Feeding a Background Image
In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. Typically you will want to take the resolution into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropriate image size then **FCSDKiOS** will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum without including a **UIImage** in the image property parameter.

```swift
@available(iOS 15, *)
@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async
@objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async
```

```swift
Expand All @@ -71,8 +71,8 @@ In order to use the **Virtual Background** feature the **SDK** consumer is requi
When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur`
```swift
@objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable {
case blur
case image
case blur
case image
}
```

Expand Down
12 changes: 6 additions & 6 deletions Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# VirtualBackground

FCSDKiOS offers virtual background support. In order to use this feature, you will need to be able to provide a UIImage to FCSDKiOS. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. It is encouraged for the SDK consumer to handle feeding a *UIImage* that has been appropriately sized in order to prevent any undefined behavior. Typically you will want to take the resoltion into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropiate image size then FCSDKiOS will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*.
FCSDKiOS offers virtual background support. This feature offers an option to blur your camera capture's background or simply provide a virtual background image.

## Overview

The Virtual Background feature has been robustly built natively for your performance. Below we describe how to set up Virtual Background.

## Feed Background Image
In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`.
## Feeding a Background Image
In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. Typically you will want to take the resolution into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropriate image size then **FCSDKiOS** will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum without including a **UIImage** in the image property parameter.

```swift
@available(iOS 15, *)
@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async
@objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async
```

```swift
Expand All @@ -23,8 +23,8 @@ In order to use the **Virtual Background** feature the **SDK** consumer is requi
When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur`
```swift
@objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable {
case blur
case image
case blur
case image
}
```

Expand Down

0 comments on commit 97ed5de

Please sign in to comment.