From 004ca7883724a6d4437cccec3f918af13caa9c71 Mon Sep 17 00:00:00 2001 From: Cole Moore Date: Tue, 14 Mar 2023 19:27:30 +0300 Subject: [PATCH 1/3] Updated docs for version 4.2.0 --- .../SwiftFCSDKiOS.docc/GettingStarted.md | 4 + .../MigratingFromLegacySDK.md | 2 + .../SwiftFCSDKiOS.docc/PictureinPicture.md | 93 +++++++++++++++++++ .../Symbols/ACBClientCall.md | 32 +++++++ .../SwiftFCSDKiOS.docc/Symbols/Constants.md | 2 + .../SwiftFCSDKiOS.docc/Version-4.2.0.md | 83 +++++++++++++++++ .../SwiftFCSDKiOS.docc/VirtualBackground.md | 67 +++++++++++++ 7 files changed, 283 insertions(+) create mode 100644 Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md create mode 100644 Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/GettingStarted.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/GettingStarted.md index 63a3433..d8c4279 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/GettingStarted.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/GettingStarted.md @@ -12,6 +12,10 @@ This Documentation contains what you need in order to build an FCSDKiOS applicat - **You must be using Swift 5.5 or greater to use use this SDK** ### Articles + + +This article is to guide you through the changes of FCSDKiOS version 4.2.0. + This article is to guide you through the changes of FCSDKiOS version 4.1.0. diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/MigratingFromLegacySDK.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/MigratingFromLegacySDK.md index eb03325..14c4ef0 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/MigratingFromLegacySDK.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/MigratingFromLegacySDK.md @@ -8,6 +8,8 @@ Starting in FCSDKiOS 4.0.0 we have migrated to a native Swift code base. We have ### Version Changes + + ### Imports diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md index f2ac86f..b4b8d99 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md @@ -1,5 +1,98 @@ # Picture in Picture +FCSDKiOS offers Picture in Picture support. In order to use this feature you will need to be able to conform to **AVPictureInPictureControllerDelegate** and/or **AVPictureInPictureSampleBufferPlaybackDelegate**. The **SDK** consumer is completely responsible for conforming to these delegates. However you will need to get a layer or view from the **SDK** to present in the **AVPictureInPicture** code. + +## Overview + +The Picture in Picture feature is set up in the consumer of the **SDK**, however **FCSDKiOS** gives you all of the needed pieces in order to build a PiP app. 2 things are required to use Picture in Picture mode: + +1. Your application must be using **FCSDKiOS**'s Preview Buffer View +2. Your application must be using **FCSDKiOS**'s Sample Buffer View + +If you have conformed to these 2 **UIView**s please see how to set up Picture in Picture below. + +### AVCaptureSession +In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your Devices is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise, you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. +```swift +@available(iOS 15, *) +@objc final public func captureSession() async -> AVCaptureSession? +``` + +An example in checking for the method is as follows: +```swift + if captureSession.isMultitaskingCameraAccessSupported { + // Use AVPictureInPictureVideoCallViewController + } else { + // Use AVPictureInPictureController + } +``` + +### Set PiP Controller +Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture. +```swift +@available(iOS 15, *) +@MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async +``` + +Now that you have an understanding of the requirements, please see the example below of how a PiP flow could be written. + +```swift +@available(iOS 15.0, *) +var pipController: AVPictureInPictureController! + + +func showPip(show: Bool) async { +if show { + if AVPictureInPictureController.isPictureInPictureSupported() { + pipController.startPictureInPicture() + } else { + self.logger.info("PIP not Supported") + } + } else { + pipController.stopPictureInPicture() + } +} + +func setUpPip() async { +guard let sampleBufferView = sampleBufferView else { return } + +if #available(iOS 16.0, *) { + guard let captureSession = captureSession else { return } + if captureSession.isMultitaskingCameraAccessSupported { + let pipVideoCallViewController = AVPictureInPictureVideoCallViewController() + pipVideoCallViewController.preferredContentSize = CGSize(width: 1080, height: 1920) + pipVideoCallViewController.view.addSubview(sampleBufferView) + + let source = AVPictureInPictureController.ContentSource( + activeVideoCallSourceView: sampleBufferView, + contentViewController: pipVideoCallViewController) + + pipController = AVPictureInPictureController(contentSource: source) + pipController.canStartPictureInPictureAutomaticallyFromInline = true + pipController.delegate = self + await self.fcsdkCallService.fcsdkCall?.call?.setPipController(pipController) + } else { + //If we are iOS 16 and we are not an m chip + await aChipLogic() + } + } else { + await aChipLogic() + } +} + +func aChipLogic() async { + let layer = sampleBufferView.layer as? AVSampleBufferDisplayLayer + guard let sourceLayer = layer else { return } + + let source = AVPictureInPictureController.ContentSource(sampleBufferDisplayLayer: sourceLayer, playbackDelegate: self) + + pipController = AVPictureInPictureController(contentSource: source) + pipController.canStartPictureInPictureAutomaticallyFromInline = true + pipController.delegate = self + await self.fcsdkCallService.fcsdkCall?.call?.setPipController(pipController) +} + +``` ## Available FCSDK Objects diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md index 946af29..1462f01 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md @@ -18,11 +18,43 @@ Please see for an explanation of how to use ACBClientCall @objc final public var remoteView: UIView? /// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications + @available(*, deprecated, message: "use sampleBufferView() async -> UIView? instead, This method will be removed in future versions of FCSDKiOS") @objc final public func remoteBufferView() async -> UIView? + /// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications + @MainActor @objc final public func sampleBufferView() async -> UIView? + /// The client should always remove the `remoteBufferView()` when finished using the bufferView. + @available(*, deprecated, message: "use removeSampleView() async instead, This method will be removed in future versions of FCSDKiOS") @objc final public func removeBufferView() async + /// The client should always remove the `remoteBufferView()` when finished using the bufferView. + @objc final public func removeSampleView() async + + /// The call's ``UIView`` used for displaying local buffer views intended for use in picture in picture/Virtual Background enabled applications + @MainActor @objc final public func previewBufferView() async -> UIView? + + /// The client should always remove the `removePreviewView()` when finished using the previewView. + @objc final public func removePreviewView() async + + /// The AVCaptureSession that the PreviewBufferView provides + @objc final public func captureSession() async -> AVCaptureSession? + + /// Feeds the AVPictureInPictureController into FCSDKiOS in order to receive the controller's delegate method notifications. + @MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async + + /// Feeds the UIImage from the consuming app into the SDK for Video Processing + @objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async + + /// Removes the Background Image from the Video Processsing flow + @objc final public func removeBackgroundImage() async + + /// An Enum that tells FCSDKiOS what mode the Virtual Background should be + @objc public enum VirtualBackgroundMode: Int, Sendable, Equatable, Hashable, RawRepresentable { + case blur + case image + } + /// A boolean value used to tell FCSDK if the client should have remote video when creating the call @objc final public var hasRemoteVideo: Bool { get } diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/Constants.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/Constants.md index d59b45d..2b3d1ff 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/Constants.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/Constants.md @@ -10,6 +10,8 @@ Constant things useful for FCSDK. LoggingLevel is especially helpful for setting /// The Version of the SDK @objc public static let SDK_VERSION_NUMBER: String + /// WebSocket connection timeout + @objc public static var WEBSOCKET_CONNECTION_TIMEOUT: Float override dynamic public init() } diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md new file mode 100644 index 0000000..48c69ca --- /dev/null +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md @@ -0,0 +1,83 @@ +# Version 4.2.0 + +This article describes changes in version 4.2.0 of FCSDKiOS + +## Overview +Version 4.2.0 reaches a new milestone in terms of Video Calls. We now support Picture in Picture and Virtual Background. The following Topics are the changes that we have made. + +### Preview Buffer View +This method is designed to create a metal rendered `UIView` for your local video during a video call. This method must be used if you desire to use our **Virtual Background** feature. +When a call is finished the **SDK** consumer is required to clean up the call by calling the `removePreviewView` method. +```swift +@available(iOS 15, *) +@MainActor @objc final public func previewBufferView() async -> UIView? +``` + +```swift +@available(iOS 15, *) +@objc final public func removePreviewView() async +``` + +### Sample Buffer View +This method is designed to create a metal rendered `UIView` for your remote video during a video call. This method must be used if you desire to use our **Picture in Picture** feature. +When a call is finished the **SDK** consumer is required to clean up the call by calling the `removeBufferView` method. +```swift +@available(iOS 15, *) +@MainActor @objc final public func sampleBufferView() async -> UIView? +``` + +```swift +@available(iOS 15, *) +@objc final public func removeSampleView() async +``` + +### AVCaptureSession +In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your Device is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. +```swift +@available(iOS 15, *) +@objc final public func captureSession() async -> AVCaptureSession? +``` + +An example in checking for the method is as follows: +```swift +if captureSession.isMultitaskingCameraAccessSupported { +// Use AVPictureInPictureVideoCallViewController +} else { +// Use AVPictureInPictureController +} +``` + +### Set PiP Controller +Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture. +```swift +@available(iOS 15, *) +@MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async +``` + +### Feed Background Image +In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. This will save your application from excessive memory and cpu usage. If you fail to feed the SDK a smaller **UIImage** we will automatically adjust the size to an acceptable limit. If you wish to blur the background instead of using an image, you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. + +```swift +@available(iOS 15, *) +@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async +``` + +```swift +@available(iOS 15, *) +@objc final public func removeBackgroundImage() async +``` + +### Virtual Background Mode +When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur` +```swift +@objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable { +case blur +case image +} +``` + +### WebSocket Connection Timeout +We have opened up the ability to set the WebSocket connection timeout on the **Constants** object. +```swift +@objc public static var WEBSOCKET_CONNECTION_TIMEOUT: Float +``` diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md new file mode 100644 index 0000000..be15dee --- /dev/null +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md @@ -0,0 +1,67 @@ +# VirtualBackground + +FCSDKiOS offers virtual background support. In order to use this feature, you will need to be able to provide a UIImage to FCSDKiOS. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. If the Image is too large, FCSDK will size the image to an acceptable size, but in practice it is encouraged for the SDK consumer to handle this to prevent any undefined behavior. Typically you will want to take the resoltion into consideration that the image will be displayed onto.(i.e. the callees video screen) + +## Overview + +The Virtual Background feature has been robustly built natively for your performance. Below we describe how to set up Virtual Background. + +## Feed Background Image +In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. This will save your application from excessive memory and cpu usage. If you fail to feed the SDK a smaller **UIImage** we will automatically adjust the size to an acceptable limit. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. + +```swift +@available(iOS 15, *) +@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async +``` + +```swift +@available(iOS 15, *) +@objc final public func removeBackgroundImage() async +``` + +## Virtual Background Mode +When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur` +```swift +@objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable { +case blur +case image +} +``` + + +## Available FCSDK Objects + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From e770d379b208a60a97907350c68ff004fe328f21 Mon Sep 17 00:00:00 2001 From: Cole Moore Date: Mon, 20 Mar 2023 16:14:19 +0300 Subject: [PATCH 2/3] Updated docs and added podspec tempate for CI/CD --- FCSDKiOS/FCSDKiOS_template.podspec | 15 +++++++++++++++ .../SwiftFCSDKiOS.docc/PictureinPicture.md | 6 +++--- .../SwiftFCSDKiOS.docc/Version-4.2.0.md | 6 +++--- .../SwiftFCSDKiOS.docc/VirtualBackground.md | 4 ++-- 4 files changed, 23 insertions(+), 8 deletions(-) create mode 100644 FCSDKiOS/FCSDKiOS_template.podspec diff --git a/FCSDKiOS/FCSDKiOS_template.podspec b/FCSDKiOS/FCSDKiOS_template.podspec new file mode 100644 index 0000000..b1dc0e3 --- /dev/null +++ b/FCSDKiOS/FCSDKiOS_template.podspec @@ -0,0 +1,15 @@ +Pod::Spec.new do |s| +s.name = 'FCSDKiOS' +s.version = 'CHANGE_ME' +s.summary = 'FCSDKiOS XCFramework' +s.homepage = 'https://github.com/cbajapan/swift-fcsdk-ios' + +s.author = { 'Name' => 'Communication Business Avenue, Inc.' } +s.license = { :type => 'Commercial', :text => 'Copyright Communication Business Avenue, Inc. Use of this software is subject to the terms and conditions located at https://github.com/cbajapan/swift-fcsdk-ios/blob/main/License.txt'} + +s.source = { :http => 'https://swift-sdk.s3.us-east-2.amazonaws.com/client_sdk/FCSDKiOS-CHANGE_ME.xcframework.zip' } + +s.platforms = { :ios => "13.0" } + +s.vendored_frameworks = 'FCSDKiOS.xcframework' +end diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md index b4b8d99..06dbd1f 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md @@ -44,9 +44,9 @@ var pipController: AVPictureInPictureController! func showPip(show: Bool) async { if show { if AVPictureInPictureController.isPictureInPictureSupported() { - pipController.startPictureInPicture() + pipController.startPictureInPicture() } else { - self.logger.info("PIP not Supported") + self.logger.info("PIP not Supported") } } else { pipController.stopPictureInPicture() @@ -76,7 +76,7 @@ if #available(iOS 16.0, *) { await aChipLogic() } } else { - await aChipLogic() + await aChipLogic() } } diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md index 48c69ca..8ce83d6 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md @@ -32,7 +32,7 @@ When a call is finished the **SDK** consumer is required to clean up the call by ``` ### AVCaptureSession -In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your Device is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. +In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your devices are supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. ```swift @available(iOS 15, *) @objc final public func captureSession() async -> AVCaptureSession? @@ -41,9 +41,9 @@ In order to check if your devices support Multitasking Camera Access, when a **P An example in checking for the method is as follows: ```swift if captureSession.isMultitaskingCameraAccessSupported { -// Use AVPictureInPictureVideoCallViewController + // Use AVPictureInPictureVideoCallViewController } else { -// Use AVPictureInPictureController + // Use AVPictureInPictureController } ``` diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md index be15dee..07f569e 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md @@ -1,13 +1,13 @@ # VirtualBackground -FCSDKiOS offers virtual background support. In order to use this feature, you will need to be able to provide a UIImage to FCSDKiOS. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. If the Image is too large, FCSDK will size the image to an acceptable size, but in practice it is encouraged for the SDK consumer to handle this to prevent any undefined behavior. Typically you will want to take the resoltion into consideration that the image will be displayed onto.(i.e. the callees video screen) +FCSDKiOS offers virtual background support. In order to use this feature, you will need to be able to provide a UIImage to FCSDKiOS. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. It is encouraged for the SDK consumer to handle feeding a *UIImage* that has been appropriately sized in order to prevent any undefined behavior. Typically you will want to take the resoltion into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropiate image size then FCSDKiOS will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. ## Overview The Virtual Background feature has been robustly built natively for your performance. Below we describe how to set up Virtual Background. ## Feed Background Image -In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. This will save your application from excessive memory and cpu usage. If you fail to feed the SDK a smaller **UIImage** we will automatically adjust the size to an acceptable limit. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. +In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. ```swift @available(iOS 15, *) From 97ed5de5b5fbee9eafa1343b39cf3b58ce41f7d1 Mon Sep 17 00:00:00 2001 From: Cole Moore Date: Wed, 22 Mar 2023 17:09:53 +0300 Subject: [PATCH 3/3] Documentation updates --- .../SwiftFCSDKiOS.docc/PictureinPicture.md | 16 +++++------ .../Symbols/ACBClientCall.md | 14 ++-------- .../SwiftFCSDKiOS.docc/Version-4.2.0.md | 28 +++++++++---------- .../SwiftFCSDKiOS.docc/VirtualBackground.md | 12 ++++---- 4 files changed, 31 insertions(+), 39 deletions(-) diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md index 06dbd1f..378f25a 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/PictureinPicture.md @@ -6,13 +6,13 @@ FCSDKiOS offers Picture in Picture support. In order to use this feature you wil The Picture in Picture feature is set up in the consumer of the **SDK**, however **FCSDKiOS** gives you all of the needed pieces in order to build a PiP app. 2 things are required to use Picture in Picture mode: -1. Your application must be using **FCSDKiOS**'s Preview Buffer View -2. Your application must be using **FCSDKiOS**'s Sample Buffer View +1. Your application must be using **FCSDKiOS**'s Local Buffer View +2. Your application must be using **FCSDKiOS**'s Remote Buffer View If you have conformed to these 2 **UIView**s please see how to set up Picture in Picture below. ### AVCaptureSession -In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your Devices is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise, you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. +In order to check if your devices support Multitasking Camera Access, when a **LocalBufferView** is created we can get the **AVCaptureSession** from the **LocalBufferView**. If your devices are supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise, you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **LocalBufferView** for your Local Video UIView. ```swift @available(iOS 15, *) @objc final public func captureSession() async -> AVCaptureSession? @@ -28,7 +28,7 @@ An example in checking for the method is as follows: ``` ### Set PiP Controller -Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture. +Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **RemoteBufferView** accordingly when the app activates Picture in Picture. ```swift @available(iOS 15, *) @MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async @@ -54,17 +54,17 @@ if show { } func setUpPip() async { -guard let sampleBufferView = sampleBufferView else { return } +guard let remoteBufferView = remoteBufferView else { return } if #available(iOS 16.0, *) { guard let captureSession = captureSession else { return } if captureSession.isMultitaskingCameraAccessSupported { let pipVideoCallViewController = AVPictureInPictureVideoCallViewController() pipVideoCallViewController.preferredContentSize = CGSize(width: 1080, height: 1920) - pipVideoCallViewController.view.addSubview(sampleBufferView) + pipVideoCallViewController.view.addSubview(remoteBufferView) let source = AVPictureInPictureController.ContentSource( - activeVideoCallSourceView: sampleBufferView, + activeVideoCallSourceView: remoteBufferView, contentViewController: pipVideoCallViewController) pipController = AVPictureInPictureController(contentSource: source) @@ -81,7 +81,7 @@ if #available(iOS 16.0, *) { } func aChipLogic() async { - let layer = sampleBufferView.layer as? AVSampleBufferDisplayLayer + let layer = remoteBufferView.layer as? AVSampleBufferDisplayLayer guard let sourceLayer = layer else { return } let source = AVPictureInPictureController.ContentSource(sampleBufferDisplayLayer: sourceLayer, playbackDelegate: self) diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md index 1462f01..74f24b7 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Symbols/ACBClientCall.md @@ -18,24 +18,16 @@ Please see for an explanation of how to use ACBClientCall @objc final public var remoteView: UIView? /// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications - @available(*, deprecated, message: "use sampleBufferView() async -> UIView? instead, This method will be removed in future versions of FCSDKiOS") @objc final public func remoteBufferView() async -> UIView? - /// The call's ``UIView`` used for displaying remote buffer views intended for use in picture in picture enabled applications - @MainActor @objc final public func sampleBufferView() async -> UIView? - /// The client should always remove the `remoteBufferView()` when finished using the bufferView. - @available(*, deprecated, message: "use removeSampleView() async instead, This method will be removed in future versions of FCSDKiOS") @objc final public func removeBufferView() async - /// The client should always remove the `remoteBufferView()` when finished using the bufferView. - @objc final public func removeSampleView() async - /// The call's ``UIView`` used for displaying local buffer views intended for use in picture in picture/Virtual Background enabled applications - @MainActor @objc final public func previewBufferView() async -> UIView? + @MainActor @objc final public func localBufferView() async -> UIView? /// The client should always remove the `removePreviewView()` when finished using the previewView. - @objc final public func removePreviewView() async + @objc final public func removeLocalBufferView() async /// The AVCaptureSession that the PreviewBufferView provides @objc final public func captureSession() async -> AVCaptureSession? @@ -44,7 +36,7 @@ Please see for an explanation of how to use ACBClientCall @MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async /// Feeds the UIImage from the consuming app into the SDK for Video Processing - @objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async + @objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async /// Removes the Background Image from the Video Processsing flow @objc final public func removeBackgroundImage() async diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md index 8ce83d6..9a5c9cc 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/Version-4.2.0.md @@ -5,34 +5,34 @@ This article describes changes in version 4.2.0 of FCSDKiOS ## Overview Version 4.2.0 reaches a new milestone in terms of Video Calls. We now support Picture in Picture and Virtual Background. The following Topics are the changes that we have made. -### Preview Buffer View +### Local Buffer View This method is designed to create a metal rendered `UIView` for your local video during a video call. This method must be used if you desire to use our **Virtual Background** feature. -When a call is finished the **SDK** consumer is required to clean up the call by calling the `removePreviewView` method. +When a call is finished the **SDK** consumer is required to clean up the call by calling the `removeLocalBufferView` method. ```swift @available(iOS 15, *) -@MainActor @objc final public func previewBufferView() async -> UIView? +@MainActor @objc final public func localBufferView() async -> UIView? ``` ```swift @available(iOS 15, *) -@objc final public func removePreviewView() async +@objc final public func removeLocalBufferView() async ``` -### Sample Buffer View +### Remote Buffer View This method is designed to create a metal rendered `UIView` for your remote video during a video call. This method must be used if you desire to use our **Picture in Picture** feature. When a call is finished the **SDK** consumer is required to clean up the call by calling the `removeBufferView` method. ```swift @available(iOS 15, *) -@MainActor @objc final public func sampleBufferView() async -> UIView? +@MainActor @objc final public func remoteBufferView() async -> UIView? ``` ```swift @available(iOS 15, *) -@objc final public func removeSampleView() async +@objc final public func removeBufferView() async ``` ### AVCaptureSession -In order to check if your devices support Multitasking Camera Access, when a **PreviewBufferView** is created we can get the **AVCaptureSession** from the **PreviewBufferView**. If your devices are supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **PreviewBufferView** for your Local Video UIView. +In order to check if your devices support Multitasking Camera Access, when a **LocalBufferView** is created we can get the **AVCaptureSession** from the **LocalBufferView**. If your device is supported by Apple, you can use the new **AVPictureInPictureVideoCallViewController** provided by Apple. Otherwise you will need to continue using **AVPictureInPictureController** and request permission from Apple for the proper entitlement. This method only works if you are using **LocalBufferView** for your Local Video UIView. ```swift @available(iOS 15, *) @objc final public func captureSession() async -> AVCaptureSession? @@ -48,18 +48,18 @@ if captureSession.isMultitaskingCameraAccessSupported { ``` ### Set PiP Controller -Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **SampleBufferView** accordingly when tha app activates Picture in Picture. +Each **ACBClientCall** object has a `setPipController` method that is required for Picture in Picture. This method informs the **SDK** who the **AVPictureInPictureController** is and allows us to receive its delegate methods. This is required for us to handle the **RemoteBufferView** accordingly when tha app activates Picture in Picture. ```swift @available(iOS 15, *) @MainActor @objc final public func setPipController(_ controller: AVPictureInPictureController) async ``` -### Feed Background Image -In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. This will save your application from excessive memory and cpu usage. If you fail to feed the SDK a smaller **UIImage** we will automatically adjust the size to an acceptable limit. If you wish to blur the background instead of using an image, you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. +### Feeding a Background Image +In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. Typically you will want to take the resolution into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropriate image size then **FCSDKiOS** will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum without including a **UIImage** in the image property parameter. ```swift @available(iOS 15, *) -@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async +@objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async ``` ```swift @@ -71,8 +71,8 @@ In order to use the **Virtual Background** feature the **SDK** consumer is requi When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur` ```swift @objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable { -case blur -case image + case blur + case image } ``` diff --git a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md index 07f569e..343c878 100644 --- a/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md +++ b/Sources/SwiftFCSDKiOS/SwiftFCSDKiOS.docc/VirtualBackground.md @@ -1,17 +1,17 @@ # VirtualBackground -FCSDKiOS offers virtual background support. In order to use this feature, you will need to be able to provide a UIImage to FCSDKiOS. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. It is encouraged for the SDK consumer to handle feeding a *UIImage* that has been appropriately sized in order to prevent any undefined behavior. Typically you will want to take the resoltion into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropiate image size then FCSDKiOS will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. +FCSDKiOS offers virtual background support. This feature offers an option to blur your camera capture's background or simply provide a virtual background image. ## Overview The Virtual Background feature has been robustly built natively for your performance. Below we describe how to set up Virtual Background. -## Feed Background Image -In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum. You are still required to feed an image. Virtual Backgrounds can also be blurred. If you have set a virtual background and only want to blur your real background, you must first remove the virtual background using the method `removeBackgroundImage()`. +## Feeding a Background Image +In order to use the **Virtual Background** feature the **SDK** consumer is required to feed the **SDK** a UIImage that the **SDK** can use as the virtual background. It is **STRONGLY** encouraged for performance reasons to feed an image that is as small as possible. The SDK consumer is responsible for setting up any UI that stores and presents images for selection. Typically you will want to take the resolution into consideration that the image will be displayed onto.(i.e. the callees video screen). If the consumer does not handle feeding FCSDKiOS an appropriate image size then **FCSDKiOS** will automatically resize the background image to an acceptable size of *CGSize(width: 1280, height: 720)*. If you wish to blur the background instead of using an image you can select **.blur** from the **VirtualBackgroundMode** enum without including a **UIImage** in the image property parameter. ```swift @available(iOS 15, *) -@objc final public func feedBackgroundImage(_ image: UIImage, mode: FCSDKiOS.VirtualBackgroundMode = .image) async +@objc final public func feedBackgroundImage(_ image: UIImage? = nil, mode: FCSDKiOS.VirtualBackgroundMode = .image) async ``` ```swift @@ -23,8 +23,8 @@ In order to use the **Virtual Background** feature the **SDK** consumer is requi When feeding your background image you can set what mode it should be. There are 2 modes `.image` or `.blur` ```swift @objc public enum VirtualBackgroundMode : Int, Sendable, Equatable, Hashable, RawRepresentable { -case blur -case image + case blur + case image } ```