Screen sharing

During Video Calling sessions, hosts use the screen sharing feature in the Agora Video SDK to share their screen content with other users or viewers in the form of a video stream. Screen sharing is typically used in the following scenarios:

Scenario	Description
Online education	Teachers share their slides, software, or other teaching materials with students for classroom demonstrations.
Game live broadcast	Hosts share their game footage with the audience.
Interactive live broadcast	Anchors share their screens and interact with the audience.
Video conferencing	Meeting participants share the screen to show a presentation or documents.
Remote control	A controlled terminal displays its desktop on the master terminal.

Agora screen sharing offers the following advantages:

• Ultra HD quality experience: Supports Ultra HD video (4K resolution, 60 FPS frame rate), giving users a smoother, high-definition, ultimate picture experience.
• Multi-app support: Compatible with many mainstream apps such as WPS Office, Microsoft Office Power Point, Visual Studio Code, Adobe Photoshop, Windows Media Player, and Scratch. This makes it convenient for users to directly share specific apps.
• Multi-device support: Supports multiple devices sharing at the same time. Screen sharing is compatible with Windows 8 systems, devices without independent graphics cards, dual graphics card devices, and external screen devices.
• Multi-platform adaptation: Supports iOS, Android, macOS, Windows, Web, Unity, Flutter, React Native, Unreal Engine, and other platforms.
• High security: Supports sharing only a single app or part of the screen. Also supports blocking specified app windows, effectively ensuring user information security.

This page shows you how to implement screen sharing in your app.

The screen sharing feature provides the following screen sharing modes for use in various scenarios:

• Share the entire screen: Share your entire screen, including all the information on the screen. This feature supports collecting and sharing information from two screens at the same time.
• Share an app window: If you don't want to share the entire screen with other users, you can share only the area within an app window.
• Share a designated screen area: If you only want to share a portion of the screen or app window, you can set a sharing area when starting screen sharing.

Screen sharing modes are available on different platforms as follows:

• Desktop (Windows and macOS): Supports all screen sharing features listed above.
• Mobile (Android and iOS): Only supports sharing the entire screen.

• Ensure that you have implemented the SDK quickstart in your project.

Since Apple does not support capturing the screen in the main process of the app, you create a separate extension for the screen sharing stream. You use the iOS native ReplayKit framework in the extension to record the screen, and then send the screen sharing stream to the main process.

Take the following steps to set up your screen sharing project:

1. Go to your project folder and open the .xcodeproj file with Xcode.

2. Navigate to File > New > Target..., select Broadcast Upload Extension in the popup window, and then click Next:

   

3. Fill in the Product Name and other information in the popup window, uncheck Include UI Extension, and click Finish. Xcode automatically creates a folder for the extension that contains the SampleHandler.swift files.

4. Under Target, select the newly created extension. Click General and set the iOS version to 12.0 or later under Deployment Info.

5. Modify project settings to implement the screen sharing code logic, and choose one of the following three methods according to your business needs:

   • If you only need to use the functionality in the AgoraReplayKitExtension.xcframework dynamic library provided by Agora, select Target as the extension you just created, and change the Value in Info corresponding to NSExtension > NSExtensionPrincipalClass from SampleHandler to AgoraReplayKitHandler:

     

   • If you need to implement some additional business logic, you can inherit the AgoraReplayKitHandler class. Modify the SampleHandler.swift file as follows:

     import ReplayKit
     

     import AgoraReplayKitExtension
     

     
     

     class SampleHandler: AgoraReplayKitHandler {
     

     override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
     

     // User request to start broadcasting
     

     super.broadcastStarted(withSetupInfo: setupInfo)
     

     }
     

     }
     

   • If you want to use the iOS native SampleHandler without inheriting the AgoraReplayKitHandler class, refer to the following code to modify the SampleHandler.swift file:

     import ReplayKit
     

     import AgoraReplayKitExtension
     

     
     

     class SampleHandler: RPBroadcastSampleHandler, AgoraReplayKitExtDelegate {
     

     override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
     

     // User request to start broadcasting
     

     AgoraReplayKitExt.shareInstance().start(self)
     

     }
     

     override func broadcastPaused() {
     

     // User request to pause the broadcast
     

     AgoraReplayKitExt.shareInstance().pause()
     

     }
     

     
     

     override func broadcastResumed() {
     

     // User request to resume broadcasting
     

     AgoraReplayKitExt.shareInstance().resume()
     

     }
     

     
     

     override func broadcastFinished() {
     

     // User request to stop broadcasting
     

     AgoraReplayKitExt.shareInstance().stop()
     

     }
     

     
     

     override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
     

     AgoraReplayKitExt.shareInstance().push(sampleBuffer, with: sampleBufferType)
     

     }
     

     
     

     func broadcastFinished(_ broadcast: AgoraReplayKitExt, reason: AgoraReplayKitExtReason) {
     

     var tip = ""
     

     switch reason {
     

     case AgoraReplayKitExtReasonInitiativeStop:
     

     tip = "AgoraReplayKitExtReasonInitiativeStop"
     

     
     

     case AgoraReplayKitExtReasonConnectFail:
     

     tip = "AgoraReplayKitExReasonConnectFail"
     

     
     

     case AgoraReplayKitExtReasonDisconnect:
     

     tip = "AgoraReplayKitExReasonDisconnect"
     

     
     

     default: break
     

     }
     

     
     

     let error = NSError(domain: NSCocoaErrorDomain,
     

     code: 0,
     

     userInfo: [NSLocalizedFailureReasonErrorKey: tip])
     

     self .finishBroadcastWithError(error as Error)
     

     }
     

     }
     

When integrating the Agora Video SDK through CocoaPods, you need to add the following content in the Podfile file to specify the integration of the screen sharing dynamic library AgoraReplayKitExtension.xcframework. Refer to the following sample code:

platform :ios, '9.0'

target 'Your App' do

# Integration of only base libraries and screen sharing dynamic libraries

# Please replace x.y.z in the following code with the specific SDK version number.

# You can get the latest version number from the release notes.

pod 'AgoraRtcEngine_iOS', 'x.y.z', :subspecs => ['RtcBasic', 'ReplayKit']

end

The screen sharing dynamic library encapsulates screen recording with Apple ReplayKit. Use the SDK capture function to get system recording data and send it to other users in the channel.

This section introduces how to implement screen sharing in your project. The basic API call sequence is shown in the figure below:

Depending on the actual business scenario, you can choose either of the following methods to call the API to implement screen sharing:

• Call startScreenCapture before joining the channel, then call joinChannelByToken [2/4] to join the channel and set publishScreenCaptureVideo to true to start screen sharing.

• Call startScreenCapture after joining the channel, then call updateChannelWithMediaOptions to update the channel media options and set publishScreenCaptureVideo to true to start screen sharing.

Call the setAudioScenario method and set the audio scenario to AgoraAudioScenarioGameStreaming (high-quality scenario) to improve the success rate of capturing system audio during screen sharing.

1. Call startScreenCapture to start screen capture and set the parameters according to your application scenario:

   • captureVideo: Whether to capture system video during screen sharing.
   • captureAudio: Whether to capture system audio during screen sharing.
   • captureSignalVolume: The volume of the captured system audio.
   • dimensions: Resolution of video encoding.
   • frameRate: Video encoding frame rate (FPS).
   • bitrate: Video encoding bitrate (Kbps).
   • contentHint: Content type of screen sharing video.
   // Set parameters for screen capture
   

   private lazy var screenParams: AgoraScreenCaptureParameters2 = {
   

   let params = AgoraScreenCaptureParameters2()
   

   params.captureVideo = true
   

   params.captureAudio = true
   

   
   

   let audioParams = AgoraScreenAudioParameters()
   

   // Set the captured system volume
   

   audioParams.captureSignalVolume = 50
   

   params.audioParams = audioParams
   

   
   

   let videoParams = AgoraScreenVideoParameters()
   

   // Set the resolution of the shared screen
   

   videoParams.dimensions = screenShareVideoDimension()
   

   // Set the video encoding frame rate
   

   videoParams.frameRate = .FPS15
   

   // Set the video encoding bitrate
   

   videoParams.bitrate = AgoraVideoBitrateStandard
   

   params.videoParams = videoParams
   

   
   

   return params
   

   }()
   

   
   

   // Enable screen capture
   

   agoraKit.startScreenCapture(screenParams)
   

2. Combine this with a manual action by the user to make the app turn on screen sharing. There are two ways to do this:

   1. Prompt the user to long click the screen recording button in the Control Center on iOS and choose to use the extension you created to start recording.

   2. Use Apple's new RPSystemBroadcastPickerView in iOS 12.0 to make the Enable Screen Sharing button pop up on the app interface, prompting the user to click the button to start recording. The sample code is as follows:

      let frame = CGRect(x: 0, y:0, width: 60, height: 60)
      

      systemBroadcastPicker = RPSystemBroadcastPickerView(frame: frame)
      

      systemBroadcastPicker?.showsMicrophoneButton = false
      

      systemBroadcastPicker?.autoresizingMask = [.flexibleTopMargin, .flexibleRightMargin]
      

      // Get the Bundle Identifier of the main application
      

      let bundleId = Bundle.main.bundleIdentifier ?? ""
      

      // Set the Extension Name
      

      systemBroadcastPicker?.preferredExtension = "\(bundleId).Agora-ScreenShare-Extension"
      
      caution

      RPSystemBroadcastPickerView has some usage limitations and may become invalid in subsequent versions of iOS.

After joining the channel, call updateChannelWithMediaOptions to publish the captured screen sharing video stream as follows:

// Define the behavior of rtcEngine when performing screen video capture (state is capturing, sourceType is screen)

func rtcEngine(_ engine: AgoraRtcEngineKit, localVideoStateChangedOf state: AgoraVideoLocalState, error: AgoraLocalVideoStreamError, sourceType: AgoraVideoSourceType) {

switch (state, sourceType) {

case (.capturing, .screen):

// Publish screen captured videos in the channel

option.publishScreenCaptureVideo = true

// Publish screen captured videos in the channel

option.publishScreenCaptureAudio = true

// Avoid release of camera-captured video

option.publishCameraTrack = false

// Update the channel settings with the options set above

agoraKit.updateChannel(with: option)

default: break

}

}

Call the setScreenCaptureScenario method to set the screen sharing scenario. Choose the scenarioType that best fits your application from the following:

• AgoraScreenScenarioDocument: Document scene
• AgoraScreenScenarioGaming: Game scene
• AgoraScreenScenarioVideo: Video scene
• AgoraScreenScenarioRDC: Remote control scene

agoraKit.setScreenCaptureScenario(.video)

If you want to update the screen sharing parameters, such as the video encoding resolution, frame rate, or bitrate, call updateScreenCapture to modify the parameter values.

agoraKit.updateScreenCapture(screenParams)

Call stopScreenCapture to stop screen sharing within the channel.

agoraKit.stopScreenCapture()

// Stop sharing captured video in the channel

option.publishScreenCaptureVideo = false

// Stop sharing captured audio in the channel

option.publishScreenCaptureAudio = false

// Publish camera-captured video in the channel

option.publishCameraTrack = true

agoraKit.updateChannel(with: option)

Be aware of the following limitations:

• After turning on screen sharing, Agora uses the resolution of the screen sharing video stream as the billing standard. Please see Pricing for details. The default resolution is 1280 × 720, but you can adjust it according to your business needs.
• Due to system limitations, screen sharing is only supported on iOS 12.0 and above.
• This feature requires a high level of device performance. Agora recommends that you use an iPhone X or above.
• After the user turns on screen sharing on an iOS device, the audio routing automatically switches to the earpiece due to system limitations.
  • If you are using call volume, you can manually switch the audio routing. For example, you can switch the audio routing to speaker as desired.
  • If you are using media volume, you cannot manually switch audio routing due to system limitations.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Agora provides open source sample projects on GitHub for your reference:

• Agora-ScreenShare-Extension: Implements the extension process.
• ScreenShare: Screen sharing in AgoraRtcEngineKit.

• startScreenCapture
• updateScreenCapture
• stopScreenCapture