Screen sharing

During Broadcast Streaming sessions, hosts use the screen sharing feature in the Agora Video SDK to share their screen content with other users or viewers in the form of a video stream. Screen sharing is typically used in the following scenarios:

Scenario	Description
Online education	Teachers share their slides, software, or other teaching materials with students for classroom demonstrations.
Game live broadcast	Hosts share their game footage with the audience.
Interactive live broadcast	Anchors share their screens and interact with the audience.
Video conferencing	Meeting participants share the screen to show a presentation or documents.
Remote control	A controlled terminal displays its desktop on the master terminal.

Agora screen sharing offers the following advantages:

• Ultra HD quality experience: Supports Ultra HD video (4K resolution, 60 FPS frame rate), giving users a smoother, high-definition, ultimate picture experience.
• Multi-app support: Compatible with many mainstream apps such as WPS Office, Microsoft Office Power Point, Visual Studio Code, Adobe Photoshop, Windows Media Player, and Scratch. This makes it convenient for users to directly share specific apps.
• Multi-device support: Supports multiple devices sharing at the same time. Screen sharing is compatible with Windows 8 systems, devices without independent graphics cards, dual graphics card devices, and external screen devices.
• Multi-platform adaptation: Supports iOS, Android, macOS, Windows, Web, Unity, Flutter, React Native, Unreal Engine, and other platforms.
• High security: Supports sharing only a single app or part of the screen. Also supports blocking specified app windows, effectively ensuring user information security.

This page shows you how to implement screen sharing in your game.

The screen sharing feature provides the following screen sharing modes for use in various scenarios:

• Share the entire screen: Share your entire screen, including all the information on the screen. This feature supports collecting and sharing information from two screens at the same time.
• Share an app window: If you don't want to share the entire screen with other users, you can share only the area within an app window.
• Share a designated screen area: If you only want to share a portion of the screen or app window, you can set a sharing area when starting screen sharing.

Screen sharing modes are available on different platforms as follows:

• Desktop (Windows and macOS): Supports all screen sharing features listed above.
• Mobile (Android and iOS): Only supports sharing the entire screen.

• Ensure that you have implemented the SDK quickstart in your project.

This section introduces how to implement screen sharing in your project.

When enabling screen sharing on Android, call the StartScreenCapture method. Depending on the actual business scenario, choose either of the following methods to implement screen sharing:

• Call StartScreenCapture before joining the channel, then call JoinChannel[2/2] to join the channel and set publishScreenCaptureVideo to true to start screen sharing.

  // Call StartScreenCapture to enable screen sharing before joining a channel.
  

  ScreenCaptureParameters2 screenCaptureParameters2 = new ScreenCaptureParameters2();
  

  screenCaptureParameters2.captureVideo = true;
  

  screenCaptureParameters2.captureAudio = true;
  

  RtcEngine.StartScreenCapture(screenCaptureParameters2);
  

  // Call JoinChannel to join a channel.
  

  ChannelMediaOptions channelMediaOptions = new ChannelMediaOptions();
  

  channelMediaOptions.publishScreenCaptureVideo.SetValue(true);
  

  channelMediaOptions.publishScreenCaptureAudio.SetValue(true);
  

  channelMediaOptions.publishCameraTrack.SetValue(false);
  

  channelMediaOptions.publishMicrophoneTrack.SetValue(false);
  

  RtcEngine.JoinChannel("token", "channelId", uid, channelMediaOptions);
  

• Call StartScreenCapture after joining the channel, then call UpdateChannelMediaOptions to set publishScreenCaptureVideo to true to start screen sharing.

  // Call JoinChannel to join the channel.
  

  RtcEngine.JoinChannel(_token, _channelName);
  

  ScreenCaptureParameters2 screenCaptureParameters2 = new ScreenCaptureParameters2();
  

  screenCaptureParameters2.captureVideo = true;
  

  screenCaptureParameters2.captureAudio = true;
  

  // Enable screen sharing.
  

  RtcEngine.StartScreenCapture(screenCaptureParameters2);
  

  ChannelMediaOptions channelMediaOptions = new ChannelMediaOptions();
  

  channelMediaOptions.publishScreenCaptureVideo.SetValue(true);
  

  channelMediaOptions.publishScreenCaptureAudio.SetValue(true);
  

  channelMediaOptions.publishCameraTrack.SetValue(false);
  

  channelMediaOptions.publishMicrophoneTrack.SetValue(false);
  

  // Update channel media options.
  

  RtcEngine.UpdateChannelMediaOptions(channelMediaOptions);
  

Since Apple does not support capturing the screen in the main process of the app, you create a separate extension for the screen sharing stream. You use the iOS native ReplayKit framework in the extension to record the screen, and then add the screen sharing stream to the channel as a user.

Take the following steps to set up your screen sharing project:

1. Package the iOS project in Unity Editor and export the Xcode project.

2. Go to your project folder and open the unity-iphone/.xcodeproj folder with Xcode.

3. Create a Broadcast Upload Extension to enable the process of screen sharing:

   1. Navigate to File > New > Target..., select Broadcast Upload Extension in the popup window, and then click Next:

      

   2. Fill in the Product Name and other information in the popup window, uncheck Include UI Extension, and click Finish. Xcode automatically creates a folder for the extension that contains the SampleHandler.m file.

   3. Under Target, select the newly created extension. Click General and set the iOS version to 12.0 or later under Deployment Info.

   4. Modify project settings to implement the screen sharing code logic, and choose one of the following two methods according to your business needs:

      • If you only need to use the functionality in the AgoraReplayKitExtension.xcframework dynamic library provided by Agora, select Target as the extension you just created, and change the Value in Info corresponding to NSExtension > NSExtensionPrincipalClass from $(PRODUCT_MODULE_NAME}.SampleHandler to AgoraReplayKitHandler:

        

      • If you need to implement some additional business logic, refer to the following code to modify the SampleHandler.m file:

        #import "SampleHandler.h"
        

        #import "AgoraReplayKitExt.h"
        

        #import <sys/time.h>
        

        
        

        @interface SampleHandler ()<AgoraReplayKitExtDelegate>
        

        
        

        @end
        

        
        

        @implementation SampleHandler
        

        
        

        - (void)broadcastStartedWithSetupInfo:(NSDictionary<NSString *,NSObject *> *)setupInfo {
        

        [[AgoraReplayKitExt shareInstance] start:self];
        

        
        

        }
        

        
        

        - (void)broadcastPaused {
        

        NSLog(@"broadcastPaused");
        

        [[AgoraReplayKitExt shareInstance] pause];
        

        }
        

        
        

        - (void)broadcastResumed {
        

        NSLog(@"broadcastResumed");
        

        [[AgoraReplayKitExt shareInstance] resume];
        

        
        

        }
        

        
        

        - (void)broadcastFinished {
        

        NSLog(@"broadcastFinished");
        

        [[AgoraReplayKitExt shareInstance] stop];
        

        
        

        }
        

        
        

        - (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType {
        

        [[AgoraReplayKitExt shareInstance] pushSampleBuffer:sampleBuffer withType:sampleBufferType];
        

        }
        

        
        

        #pragma mark - AgoraReplayKitExtDelegate
        

        
        

        - (void)broadcastFinished:(AgoraReplayKitExt *_Nonnull)broadcast reason:(AgoraReplayKitExtReason)reason {
        

        switch (reason) {
        

        case AgoraReplayKitExtReasonInitiativeStop:
        

        {
        

        
        

        }
        

        break;
        

        case AgoraReplayKitExtReasonConnectFail:
        

        {
        

        
        

        }
        

        break;
        

        
        

        case AgoraReplayKitExtReasonDisconnect:
        

        {
        

        
        

        }
        

        break;
        

        default:
        

        break;
        

        }
        

        }
        

        
        

        @end
        

4. Select the extension you created in TARGETS and add all frameworks under the path Frameworks/Agora-RTC-Plugin/Agora-Unity-RTC-SDK/Plugins/iOS/ in General/Frameworks and Libraries.

5. To start screen sharing, call StartScreenCapture combined with the user's action. For example, prompt the user to press and hold the Screen Recording button in the Control Center on iOS, and select the extension you created to start recording.

Information

• Make sure your app and extension have the same TARGETS/Deployment/iOS version. The memory usage of Broadcast Upload Extension is limited to 50 MB.
• Make sure that the memory usage of Screen Sharing Extension does not exceed 50 MB.

Windows and macOS systems assign a unique Display ID to each screen and a unique Window ID to each window. Agora currently supports the following screen sharing options for macOS and Windows:

• Share a specific screen or a portion of a specified screen: Call GetScreenCaptureSources to get the Display ID and then call StartScreenCaptureByDisplayId to start screen sharing.
• Share a specific window or a portion of a specified window: Call GetScreenCaptureSources to get the Window ID and then call StartScreenCaptureByWindowId to start screen sharing.

Take the following steps to implement screen sharing on macOS and Windows:

1. Get a list of screens and windows that can be shared. Call GetScreenCaptureSources method to get the Display ID of the screen or the Window ID of the window to be shared:

   SIZE thumbSize = new SIZE(360,240);
   

   SIZE iconSize = new SIZE(360,240);
   

   ScreenCaptureSourceInfo[] screenCaptureSourceInfos = RtcEngine.GetScreenCaptureSources(thumbSize, iconSize, true);
   

2. Share the specified screen or window. Depending on the sharing object, call the StartScreenCaptureByDisplayId or StartScreenCaptureByWindowId method to start screen sharing:

   ScreenCaptureSourceInfo info = screenCaptureSourceInfos[0];
   

   if (info.type == ScreenCaptureSourceType.ScreenCaptureSourceType_Screen)
   

   {
   

   ulong displayId = info.sourceId;
   

   RtcEngine.StartScreenCaptureByDisplayId((uint)displayId, default(Rectangle),default(ScreenCaptureParameters));
   

   }
   

   else if (info.type == ScreenCaptureSourceType.ScreenCaptureSourceType_Window)
   

   {
   

   ulong windowId = info.sourceId;
   

   RtcEngine.StartScreenCaptureByWindowId(windowId, default(Rectangle),default(ScreenCaptureParameters));
   

   }
   

3. Join a channel and publish screen sharing stream.

   • To publish only tje screen sharing stream, add the following code to your project:

     ChannelMediaOptions options = new ChannelMediaOptions();
     

     options.publishCameraTrack.SetValue(false);
     

     options.publishScreenTrack.SetValue(true);
     

     
     

     #if UNITY_ANDROID || UNITY_IPHONE
     

     options.publishScreenCaptureAudio.SetValue(true);
     

     options.publishScreenCaptureVideo.SetValue(true);
     

     #endif
     

     RtcEngine.UpdateChannelMediaOptions(options);
     

   • To publish a screen sharing stream and the video stream captured by a local camera simultaneously, add the following code to your project:

     // Join a channel and push the camera stream
     

     RtcEngine.EnableAudio();
     

     RtcEngine.EnableVideo();
     

     RtcEngine.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
     

     
     

     ChannelMediaOptions options = new ChannelMediaOptions();
     

     options.autoSubscribeAudio.SetValue(true);
     

     options.autoSubscribeVideo.SetValue(true);
     

     
     

     options.publishCameraTrack.SetValue(true);
     

     options.publishScreenTrack.SetValue(false);
     

     options.enableAudioRecordingOrPlayout.SetValue(true);
     

     options.clientRoleType.SetValue(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
     

     RtcEngine.JoinChannel(_token, _channelName, this.Uid1, options);
     

     
     

     // Join a channel and push the screen sharing stream
     

     ChannelMediaOptions options = new ChannelMediaOptions();
     

     options.autoSubscribeAudio.SetValue(false);
     

     options.autoSubscribeVideo.SetValue(false);
     

     options.publishCameraTrack.SetValue(false);
     

     options.publishScreenTrack.SetValue(true);
     

     options.enableAudioRecordingOrPlayout.SetValue(false);
     

     #if UNITY_ANDROID || UNITY_IPHONE
     

     options.publishScreenCaptureAudio.SetValue(true);
     

     options.publishScreenCaptureVideo.SetValue(true);
     

     #endif
     

     options.clientRoleType.SetValue(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
     

     var ret = RtcEngine.JoinChannelEx(_token, new RtcConnection(_channelName, this.Uid2), options);
     

Be aware of the following limitations:

• The video unit price for a screen-sharing stream is based on the video resolution you set in ScreenCaptureParameters. If you do not pass dimensions in ScreenCaptureParameters, Agora bills you at the default resolution of 1920 x 1080 (2,073,600). See Pricing for details.
• Due to system limitations, screen sharing is only supported on iOS 12.0 and above.
• This feature requires a high level of device performance. Agora recommends that you use an iPhone X or above.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Agora provides an open-source Unity sample project on GitHub. Download and explore this project for a more detailed example.

• Common

  • EnableAudio
  • EnableVideo
  • SetClientRole [1/2]
  • UpdateChannelMediaOptions
  • ChannelMediaOptions
  • JoinChannel [2/2]
  • JoinChannelEx

• Android

  • StartScreenCapture [1/2]
  • StopScreenCapture [1/2]
  • JoinChannel [2/2]

• iOS

  • StartScreenCapture [1/2]
  • StopScreenCapture [1/2]
  • UpdateScreenCaptureParameters

• macOS, Windows

  • GetScreenCaptureSources
  • StartScreenCaptureByDisplayId
  • StartScreenCaptureByWindowId