BytePlus Effects

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. This guide is provided by BytePlus. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. By calling the setExtensionProperty or setExtensionPropertyWithVendor method of the Agora Video SDK v4.x and passing in the corresponding key and value, you can quickly integrate BytePlus's capabilities.

Taking setExtensionProperty as an example, the key is named after the BytePlus API, and the value wraps either certain or all of the parameters of that API in JSON. When you call setExtensionProperty and pass in the pair of key and value, it is equivalent to calling the corresponding BytePlus API. The case is the same for setExtensionPropertyWithVendor.

Note

Currently, the extension encapsulates only a part of the APIs of the BytePlus Effects SDK. For details, see the key-value overview.

The development environment has to meet the following requirements:

• Android Studio 4.1 or later.
• A physical device (not an emulator) running Android 5.0 or later.

The BytePlus Effects extension works together with the Video SDK v4.x. Refer to the following doc to integrate the SDK and start a basic video call:

• SDK quickstart

1. Download the Android package of BytePlus Effects.

2. Unzip the package, and save all .aar files to the /app/libs path of your project folder.

3. Contact Agora to get the resource package of the BytePlus extension. Save the files you need to the /app/src/main/assets path of the project folder. For details, see Resource package structure.

4. In the app/build.gradle file, add the following line in dependencies:

   
   

   _1

   implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

   
   

5. Import the required classes:

   
   

   _7

   import io.agora.rtc2.Constants;

   _7

   import io.agora.rtc2.IMediaExtensionObserver;

   _7

   import io.agora.rtc2.IRtcEngineEventHandler;

   _7

   import io.agora.rtc2.RtcEngine;

   _7

   import io.agora.rtc2.RtcEngineConfig;

   _7

   import io.agora.rtc2.video.VideoCanvas;

   _7

   import io.agora.rte.extension.bytedance.ExtensionManager;

   
   

This section describes the call sequence of using the extension. For a detailed parameter description, see the API reference.

When initializing RtcEngine, call enableExtension to enable the extension:

_7

RtcEngineConfig config = new RtcEngineConfig();

_7

// Listen to extension events to receive onEvent callbacks

_7

config.mExtensionObserver = this;

_7

config.addExtension("AgoraByteDanceExtension");

_7

mRtcEngine = RtcEngine.create(config);

_7

// Enable the plugin

_7

mRtcEngine.enableExtension("ByteDance", "Effect", enabled);

To initialize the extension, call setExtensionProperty, and pass in the corresponding keys and values:

1. Check the license Set the key as bef_effect_ai_check_license and the value as the path of the license file LicenseBag.bundle.

2. Initialize the effects Set the key as bef_effect_ai_init and the value as the path of the model file ModelResource.bundle and the device name.

   
   

   _39

   private void initExtension() {

   _39

   String[] resources = new String[] {"byte_dance/LicenseBag.bundle", "byte_dance/ModelResource.bundle"};

   _39

   loadBundle(resources, () -> {

   _39

   File destFile = getExternalFilesDir(null);

   _39

   _39

   // Get the path of the license file

   _39

   File licensePath = new File(

   _39

   destFile,

   _39

   "byte_dance/LicenseBag.bundle/" + io.agora.rte.extension.bytedance.example.Constants.mLicenseName);

   _39

   _39

   // Check the license

   _39

   try {

   _39

   JSONObject jsonObject = new JSONObject();

   _39

   jsonObject.put("licensePath", licensePath.getPath());

   _39

   setExtensionProperty("bef_effect_ai_check_license", jsonObject.toString());

   _39

   } catch (JSONException e) {

   _39

   Log.e(TAG, e.toString());

   _39

   }

   _39

   _39

   // Get the path of the algorithm model files

   _39

   File strModelDir = new File(destFile, "byte_dance/ModelResource.bundle");

   _39

   _39

   // Initialize the effects

   _39

   try {

   _39

   JSONObject jsonObject = new JSONObject();

   _39

   jsonObject.put("strModelDir", strModelDir.getPath());

   _39

   jsonObject.put("deviceName", "");

   _39

   setExtensionProperty("bef_effect_ai_init", jsonObject.toString());

   _39

   } catch (JSONException e) {

   _39

   Log.e(TAG, e.toString());

   _39

   }

   _39

   });

   _39

   }

   _39

   _39

   _39

   // Convenient for multiple calls to setExtensionProperty

   _39

   private void setExtensionProperty(String key, String property) {

   _39

   mRtcEngine.setExtensionProperty("ByteDance", "Effect", key, property);

   _39

   }

   
   

Call setExtensionProperty, and pass in the corresponding keys and values. You can implement the following functions:

• Set the orientation of the mobile phone
• Set overlay effects (face beautification, shape beautification, and beauty makeup) and their intensity
• Set stickers
• Set filters and their intensity

You can combine calls according to your needs. For the corresponding key and value, refer to the API reference. To use superimposed special effects in an environment that does not use stickers, initialize the composer first. To remove an effect, remove the composer node. See the key-value overview for details.

The complete sample project is available on GitHub:

Platform	Language	Sample project
Android	Java	ByteDance/android

Take the following steps to run the sample project:

1. Clone the repository using the following command:

   
   

   _1

   git clone https://github.com/AgoraIO-Community/AgoraMarketPlace.git

   
   

2. Open the the Agora Console > Cloud Market > BytePlus Effects page, click to download the Android extension package, and then click Contact Us to obtain an exclusive certificate file.

3. Unzip the downloaded Android extension package and save all .aar files to the project folder ByteDance/android/app/libs.

   

4. Click to download the material package of the extension and save all files to ByteDance/android/app/src/main/assets/Resource.

   

5. Open the sample project ByteDance/android in Android Studio.

6. Sync project with Gradle files.

7. Save your certificate file with the extension .licbag to the path ByteDance/android/app/src/main/assets/LicenseBag.bundle.

   

8. Open the Config.java file and make the following modifications:

   • Replace <YOUR_APP_ID> with your App ID.

   • Replace <YOUR_LICENSE_NAME> with your certificate file name. For example, bytedance_4.2.3.licbag.

   
   

   _5

   public interface Config {

   _5

   String mAppId = "<YOUR_APP_ID>";

   _5

   String mToken = null;

   _5

   String mLicenseName = "<YOUR_LICENSE_NAME>";

   _5

   }

   
   

9. Connect a physical Android device (not an emulator), and run the project.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

​

This section lists the APIs related to using extensions with the Agora SDK.

• addExtension in the RtcEngineConfig class
• enableExtension in the RtcEngine class
• setExtensionProperty in the RtcEngine class
• onChannelMediaRelayEvent in the IRtcEngineEventHandler class

​

This section might not provide the latest information. You can find a more detailed explanation from Effects Interface - C, where you use the name of the key to search the corresponding API.

To implement the BytePlus Effects extension in your app, pass in the corresponding key-value pair when calling setExtensionProperty or setExtensionPropertyWithVendor:

• key: bef_effect_ai_check_license
• value: licensePath. String. The path of the license file.

See Special effects handle license.

• key: bef_effect_ai_init
• value: The value includes:
  • strModelDir: String. The path of the resource file.
  • deviceName: String. The device name. Generally you can enter an empty string ("").

See Initialize special effects handle.

• key: bef_effect_ai_set_composer
• value: strPath. String. The path to composer resource files.

See Initialize composer.

• key: bef_effect_ai_composer_remove_nodes
• value: The value includes:
  • nodePaths: String. The array containing effect resource paths.
  • nodeNum: Integer. The length of the array containing effect resource paths.

See Remove composer.

• key: bef_effect_ai_set_orientation
• value: orientation. The parameter type is bef_ai_rotate_type. You can set it to 0 (no rotation), 1（rotate 90 degrees clockwise), 2（rotate 180 degrees clockwise), or 3（rotate 270 degrees clockwise.

See Set the phone angle.

• key: bef_effect_ai_composer_set_nodes
• value: nodePaths. String. The array containing effect resource paths.

See Set superimposed effects.

• key: bef_effect_ai_composer_update_node
• value: The value includes:
  • nodePath: String. The path of the effect resource.
  • nodeTag: String. The key of the effect. See Functions of Resource Keys.
  • value: Float. The intensity of a single node for the combined effects.

See Set the intensity of superimposed effects.

• key: bef_effect_ai_set_effect
• value: strPath. String. The path of the effect resource.

See Set stickers.

• key: bef_effect_ai_set_color_filter_v2
• value: strPath. String. The path of the effect resource.

See Set filters.

• key: bef_effect_ai_set_intensity
• value: fIntensity. Float. The value ranges between [0.0, 1.0], where 0 means disabling the filter.

See Set filter intensity.