Integrate an extension

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

To test the code used in this page you need to have:

• Physical media input devices, such as a camera and a microphone.
• A JavaScript package manager such as npm.

• An Agora account and project.

• A computer with Internet access.

  Ensure that no firewall is blocking your network communication.

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.

   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

      You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

You are now ready to integrate the extension in your app.

This section shows you how to implement the video filter extension in your app:

1. Add the required variables

   To integrate the extension, in preload.js, add the following variables to declarations:

   var path = "agora_segmentation_extension";
   

   var provider = "agora_video_filters_segmentation";
   

   var extension = "portrait_segmentation";
   

   var enableExtension = false;
   

2. Implement the logic to load and enable the extension

   To load the extension in your app, call loadExtensionProvider and pass the extension path. To enable the extension, call enableExtension and pass the provider name and extension name. To implement this workflow, in preload.js, add the following method before window.onload = () =>:

   function enableExtension()
   

   {
   

   if (!path) {
   

   console.log('path is invalid');
   

   return;
   

   }
   

   if (!provider) {
   

   console.log('provider is invalid');
   

   return;
   

   }
   

   if (!extension) {
   

   console.log('extension is invalid');
   

   return;
   

   }
   

   agoraEngine.loadExtensionProvider(path);
   

   agoraEngine.enableExtension(provider, extension, enableExtension);
   

   };
   

3. Implement the logic to disable the extension

   To disable the extension, call enableExtension and pass the provider, extension, and enableExtension variables as parameters. The enableExtension variable is set to false to disable the extension. To implment this logic, in preload.js, add the following method before window.onload = () =>:

   function disableExtension()
   

   {
   

   agoraEngine.enableExtension(provider, extension, enableExtension);
   

   };
   

4. Setup an event listener to enable and disable the extension with a button

   When the user presses Enable Extension, your app loads and starts the extension with a call to the enableExtension method. When the user presses the button again, your app disables the extension with a call to the disableExtension method. To implement this workflow, in preload.js, add the following method before document.getElementById("leave").onclick = async function ():

   document.getElementById("extension").onclick = async function ()
   

   {
   

   enableExtension = !enableExtension;
   

   if(isExtension)
   

   {
   

   enableExtension();
   

   document.getElementById("extension").innerHTML = "Disable Extension";
   

   }
   

   {
   

   disableExtension();
   

   document.getElementById("extension").innerHTML = "Enable Extension";
   

   }
   

   }
   

5. Implement the required callbacks to manage the extension

   Video SDK provides the following callbacks to manage an extension:

   • onExtensionStarted: Occurs when the extension is enabled.
   • onExtensionError: Occurs when the extension runs incorrectly.
   • onExtensionStopped: Occurs when the extension is disabled.
   • onExtensionEvent: The event callback of the extension.

   To implement these callbacks, in preload.js, add the following callbacks before onUserJoined:

   onExtensionError:( provider, extName, error, msg) =>
   

   {
   

   console.log("Error :" + error, "Error detail :" + msg);
   

   },
   

   onExtensionEvent:( provider, extName, key, value) =>
   

   {
   

   },
   

   onExtensionStarted:(provider, extName) =>
   

   {
   

   },
   

   onExtensionStopped:(provider, extName) =>
   

   {
   

   }
   

To ensure that you have integrated the extension in your app:

1. In preload.js, update appID, channel and token with your values.

2. Run the app

   Execute the following command in the terminal:

   npm start
   

   You see your app opens a window named Get started with Broadcast Streaming.

3. Select Host and press Join.

4. Press Enable Extension. Your app start using the Agora video filter extension.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

• loadExtensionProvider
• setExtensionProperty
• enableExtension
• onExtensionStarted
• onExtensionStopped
• onExtensionError
• onExtensionEvent