Skip to main content
Android
iOS
Web
macOS
Windows
Flutter
React Native

Dubbing AI Voice Changer

Dubbing AI Voice Changer extension enables you to change the audio timbre.

This page shows you how to integrate the Dubbing AI Voice Changer extension in your app.

Understand the tech

Using the setExtensionProperty method provided in Agora SDK v4.x and passing in the specified key and value parameters, you can quickly integrate real-time AI voice conversion ability into your app.

The key parameter you specify in the setExtensionProperty method corresponds to the name of the API, and the value parameter wraps some or all of the parameters of the API in JSON format. By passing in the specified key and value parameters, you can call the corresponding API to realize the functions related to real-time AI sound conversion.

Prerequisites

Ensure that your development environment meets the following requirements:

  • Android Studio 4.1 or later.
  • A physical device (not an emulator) running Android 5.0 or later.
  • Dubbing AI Voice Changer is used with Agora Voice SDK v4.x.

    Refer to the SDK quickstart to integrate Voice SDK v4.x and implement basic voice calling.

Integrate the extension

This section shows you how to integrate the Dubbing AI Voice Changer extension, and call the core API to implement the voice changing function.

  1. Integrate the Dubbing AI Voice Changer extension

    To integrate the extension in your project:

    1. Enter the Extensions Marketplace and download the Android Dubbing AI Voice Changer extension package. After unzipping, save all .aar files to the project folder /app/libs.

    2. Get the following resource files and save them in the same directory of the project folder. For example, a new vc_model directory:

      • License file and tone files: Contact Agora to obtain these files. The suffix of the tone file is .dat, and the tone file is issued according to the license.
      • Model file: Download the required resources. Resource Files
    3. Open the app/build.gradle file and add the following lines under dependencies:


      _1
      implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

  2. Enable the plugin

    After creating and initializing RtcEngine, call enableExtension to enable the plug-in, before you call other APIs such as enableVideo and joinChannel.


    _22
    // Declare parameters
    _22
    private val EXTENSION_NAME = "dubbing_vc"
    _22
    private val EXTENSION_VENDOR_NAME = "Dubbing"
    _22
    private val EXTENSION_AUDIO_FILTER = "DubbingVC"
    _22
    _22
    private val changeSpeaker_ = "changeSpeaker"
    _22
    private val startRealTimeTranscribe_ = "startRealTimeTranscribe"
    _22
    private val stopRealTimeTranscribe_ = "stopRealTimeTranscribe"
    _22
    private val getSpeakersInfo_ = "getSpeakersInfo"
    _22
    _22
    val config = RtcEngineConfig()
    _22
    config.mContext = baseContext
    _22
    config.mAppId = appId
    _22
    config.mEventHandler = mRtcEventHandler
    _22
    // Load plugin
    _22
    config.addExtension(EXTENSION_NAME)
    _22
    config.mExtensionObserver = extensionObserver
    _22
    // Create and initialize RtcEngine
    _22
    mRtcEngine = RtcEngine.create(config)
    _22
    mRtcEngine.enableAudio()
    _22
    // Enable plugin
    _22
    mRtcEngine.enableExtension(EXTENSION_VENDOR_NAME, EXTENSION_AUDIO_FILTER, enable)

  3. Set the resource file path

    When you download and integrate the plug-in, the license, sound, and model files are saved to the specified directory. In this step, you specify the path where these resource files are located.


    _1
    val modelPath: String = "${context.filesDir}${File.separator}vc_model"

  4. Get the tone list

    After receiving the onExtensionStarted callback from Agora SDK, call getExtensionProperty passing in the getSpeakersInfo key to get the timbre list:


    _7
    val speakerList = mRtcEngine.getExtensionProperty(
    _7
    EXTENSION_VENDOR_NAME,
    _7
    EXTENSION_AUDIO_FILTER,
    _7
    getSpeakersInfo_
    _7
    )
    _7
    // Convert JSON to array
    _7
    val arr = JSONArray(speakerList)

    The timbre list is returned as JSON data that you parse yourself.

  5. Start changing

    Call setExtensionProperty and pass in the corresponding key and value:


    _6
    mRtcEngine.setExtensionProperty(
    _6
    EXTENSION_VENDOR_NAME,
    _6
    EXTENSION_AUDIO_FILTER,
    _6
    startRealTimeTranscribe_,
    _6
    "true"
    _6
    )

  6. Select timbre

    Pass in the timbre ID from the obtained timbre list to set the corresponding timbre:


    _6
    mRtcEngine.setExtensionProperty(
    _6
    EXTENSION_VENDOR_NAME,
    _6
    EXTENSION_AUDIO_FILTER,
    _6
    changeSpeaker_,
    _6
    id
    _6
    )

  7. Stop changing

    Call the API to stop the voice changer:


    _6
    mRtcEngine.setExtensionProperty(
    _6
    EXTENSION_VENDOR_NAME,
    _6
    EXTENSION_AUDIO_FILTER,
    _6
    stopRealTimeTranscribe_,
    _6
    "true"
    _6
    )

  8. Release resources

    Close the plug-in and release the resources used by the plug-in:


    _1
    mRtcEngine.enableExtension(EXTENSION_VENDOR_NAME, EXTENSION_AUDIO_FILTER, false)

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Sample project

The Dubbing AI Voice Changer extension provides a GitHub sample project for testing. To clone and run the project:

  1. Clone the repository


    _1
    git clone https://github.com/Dubbing-AI-Voice-Changer/DubbingAgoraDemo.git

  2. Refer to the README.md file in the repository to run the demo.

API reference

vundefined