Custom audio source

By default, Voice SDK uses the standard audio module on the device your app runs on. However, there are certain scenarios where you want to integrate a custom audio source into your app, such as:

Your app has its own audio module.
You want to use a non-microphone source, such as recorded audio data.
You need to process the captured audio with a pre-processing library for audio enhancement.
You need flexible device resource allocation to avoid conflicts with other services.

Understand the tech

To set an external audio source, you configure the Agora Engine before joining a channel. To manage the capture and processing of audio frames, you use methods from outside the Voice SDK that are specific to your custom source. Voice SDK enables you to push processed audio data to subscribers in a channel.

The following figure shows the workflow you need to implement to stream a custom audio source in your app.

Process custom audio

Prerequisites

To test the code used in this page you need to have:

Implemented either of the following:
- UI Kit quickstart
- SDK quickstart

An Agora account and project.
A computer with Internet access.

Ensure that no firewall is blocking your network communication.

Integrate custom audio or video

To stream from a custom source, you convert the data stream into a suitable format and push this data using Video SDK.

Implement a custom audio source

To push audio from a custom source to a channel, take the following steps:

Add the required imports

import io.agora.rtc2.ChannelMediaOptions
import io.agora.rtc2.Constants
import io.agora.rtc2.audio.AudioTrackConfig
import java.io.IOException
import java.io.InputStream

Add the required variables

// Custom audio parameters
private var customAudioTrackId = -1
private val audioFile = "applause.wav" // raw audio file
private val sampleRate = 44100
private val numberOfChannels = 2
private val bitsPerSample = 16
private val samples = 441
private val bufferSize = samples * bitsPerSample / 8 * numberOfChannels
private val pushInterval = samples * 1000 / sampleRate
private var inputStream: InputStream? = null
private var pushingTask: Thread? = null
var pushingAudio = false

Enable custom audio track publishing

To enable custom audio track publishing, you set ChannelMediaOptions to disable the microphone audio track and enable the custom audio track. You also enable custom audio local playback and set the external audio source.

fun playCustomAudio() {
    // Create a custom audio track
    val audioTrackConfig = AudioTrackConfig()
    audioTrackConfig.enableLocalPlayback = true
    customAudioTrackId = agoraEngine!!.createCustomAudioTrack(
        Constants.AudioTrackType.AUDIO_TRACK_MIXABLE,
        audioTrackConfig
    )
    // Set custom audio publishing options
    val options = ChannelMediaOptions()
    options.publishCustomAudioTrack = true // Enable publishing custom audio
    options.publishCustomAudioTrackId = customAudioTrackId
    options.publishMicrophoneTrack = false // Disable publishing microphone-captured audio
    agoraEngine!!.updateChannelMediaOptions(options)
    // Open the audio file
    openAudioFile()
    // Start the pushing task
    pushingTask = Thread(PushingTask(this))
    pushingAudio = true
    pushingTask?.start()
}
private fun openAudioFile() {
    // Open the audio file
    try {
        inputStream = mContext.resources.assets.open(audioFile)
        // Use the inputStream as needed
    } catch (e: IOException) {
        e.printStackTrace()
    }
}
fun stopCustomAudio() {
    pushingAudio = false
    pushingTask?.interrupt()
}

Read the input stream into a buffer

You read data from the input stream into a buffer.

private fun readBuffer(): ByteArray? {
    // Read the audio file buffer
    val byteSize = bufferSize
    val buffer = ByteArray(byteSize)
    try {
        if (inputStream!!.read(buffer) < 0) {
            inputStream!!.reset()
            return readBuffer()
        }
    } catch (e: IOException) {
        e.printStackTrace()
    }
    return buffer
}

Push the audio frames

You push the data in the buffer as an audio frame using a separate process.

internal class PushingTask(private val manager: CustomVideoAudioManager) : Runnable {
    override fun run() {
        Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO)
        while (manager.pushingAudio) {
            val before = System.currentTimeMillis()
            manager.agoraEngine?.pushExternalAudioFrame(manager.readBuffer(),
                System.currentTimeMillis(),
                manager.sampleRate,
                manager.numberOfChannels,
                Constants.BytesPerSample.TWO_BYTES_PER_SAMPLE,
                manager.customAudioTrackId
            )
            val now = System.currentTimeMillis()
            val consuming = now - before
            if (consuming < manager.pushInterval) {
                try {
                    Thread.sleep(manager.pushInterval - consuming)
                } catch (e: InterruptedException) {
                    e.printStackTrace()
                }
            }
        }
    }
}

pushExternalAudioFrame

Test custom streams

To ensure that you have implemented streaming from a custom source into your app:

Load the web demo
1. Generate a temporary token in Agora Console
2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.
Clone the documentation reference app
- Android documentation app
Configure the project
1. Open the file <samples-root>/agora-manager/res/raw/config.json
2. Set appId to the AppID of your project.
3. Choose one of the following authentication methods:
  - Temporary token
    1. Generate an RTC token using your uid and channelName and set rtcToken to this value in config.json.
    2. Set channelName to the name of the channel you used to create the rtcToken.
  - Authentication server
    1. Setup an Authentication server
    2. In config.json, set:
      - channelName to the name of a channel you want to join.
      - token and rtcToken to empty strings.
      - serverUrl to the base URL for your token server. For example: https://agora-token-service-production-yay.up.railway.app.
Run the reference app
1. In Android Studio, connect a physical Android device to your development machine.
2. Click Run to launch the app.
3. A moment later you see the project installed on your device.
If this is the first time you run the project, grant microphone access to the app.

Choose this sample in the reference app

From the main screen of the app, choose Voice Calling from the dropdown and then select Custom video and audio.
Test the custom audio source

Press Join. You hear the audio file streamed to the web demo app.

To use this code for streaming data from your particular custom audio source, modify the readBuffer() method to read the audio data from your source, instead of a raw audio file.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Raw audio processing