Play Audio Effect or Music File

During real-time audio and video interaction, for better atmosphere or entertainment, users often need to play audio effects or background music and make all users in the channel hear them. For example, users might add fighting sounds when playing an action game or include an accompaniment when singing. Agora provides two independent groups of methods so that you can play audio effect files and a music file separately.

Agora provides the following open-source demo projects on GitHub that implements audio effect files and a music file. You can view the source code on Github or download the project to try it out.

• iOS: AudioMixing
• macOS: AudioMixing

In this context, the audio effects are ambient sounds with a short duration, such as applause, cheers, fighting sounds, and gunshots. Usually, you can play multiple audio effects at the same time.

Agora provides a group of methods to play and manage audio effect files. Supported functions include the following:

• Play local or online audio effect files.
• Set the spatial position, number of loops, playback position, volume, and other playback options of an audio effect.
• Flexibly control the playing, pausing, resumption, and stopping of audio effect files, individually or globally.

Supported audio effect file formats include MP3, AAC, M4A, MP4, WAV, and 3GP. See Best Practices for iOS Audio.

To play audio effect files, see the following steps:

1. Before joining a channel, call the preloadEffect method to preload the local audio effect file.
2. After joining a channel, call the playEffect method to play the audio effect file. After completing playing an audio effect file, the SDK triggers the rtcEngineDidAudioEffectFinish callback.

Before playing an audio effect file, you need to set filePath and soundId to specify the audio effect file. The representing of two parameters is as follows:

• filePath: The audio effect file path, including the local and online file path. The SDK searches for audio effect files in this path.
• soundId: The audio effect ID, which you define. This ID must be unique. The SDK identifies an audio effect file based on its audio effect ID. Common solutions for defining audio effect IDs include incrementing the ID and using hashCode for the audio effect file name.

// Sets the audio effect ID.

int EFFECT_ID = 1;

// Sets the path of the audio effect file.

NSString *filePath = "your filepath";

// Sets the audio effect ID.

let EFFECT_ID:Int32 = 1

// Sets the path of the audio effect file.

let filePath = "your filepath"

The SDK supports preloading audio effect files. To improve performance, you can add the audio effect files into the memory in advance. Preloading is not required. Agora recommends that you choose whether to preload audio effect files according to your needs.

• If you need to play a specified audio effect repeatedly, Agora recommends preloading the audio effect file.
• If an audio effect file size is large, Agora recommends not preloading the audio effect file.

To preload multiple audio effect files, you need to call preloadEffect multiple times.

The SDK only supports preloading local audio effect files.

Call preloadEffect before joining a channel.

After calling preloadEffect, the audio effect file occupies memory until you call unloadEffect or the user leaves the channel.

// Preloads the specified local audio effect file into the memory.

[agoraKit preloadEffect: EFFECT_ID filePath: filePath];

// Unloads the preloaded audio effect file.

[agoraKit unloadEffect: EFFECT_ID];

// Preloads the specified local audio effect file into the memory.

agoraKit.preloadEffect(EFFECT_ID, filePath: filePath)

// Unloads the preloaded audio effect file.

agoraKit.unloadEffect(EFFECT_ID)

Call playEffect to play the audio effect file. You can call playEffect multiple times to play multiple audio effect files at the same time according to your requirements. When playing an audio effect file, you can set the playback options, such as number of loops, pitch, volume, and playback position.

Call playEffect after joining a channel.

// Sets the number of times the audio effect loops. -1 represents an infinite loop.

int loopCount = 1;

// Sets the pitch of the audio effect. The value range is 0.5 to 2.0, where 1.0 is the original pitch.

double pitch = 1;

// Sets the spatial position of the audio effect. The value range is -1.0 to 1.0.

// -1.0 represents the audio effect occurs on the left; 0 represents the audio effect occurs in the front; 1.0 represents the audio effect occurs on the right.

double pan = 1;

// Sets the volume of the audio effect. The value range is 0 to 100. 100 represents the original volume.

double gain = 100.0;

// Sets whether to publish the audio effect to the remote users. true represents that both the local user and remote users can hear the audio effect; false represents that only the local user can hear the audio effect.

BOOL publish = true;

// Sets the playback position (ms) of the audio effect file. 500 represents that the playback starts at the 500 ms mark of the audio effect file.

int startPos = 500;

// Plays the specified audio effect file.

[agoraKit playEffect: EFFECT_ID filePath: filePath loopCount: loopCount pitch: pitch pan: pan gain: gain publish: publish startPos: startPos];

// Occurs when the local audio effect playback finishes.

- (void)rtcEngineDidAudioEffectFinish:(AgoraRtcEngineKit* _Nonnull)engine EFFECT_ID;

// Sets the number of times the audio effect loops. -1 represents an infinite loop.

let loopCount = 1

// Sets the pitch of the audio effect. The value range is 0.5 to 2.0, where 1.0 is the original pitch.

let pitch = 1

// Sets the spatial position of the audio effect. The value range is -1.0 to 1.0.

// -1.0 represents the audio effect occurs on the left; 0 represents the audio effect occurs in the front; 1.0 represents the audio effect occurs on the right.

let pan = 1

// Sets the volume of the audio effect. The value range is 0 to 100. 100 represents the original volume.

let gain = 100.0

// Sets whether to publish the audio effect to the remote users. true represents that both the local user and remote users can hear the audio effect; false represents that only the local user can hear the audio effect.

let publish = true

// Sets the playback position (ms) of the audio effect file. 500 represents that the playback starts at the 500 ms mark of the audio effect file.

let startPos = 500;

// Plays the specified audio effect file.

agoraKit.playEffect(EFFECT_ID, filePath: filePath, loopCount: Int32(loopCount), pitch: pitch, pan: pan, gain: gain, publish: publish, startPos: startPos)

// Occurs when the local audio effect playback finishes.

func rtcEngineDidAudioEffectFinish(_ engine: AgoraRtcEngineKit, EFFECT_ID) {

}

After successfully playing audio effect files, you can stop playing either a specified file or all audio effect files.

// Stops playing a specified audio effect file.

[agoraKit stopEffect: EFFECT_ID];

// Stops playing all audio effect files.

[agoraKit stopAllEffects];

// Stops playing a specified audio effect file.

agoraKit.stopEffect(EFFECT_ID)

// Stops playing all audio effect files.

agoraKit.stopAllEffects()

When you play audio effect files, you can pause or resume playing either a specified file or all audio effect files.

Call this group of methods after playEffect.

// Pauses playing a specified audio effect file.

[agoraKit pauseEffect: EFFECT_ID];

// Pauses playing all audio effect files.

[agoraKit pauseAllEffects];

// Resumes playing a specified audio effect file.

[agoraKit resumeEffect: EFFECT_ID];

// Resumes playing all audio effect files.

[agoraKit resumeAllEffects];

// Pauses playing a specified audio effect file.

agoraKit.pauseEffect(EFFECT_ID)

// Pauses playing all audio effect files.

agoraKit.pauseAllEffects()

// Resumes playing a specified audio effect file.

agoraKit.resumeEffect(EFFECT_ID)

// Resumes playing all audio effect files.

agoraKit.resumeAllEffects()

If you need to adjust the playback position after playing the audio effect file, you can call this group of methods. For example, you can call this group of methods to adjust the playback position during looped playback of the audio effect file without stopping the playback.

Call getEffectDuration after joining a channel, and call other methods in this group after playEffect.

This group of methods only applies to the local audio effect files.

// Gets the total duration of a specified local audio effect file.

[agoraKit getEffectDuration: filePath: filePath];

// Sets the playback position (ms) of a specified local audio effect file. 500 represents that the playback starts at the 500 ms mark of the audio effect file.

[agoraKit setEffectPosition: EFFECT_ID pos: 500];

// Gets the current playback position of a specified local audio effect file.

[agoraKit getEffectCurrentPosition: EFFECT_ID];

// Gets the total duration of a specified local audio effect file.

agoraKit.getEffectDuration(filePath: filePath)

// Sets the playback position (ms) of a specified local audio effect file. 500 represents that the playback starts at the 500 ms mark of the audio effect file.

agoraKit.setEffectPosition(EFFECT_ID, pos: 500)

// Gets the current playback position of a specified local audio effect file.

agoraKit.getEffectCurrentPosition(EFFECT_ID)

After the audio effect file starts to play, you can call this group of methods to adjust the playback volume. For example, you can call this group of methods to adjust the playback volume during looped playback of the audio effect file without stopping the playback.

Call this group of methods after playEffect.

// Sets the playback volume of all audio effect files. The value range is 0 to 100. 100 represents the original volume.

[agoraKit setEffectsVolume: volume: 50.0];

// Sets the playback volume of a specified audio effect file. The value range is 0 to 100. 100 represents the original volume.

[agoraKit setVolumeOfEffect: EFFECT_ID volume: 50.0];

// Gets the playback volume of all audio effect files. The volume range is 0 to 100. 100 represents the original volume.

[agoraKit getEffectsVolume];

// Sets the playback volume of all audio effect files. The value range is 0 to 100. 100 represents the original volume.

agoraKit.setEffectsVolume(volume: 50.0)

// Sets the playback volume of a specified audio effect file. The value range is 0 to 100. 100 represents the original volume.

agoraKit.setVolumeOfEffect(EFFECT_ID, volume: 50.0)

// Gets the playback volume of all audio effect files. The volume range is 0 to 100. 100 represents the original volume.

agoraKit.getEffectsVolume()

• preloadEffect
• unloadEffect
• playEffect
• stopEffect
• stopAllEffects
• pauseEffect
• pauseAllEffects
• resumeEffect
• resumeAllEffects
• getEffectDuration
• setEffectPosition
• getEffectCurrentPosition
• setEffectsVolume
• setVolumeOfEffect
• getEffectsVolume
• rtcEngineDidAudioEffectFinish

Audio mixing means mixing a music file with the audio captured by a microphone. Users who use the audio mixing function often play a relatively long music file and only play one music file at a time. For example, users can play accompaniment when singing or play background music when chatting.

Agora provides a group of methods to play and manage the music file. Supported functions include the following:

• Play the local or online music file.
• Set the number of playbacks , playback position, volume, pitch, and other playback options of a music file.
• Flexibly control the playing, pausing, resumption, and stopping of the music file.
• Report the current playback state of the music file and the reason for the change.

Supported music file formats include MP3, AAC, M4A, MP4, WAV, and 3GP. See Best Practices for iOS Audio.

After successfully calling startAudioMixing, the SDK triggers the localAudioMixingStateDidChanged callback when the playback state of the music file changes.

Call startAudioMixing to play a music file. When playing a music file, you can set the playback options, such as number of loops and playback position.

If you call startAudioMixing again when the SDK is playing a music file, the SDK automatically stops playing the previous music file and starts playing the next music file.

// Specifies the absolute path of the local or online music file that you want to play.

NSString *filePath = @"your file path";

// Sets whether to only play a music file on the local client. true represents that only the local user can hear the music; false represents that both the local user and remote users can hear the music.

BOOL loopback = NO;

// Sets whether to replace the audio captured by the microphone with a music file. true represents that the user can only hear music; false represents that the user can hear both the music and the audio captured by the microphone.

BOOL replace = NO;

// Sets the number of times to play the music file. 1 represents play the file once.

NSInteger cycle = 1;

// Sets the playback position (ms) of the music file. 500 represents that the playback starts at the 500 ms mark of the music file.

NSInteger startPos = 500;

// Starts to play the music file.

[agoraKit startAudioMixing: filePath loopback: loopback replace: replace cycle: cycle startPos: startPos];

// Occurs when the state of the local user's music file changes.

- (void)rtcEngine:(AgoraRtcEngineKit* _Nonnull)engine localAudioMixingStateDidChanged:(AgoraAudioMixingStateCode)state reason:(AgoraAudioMixingReasonCode)reason;

// Specifies the absolute path of the local or online music file that you want to play.

let filePath = "your file path"

// Sets whether to only play a music file on the local client. true represents that only the local user can hear the music; false represents that both the local user and remote users can hear the music.

let loopback = false

// Sets whether to replace the audio captured by the microphone with a music file. true represents that the user can only hear music; false represents that the user can hear both the music and the audio captured by the microphone.

let replace = false

// Sets the number of times to play the music file. 1 represents play the file once.

let cycle = 1

// Sets the playback position (ms) of the music file. 500 represents that the playback starts at the 500 ms mark of the music file.

let startPos = 500

// Starts to play the music file.

agoraKit.startAudioMixing(filePath, loopback: loopback, replace: replace, cycle: cycle, startPos: startPos)

// Occurs when the state of the local user's music file changes.

func rtcEngine(_ engine: AgoraRtcEngineKit, localAudioMixingStateDidChanged state: AgoraAudioMixingStateCode, reason: AgoraAudioMixingReasonCode) {

}

After successfully playing the music file, you can call stopAudioMixing to stop the playback.

// Stops playing the music file.

[agoraKit stopAudioMixing];

// Stops playing the music file.

agoraKit.stopAudioMixing()

When you play the music file, you can pause or resume playing the music file.

Call this group of methods after calling startAudioMixing.

// Pauses playing the music file.

[agoraKit pauseAudioMixing];

// Resumes playing the music file.

[agoraKit resumeAudioMixing];

// Pauses playing the music file.

agoraKit.pauseAudioMixing()

// Resumes playing the music file.

agoraKit.resumeAudioMixing()

When the music file is playing, you can call this group of methods to adjust the playback position of the music file without stopping the playback.

Call this group of methods after calling startAudioMixing and receiving the localAudioMixingStateDidChanged(AgoraAudioMixingStatePlaying) callback.

// Gets the total duration of the current music file.

[agoraKit getAudioMixingDuration];

// Sets the playback position (ms) of the current music file. 500 represents that the playback starts at the 500 ms mark of the music file.

[agoraKit setAudioMixingPosition: pos: 500];

// Gets the current playback position of the music file.

[agoraKit getAudioMixingCurrentPosition];

// Gets the total duration of the current music file.

agoraKit.getAudioMixingDuration()

// Sets the playback position (ms) of the current music file. 500 represents that the playback starts at the 500 ms mark of the music file.

agoraKit.setAudioMixingPosition(pos: 500)

// Gets the current playback position of the music file.

agoraKit.getAudioMixingCurrentPosition()

After successfully playing the music file, you can call this group of methods to adjust the playback volume and pitch of the music file without stopping the playback.

Call this group of methods after calling startAudioMixing and receiving the localAudioMixingStateDidChanged(AgoraAudioMixingStatePlaying) callback.

// Adjusts the playback volume of the current music file for both local and remote playback. The value range is 0 to 100. 100 represents the original volume.

[agoraKit adjustAudioMixingVolume: volume: 50];

// Adjusts the playback volume of the current music file for the remote playback. The value range is 0 to 100. 100 represents the original volume.

[agoraKit adjustAudioMixingPublishVolume: volume: 50];

// Adjusts the playback volume of the current music file for the local playback. The value range is 0 to 100. 100 represents the original volume.

[agoraKit adjustAudioMixingPlayoutVolume: volume: 50];

// Gets the playback volume of the current music file for the local playback. The volume range is 0 to 100. 100 represents the original volume.

[agoraKit getAudioMixingPlayoutVolume];

// Gets the playback volume of the current music file for the remote playback. The volume range is 0 to 100. 100 represents the original volume.

[agoraKit getAudioMixingPublishVolume];

// Sets the pitch of the current music file. The value range is -12 to 12. 0 represents the original pitch; 1 represents raising the original pitch by a semitone.

[agoraKit setAudioMixingPitch: pitch: 5];

// Adjusts the playback volume of the current music file for both local and remote playback. The value range is 0 to 100. 100 represents the original volume.

agoraKit.adjustAudioMixingVolume(volume: 50)

// Adjusts the playback volume of the current music file for the remote playback. The value range is 0 to 100. 100 represents the original volume.

agoraKit.adjustAudioMixingPublishVolume(volume: 50)

// Adjusts the playback volume of the current music file for the local playback. The value range is 0 to 100. 100 represents the original volume.

agoraKit.adjustAudioMixingPlayoutVolume(volume: 50)

// Gets the playback volume of the current music file for the local playback. The volume range is 0 to 100. 100 represents the original volume.

agoraKit.getAudioMixingPlayoutVolume()

// Gets the playback volume of the current music file for the remote playback. The volume range is 0 to 100. 100 represents the original volume.

agoraKit.getAudioMixingPublishVolume()

// Sets the pitch of the current music file. The value range is -12 to 12. 0 represents the original pitch; 1 represents raising the original pitch by a semitone.

agoraKit.setAudioMixingPitch(pitch: 5)

After successfully playing a music file, you can call setAudioMixingPlaybackSpeed to set the playback speed of the current music file.

Call this method after calling startAudioMixing and receiving the localAudioMixingStateDidChanged(AgoraAudioMixingStatePlaying) callback.

// Agora recommends that you limit this value to between 50 and 400.

// 50 represents half the original speed, 100 represents the original speed, and 400 represents 4 times the original speed.

[agoraKit setAudioMixingPlaybackSpeed: speed: 50];

// Agora recommends that you limit this value to between 50 and 400.

// 50 represents half the original speed, 100 represents the original speed, and 400 represents 4 times the original speed.

agoraKit.setAudioMixingPlaybackSpeed(speed: 50)

After successfully playing a music file, you can call this group of methods to set the playback track of the current music file.

This group of methods apply for iOS only.

Call this group of methods after calling startAudioMixing and receiving the localAudioMixingStateDidChanged(AgoraAudioMixingStatePlaying) callback.

// Gets the number of audio tracks of the current music file.

[agoraKit getAudioTrackCount];

// Specifies the playback track of the current music file.

// The range of index is [0, getAudioTrackCount()).

[agoraKit selectAudioTrack: index: 2];

// Gets the number of audio tracks of the current music file.

agoraKit.getAudioTrackCount()

// Specifies the playback track of the current music file.

// The range of index is [0, getAudioTrackCount()).

agoraKit.selectAudioTrack(index: 2)

After successfully playing a music file, you can call setAudioMixingDualMonoMode to set the following channel mode of the current music file:

• AgoraAudioMixingDualMonoAuto: Original mode.
• AgoraAudioMixingDualMonoL: Left channel mode. This mode replaces the audio of the right channel with the audio of the left channel, which means the user can only hear the audio of the left channel.
• AgoraAudioMixingDualMonoL: Right channel mode. This mode replaces the audio of the left channel with the audio of the right channel, which means the user can only hear the audio of the right channel.
• AgoraAudioMixingDualMonoMix: Mixed channel mode. This mode mixes the audio of the left channel and the right channel, which means the user can hear the audio of the left channel and the right channel at the same time.

Call this method after calling startAudioMixing and receiving the localAudioMixingStateDidChanged(AgoraAudioMixingStatePlaying) callback.

This method only applies to stereo audio files.

[agoraKit setAudioMixingDualMonoMode: AgoraAudioMixingDualMonoAuto];

agoraKit.setAudioMixingDualMonoMode(.auto)

• startAudioMixing
• stopAudioMixng
• pauseAudioMixing
• resumeAudioMixing
• getAudioMixingDuration
• setAudioMixingPosition
• getAudioMixingCurrentPosition
• adjustAudioMixingVolume
• adjustAudioMixingPublishVolume
• adjustAudioMixingPlayoutVolume
• getAudioMixingPlayoutVolume
• getAudioMixingPublishVolume
• setAudioMixingPitch
• setAudioMixingPlaybackSpeed
• getAudioTrackCount
• selectAudioTrack
• setAudioMixingDualMonoMode
• localAudioMixingStateDidChanged

When playing a music file, you can also refer to the following articles:

• What formats of audio files does the Agora Video SDK support?