How can I enable image enhancement?

During a video call or live streaming, users often want to improve their on-screen appearance, which can help improve their confidence. The Agora Video SDK provides APIs to help you easily implement basic image enhancement. Users can enable this functionality and then adjust a number of image enhancement options including skin smoothing, acne removal, and a "rosy cheeks" effect to achieve natural-looking enhancements.

If the basic image enhancement function provided by the Agora Video SDK does not meet your needs and scenarios, you can integrate a third-party image enhancement SDK with the Agora Video SDK to implement real-time video interaction with advanced image enhancement functions. Agora provides an open-source sample project for your reference.

Implementation

Ensure that you have implemented basic real-time functions in your project. See Start Interactive Live Video Streaming or Start a Video Call.
Call setBeautyEffectOptions to enable image enhancement and set the image enhancement options.

As of v3.6.0, the Agora Video SDK for Native platforms and third-party frameworks updates the Agora image enhancement algorithm, which improves the image enhancement effects and supports sharpness adjustment. If you want to experience the optimized image enhancement effects or set the sharpness, upgrade your SDK and make sure the following dynamic libraries are integrated before you call setBeautyEffectOptions:

Android: libagora_video_process_extension.so
iOS: AgoraVideoProcessExtension.xcframework
macOS: AgoraVideoProcessExtension.framework or AgoraVideoProcessExtension.xcframework
Windows: libagora_video_process_extension.dll

Sample code

Java

// Java
mRtcEngine.setBeautyEffectOptions(true, new BeautyOptions(LIGHTENING_CONTRAST_NORMAL, 0.5F, 0.5F, 0.5F));

Swift

// Swift
let options = AgoraBeautyOptions()
options.lighteningContrastLevel = .normal
options.rednessLevel = 0
options.smoothnessLevel = 0
options.lighteningLevel = 0
agoraKit.setBeautyEffectOptions(true, options: options)

Objective-C

// Objective-C
AgoraBeautyOptions *options = [[AgoraBeautyOptions alloc] init];
options.lighteningContrastLevel = AgoraLighteningContrastNormal;
options.rednessLevel = 0;
options.smoothnessLevel = 0;
options.lighteningContrastLevel = 0;
[self.agoraKit setBeautyEffectOptions:YES options:options];

C++

// C++
bool enabled = true;
agora::rtc::BeautyOptions options;
options.lighteningContrastLevel = BeautyOptions::LIGHTENING_CONTRAST_NORMAL;
options.lighteningLevel = 0.7;
options.smoothnessLevel = 0.5;
options.rednessLevel = 0.1;
m_lpAgoraEngine->setBeautyEffectOptions(enabled, options);

Web 3.x

// Web 3.x
// setBeautyEffectOptions is an asynchronous method and must be called with Promise or async/await keywords.
// To enable image enhancement immediately after creating a video stream, you can call setBeautyEffectOptions in the Client.on("stream-published") callback.
var streamPublishedHandler = async function () {
  await localStream.setBeautyEffectOptions(true, {
    lighteningContrastLevel: 1,
    lighteningLevel: 0.7,
    smoothnessLevel: 0.5,
    rednessLevel: 0.1
  });
  client.off('stream-published', streamPublishedHandler);
};
client.on('stream-published', streamPublishedHandler);

Web 4.x

// Web 4.x
// Call setBeautyEffect in LocalVideoTrack to set the basic image enhancement function.
// The localVideoTrack in the following example is a local camera video track object created with AgoraRTC.createCameraVideoTrack.
localVideoTrack
  .setBeautyEffect(true, {
    lighteningContrastLevel: 1,
    lighteningLevel: 0.7,
    smoothnessLevel: 0.5,
    rednessLevel: 0.1
  })
  .then(() => {
    console.log('set Beauty Effect Options success!');
  });

Agora also provides an Online Demo for you to experience the basic image enhancement function.

Web 3.x: setBeautyEffectOptions
Web 4.x: BeautyEffectOptions

Third-party frameworks developed from Native

Considerations

The image enhancement function involves real-time processing that is resource intensive. Therefore, enabling this function can reduce the system performance of low-end devices below acceptable levels. Agora does not recommend enabling image enhancement for low-end devices where the user's video-encoding profile is 360P at 30 fps or 720P at 15 fps or higher.
The image enhancement function of the Agora Web SDK is not supported on mobile devices and only supported on certain browsers. See Web 3.x or 4.x API reference.
When you use the setBeautyEffectOptions method of the Agora Web 3.x SDK, note that this method has call sequence restrictions and image enhancement options are only available for high-quality video streams. See Web 3.x API reference.