Core concepts

RTC (Real-Time Communication) refers to real-time communication technology, which allows almost instant exchange of audio, video, and other data between the sender and the receiver.

Agora SDKs provide real-time audio and video interaction services, with multi-platform and multi-device support. This includes high-definition video calls, voice-only calls, interactive live streaming, as well as one-on-one and multi-group chats.

This guide introduces the key processes and concepts you need to know to use Agora SDKs.

Agora Console is the main dashboard where you manage your Agora projects and services. Before you can use Agora's SDKs, you must first create a project in the Agora Console. See Agora account management for details.

Agora Console provides an intuitive interface for developers to query and manage their Agora account. After registering an Agora Account, you use the Agora Console to perform the following tasks:

• Manage the account
• Create and configure Agora projects and services
• Get an App ID
• Manage members and roles
• Check call quality and usage
• Check bills and make payments
• Access product resources

Agora also provides RESTful APIs that you use to implement features such as creating a project and fetching usage numbers programmatically.

See Agora account management for details on how to manage all aspects of your Agora account.

Agora uses the following basic concepts:

The App ID is a unique key generated by Agora's platform to identify each project. Each project in your account is assigned its own unique App ID. The App ID is critical for connecting users within your app. It's used to initialize the Agora Engine in your app, and as one of the required keys to create authentication tokens for secure communication. Retrieve your App ID using the Agora Console.

Agora uses the App ID to identify each app and provide billing and other statistical data services.

App IDs are stored on the front-end client and do not provide access control. Projects using only an App ID allow any user with the App ID to join voice and video streams.

For applications requiring access controls, such as those in production environments, choose an App ID + Token mechanism for user authentication when creating a new project. Without an authentication token, your environment is open to anyone with your App ID.

An App Certificate is a unique key generated by the Agora Console to secure projects through token authentication. It is required, along with the App ID, to generate a token that proves authorization between your systems and Agora's network. App Certificates are used to generate Video SDK or Signaling authentication tokens.

App Certificates should be stored securely in your backend systems. If your App Certificate is compromised or to meet security compliance requirements, you can invalidate certificates and create new ones through the Agora Console.

A token is a dynamic key generated using the App ID, App Certificate, user ID, and expiration timestamp. Tokens authenticate and secure access to Agora's services, ensuring only authorized users can join a channel and participate in real-time communication.

Tokens are generated on your server and passed to the client for use in the Video SDK or Signaling. The token generation process involves digitally signing the App ID, App Certificate, user ID, and expiration timestamp using a specific algorithm, preventing tampering or forgery.

For testing and during development, use the Agora Console to generate temporary tokens. For production environments, implement a token server as part of your security infrastructure to control access to your channels.

For information on setting up a token server for generating and managing tokens, refer to the guide on Secure authentication with tokens.

In Agora's platform, a channel is a way of grouping users together and is identified by a unique channel name. Users who connect to the same channel can communicate with each other. A channel is created when the first user joins and ceases to exist when the last user leaves.

Channels are created by calling the methods for transmitting real-time data. Agora uses different channels to transmit different types of data:

• The Video SDK channel is used for transmitting audio or video data.
• The Signaling channel is used for transmitting messaging or signaling data.

These channels are independent of each other.

Additional services provided by Agora, such as Cloud Recording and Real-Time Speech-To-Text, join the Video SDK channel to provide real-time recording, transmission acceleration, media playback, and content moderation.

The SDK applies different optimization methods according to the selected channel profile. Agora supports the following channel profiles:

Channel profile	Description
COMMUNICATION	This profile is suitable for one-on-one or group calls, where all users in the channel talk freely.
LIVE_BROADCASTING	In a live streaming channel, users have two client roles: host and audience. The host sends and receives audio or video, while the audience only receives audio or video with the sending function disabled.

A stream is a sequence of digitally-encoded coherent signals that contains audio or video data. Users in a channel publish local streams and subscribe to remote streams from other users.

Publishing is the act of sending a user’s audio or video data to the channel. Usually, the published stream is created by the audio data sampled from a microphone or the video data captured by a camera. You can also publish media streams from other sources, such as an online music file or the user’s screen. After successfully publishing a stream, the SDK continues sending media data to other users in the channel. By publishing the local stream and subscribing to remote streams, users communicate with each other in real-time.

Subscribing is the act of receiving media streams published by remote users to the channel. A user receives audio and video data from other users by subscribing to one or more of their streams. You either directly play the subscribed streams or process incoming data for other purposes such as recording or capturing screenshots.

In Agora's platform, the UID is an integer value that is a unique identifier assigned to each user within the context of a specific channel. When joining a channel, you have the choice to either assign a specific UID to the user or pass 0 or null and allow Agora's platform to automatically generate and assign a UID for the user. If two users attempt to join the same channel with the same UID, it can lead to unexpected behavior.

The UID is used by Agora's services and components to identify and manage users within a channel. Developers should ensure that UIDs are properly assigned to prevent conflicts.

A user role is used to define whether users in the channel have permission to publish streams. There are two user roles:

• Host: A user who can publish streams in a channel.
• Audience: A user who cannot publish streams in a channel. Users with this role can only subscribe to remote audio and video streams.

The connection between the SDK and the channel. When you need to publish or receive multiple streams in multiple channels, a connection is used to specify the target channel.

Agora's core engagement services are powered by its Software-Defined Real-time Network (SD-RTN™), which is accessible and available anytime, anywhere around the world. Unlike traditional networks, the software-defined network is not confined by device, phone numbers, or a telecommunication provider's coverage area. Agora SD-RTN™ has data centers globally, covering over 200 countries and regions. The network delivers sub-second latency and high availability of real-time video and audio anywhere on the globe. With Agora SD-RTN™, Agora can deliver live user engagement experiences in the form of real-time communication (RTC) with the following advantages:

• Unmatched quality of service
• High availability and accessibility
• True scalability
• Low cost

The following diagram shows the basic workflow of using the Agora SDK to implement basic audio and video interaction.

Agora uses the following basic concepts:

In audio interaction, the main functions of the audio module are as shown in the figure below:

After you call registerAudioFrameObserver, you can obtain the raw audio data at the following observation points in the audio transmission process:

1. Obtain the raw audio data of ear monitoring through the onEarMonitoringAudioFrame callback.
2. Obtain the captured raw audio data through the onRecordAudioFrame callback.
3. Obtain the raw audio playback data of each individual stream through the onPlaybackAudioFrameBeforeMixing callback.
4. Obtain the raw audio playback data of all mixed streams through the onPlaybackAudioFrame callback.
5. Obtain the raw audio data after mixing the captured and playback audio through the onMixedAudioFrame callback.
   (5) onMixedAudioFrame = (2) onRecordAudioFrame + (4) onPlaybackAudioFrame

The audio output device used by the app when playing audio. Common audio routes include wired headphones, earpieces, speakers, Bluetooth headphones, and others.

The APIs used by the audio module are as follows:

• Enable local audio collection: enableLocalAudio
• Set local playback device: setPlaybackDevice
• Set up audio routing: setDefaultAudioRouteToSpeakerphone

The following diagram shows the main functions of the video module in video interaction:

The figure shows the following observation points:

1. POSITION_POST_CAPTURER_ORIGIN.
2. POSITION_POST_CAPTURER, corresponds to the onCaptureVideoFrame callback.
3. POSITION_PRE_ENCODER, corresponds to the onPreEncodeVideoFrame callback.
4. POSITION_PRE_RENDERER, corresponds to the onRenderVideoFrame callback.

The APIs used by the video module are as follows:

• Enable local video collection: enableLocalVideo
• Local preview: setupLocalVideo → startPreview
• Video rendering shows: setupRemoteVideo