Skip to main content
  • Core Products
  • Add-Ons
  • Extensions Marketplace
Android
Linux C

Multi-streaming

Some special scenarios require live streaming over two or more separate channels. For example, consider the case of a real-time monitoring system, where camera feed is shared over two channels for two types of users. There are also certain applications where you want to send multiple streams over the same channel. Consider the case of a monitoring system where feeds from multiple cameras are shared on the same channel.

Agora IoT SDK multi-streaming allows you to join multiple channels at the same time or send multiple streams over a single channel. This page shows you how to implement two different multi-streaming methods into your app using IoT SDK. Choose the method that best fits your particular scenario.

Understand the tech​

Agora IoT SDK provides the following approaches to implementing multi-streaming:

  • Push multiple streams to a single channel

    To push multiple streams to a single channel, you create multiple connections using Agora engine. You call the join channel method multiple times with the same channel name but different connection Ids and distinct user Ids to set up multiple streams. To send audio or video data, you use the connection Id of the intended stream.

  • Stream to multiple channels

    To stream over multiple channels, you create multiple connections using Agora engine. To join each channel, you use a distinct channel name and a dedicated connection Id. You use the connection Id to specify the intended channel when sending audio or video data, when leaving a particular channel, or when closing a connection.

The following figure shows the workflow you need to implement to add multi-streaming to your app:

Live streaming over multiple channels

Prerequisites​

To follow this procedure you must have implemented the SDK quickstart for IoT SDK.

Project setup​

In order to create the environment necessary to implement Agora multi-streaming feature into your app, open the IoT SDK SDK quickstart project you created previously.

Implement multi-streaming​

This section shows you how to implement the following multi-streaming methods:

Add pthread library to your project​

The pthread library enables you to control multiple different flows of work that overlap in time. To integrate pthread into your app:

  1. Add the library into your project

    In the project you implemented for the SDK quickstart for IoT SDK, open <project-root>/agora_rtsa_sdk/example/hello_rtsa/CMakeLists.txt and add the following line after target_link_libraries(hello_rtsa_multi agora-rtc-sdk file_parser ${LIBS}):

    target_link_libraries(hello_rtsa_multi -lpthread)
    Copy

  2. Include the header in your code

    Open <project-root>/agora_rtsa_sdk/example/hello_rtsa/hello_rtsa_multi.c and add the following to the list of #include statements:

    #include <pthread.h>
    Copy

Push multiple streams to a single channel​

Pushing multiple audio and video streams to a single channel is implemented in <project-root>/agora_rtsa_sdk/example/hello_rtsa/hello_rtsa_multi.c in the sample project. The following section shows and explains code snippets that you can reuse in your own app to integrate this functionality.

  1. Initialize the SDK, verify license, and set connection count

    The following code initializes the Agora Engine event handler, configures options, and sets the connection count. That is, the number of streams you push to a single channel:

    // Initialize the event handler
    agora_rtc_event_handler_t event_handler = { 0 };
    app_init_event_handler(&event_handler, config);

    // Set Agora Engine options
    rtc_service_option_t service_opt = { 0 };
    service_opt.area_code = config->area;
    service_opt.log_cfg.log_level = RTC_LOG_INFO;
    service_opt.log_cfg.log_path = config->p_sdk_log_dir;
    snprintf(service_opt.license_value, sizeof(service_opt.license_value), "%s", config->p_license);

    // Initialize Agora rtc sdk
    rval = agora_rtc_init(p_appid, &event_handler, &service_opt);
    if (rval < 0) {
    LOGE("Failed to initialize Agora sdk, reason: %s", agora_rtc_err_2_str(rval));
    return -1;
    }

    // Set bandwidth parameters
    agora_rtc_set_bwe_param(CONNECTION_ID_ALL, DEFAULT_BANDWIDTH_ESTIMATE_MIN_BITRATE,
    DEFAULT_BANDWIDTH_ESTIMATE_MAX_BITRATE, DEFAULT_BANDWIDTH_ESTIMATE_START_BITRATE);

    // Connection count
    g_app.conn_cnt = 3;
    Copy

  2. Create multiple connections

    When the app starts, you create the multiple connections that you use to send audio or video data to a channel:

    // Create connections
    for (i = 0; i < g_app.conn_cnt; i++)
    {
    rval = agora_rtc_create_connection(&g_app.conn_ids[i]);
    if (rval < 0) {
    LOGE("Failed to create connection, reason: %s", agora_rtc_err_2_str(rval));
    return -1;
    }
    }
    Copy

  3. Handle Agora Engine events

    IoT SDK uses callbacks in the event handler to notify you of important events. For example, users joining or leaving a channel, and receipt of audio or video data. The following code adds some important methods to handle IoT SDK events and initializes the event handler.

    // Important events
    static void __on_join_channel_success(connection_id_t conn_id, uint32_t uid, int elapsed)
    {
    int i = 0;
    for (i = 0; i < g_app.conn_cnt; i++) {
    if (conn_id == g_app.conn_ids[i]) {
    g_app.conn_flags[i] = true;
    }
    }
    g_app.b_connected_flag = true;
    LOGI("[conn-%u] Join the channel successfully, uid %u elapsed %d ms", conn_id, uid, elapsed);

    connection_info_t conn_info = { 0 };
    if (agora_rtc_get_connection_info(conn_id, &conn_info) == 0) {
    LOGI("conn_id=%u uid=%u cname=%s", conn_info.conn_id, conn_info.uid, conn_info.channel_name);
    }
    }

    static void __on_connection_lost(connection_id_t conn_id)
    {
    int i = 0;
    for (i = 0; i < g_app.conn_cnt; i++) {
    if (conn_id == g_app.conn_ids[i]) {
    g_app.conn_flags[i] = true;
    }
    }
    g_app.b_connected_flag = false;
    LOGW("[conn-%u] Lost connection from the channel", conn_id);

    connection_info_t conn_info = { 0 };
    if (agora_rtc_get_connection_info(conn_id, &conn_info) == 0) {
    LOGI("conn_id=%u uid=%u cname=%s", conn_info.conn_id, conn_info.uid, conn_info.channel_name);
    }
    }
    static void __on_user_joined(connection_id_t conn_id, uint32_t uid, int elapsed_ms)
    {
    LOGI("[conn-%u] Remote user \"%u\" has joined the channel, elapsed %d ms", conn_id, uid, elapsed_ms);
    if (uid == 1 || uid == 2) {
    LOGI("dddd");
    }
    }

    static void __on_audio_data(connection_id_t conn_id, const uint32_t uid, uint16_t sent_ts, const void *data, size_t len,
    const audio_frame_info_t *info_ptr)
    {
    LOGI("[conn-%u] on_audio_data, uid %u sent_ts %u data_type %d, len %zu", conn_id, uid, sent_ts,
    info_ptr->data_type, len);
    //write_file(g_app.audio_file_writer, data_type, data, len);
    }

    static void __on_video_data(connection_id_t conn_id, const uint32_t uid, uint16_t sent_ts, const void *data, size_t len,
    const video_frame_info_t *info_ptr)
    {
    LOGI("[conn-%u] on_video_data: uid %u sent_ts %u data_type %d frame_type %d stream_type %d len %zu", conn_id,
    uid, sent_ts, info_ptr->data_type, info_ptr->frame_type, info_ptr->stream_type, len);
    //write_file(g_app.video_file_writer, info_ptr->data_type, data, len);
    }

    // Initialize the event_handler
    static void app_init_event_handler(agora_rtc_event_handler_t *event_handler, app_config_t *config)
    {
    event_handler->on_join_channel_success = __on_join_channel_success;
    event_handler->on_connection_lost = __on_connection_lost;
    event_handler->on_rejoin_channel_success = __on_rejoin_channel_success;
    event_handler->on_user_joined = __on_user_joined;
    event_handler->on_user_offline = __on_user_offline;
    event_handler->on_user_mute_audio = __on_user_mute_audio;
    event_handler->on_user_mute_video = __on_user_mute_video;
    event_handler->on_target_bitrate_changed = __on_target_bitrate_changed;
    event_handler->on_key_frame_gen_req = __on_key_frame_gen_req;
    event_handler->on_video_data = __on_video_data;
    event_handler->on_error = __on_error;

    if (config->enable_audio_mixer) {
    event_handler->on_mixed_audio_data = __on_mixed_audio_data;
    } else {
    event_handler->on_audio_data = __on_audio_data;
    }
    }
    Copy

  4. Join the same channel multiple times

    You configure rtc_channel_options_t and join the same channel multiple times with different user Ids and connection Ids.

    // Set Channel options
    rtc_channel_options_t channel_options;
    memset(&channel_options, 0, sizeof(channel_options));
    channel_options.auto_subscribe_audio = true;
    channel_options.auto_subscribe_video = true;
    channel_options.subscribe_local_user = false;

    // Join multiple channel with different user Ids and connection Ids.
    for (i = 0; i < g_app.conn_cnt; i++) {
    rval = agora_rtc_join_channel(g_app.conn_ids[i], config->p_channel, config->uid + i + 1, config->p_token,
    &channel_options);
    if (rval < 0) {
    LOGE("Failed to join channel \"%s\", reason: %s", config->p_channel, agora_rtc_err_2_str(rval));
    return -1;
    }
    }
    Copy

  5. Stream audio and video data

    You create tasks, that is, asynchronous threads, to send video and audio frames.

    // Create tasks sending video and audio frames
    pthread_t video_thread_id;
    pthread_t audio_thread_id;

    // Video stream task
    rval = pthread_create(&video_thread_id, NULL, video_send_thread, 0);
    if (rval < 0) {
    printf("Unable to create video push thread\n");
    return -1;
    }

    // Audio stream task
    rval = pthread_create(&audio_thread_id, NULL, audio_send_thread, 0);
    if (rval < 0) {
    printf("Unable to create audio push thread\n");
    return -1;
    }

    pthread_join(video_thread_id, NULL);
    pthread_join(audio_thread_id, NULL);
    Copy

  6. Send an audio frame to through muliple connections

    To send audio, you read data from an audio file and call agora_rtc_send_audio_data. The following code sends an audio frame each time it is called:

    static int app_send_audio(void)
    {
    app_config_t *config = &g_app.config;
    int i = 0;

    // Get audio frame data
    frame_t frame;
    if (file_parser_obtain_frame(g_app.audio_file_parser, &frame) < 0) {
    LOGE("The file parser failed to obtain audio frame");
    return -1;
    }

    // Send audio frames through all connections
    audio_frame_info_t info = { 0 };
    info.data_type = config->audio_data_type;

    for (i = 0; i < g_app.conn_cnt; i++) {
    if (!g_app.conn_flags[i]) {
    continue;
    }
    int rval = agora_rtc_send_audio_data(g_app.conn_ids[i], frame.ptr, frame.len, &info);
    if (rval < 0) {
    LOGE("Failed to send audio data, reason: %s", agora_rtc_err_2_str(rval));
    }
    }

    file_parser_release_frame(g_app.audio_file_parser, &frame);
    return 0;
    }
    Copy

  7. Send a video frame to through muliple connections

    To send video, you read frame data from a video file, create a video_frame_info_t, and call agora_rtc_send_video_data. The following code sends a video frame each time it is called:

    static int app_send_video(void)
    {
    app_config_t *config = &g_app.config;
    int i = 0;

    // Get video frame data
    frame_t frame;
    if (file_parser_obtain_frame(g_app.video_file_parser, &frame) < 0) {
    LOGE("The file parser failed to obtain audio frame");
    return -1;
    }

    // Send video frames through all connections
    video_frame_info_t info;
    info.frame_type = frame.u.video.is_key_frame ? VIDEO_FRAME_KEY : VIDEO_FRAME_DELTA;
    info.frame_rate = config->send_video_frame_rate;
    info.data_type = config->video_data_type;
    info.stream_type = VIDEO_STREAM_HIGH;

    for (i = 0; i < g_app.conn_cnt; i++) {
    if (!g_app.conn_flags[i]) {
    continue;
    }
    int rval = agora_rtc_send_video_data(g_app.conn_ids[i], frame.ptr, frame.len, &info);
    if (rval < 0) {
    LOGE("Failed to send video data, reason: %s", agora_rtc_err_2_str(rval));
    }
    }

    file_parser_release_frame(g_app.video_file_parser, &frame);
    return 0;
    }
    Copy

  8. Leave a channel and clean up all connections

    To leave a channel and destroy all connections you call agora_rtc_leave_channel and agora_rtc_destroy_connection with their connection Ids.

    for (i = 0; i < g_app.conn_cnt; i++)
    {
    // Leave channel
    agora_rtc_leave_channel(g_app.conn_ids[i]);
    // Destroy connection
    agora_rtc_destroy_connection(g_app.conn_ids[i]);
    }
    Copy

Stream to multiple channels​

To send audio and video streams to multiple channels, open the same sample project you implemented for Push multiple streams to a single channel and make the following changes:

  1. Declare the variables you need

    To manage other connections and join additional channels, add the following variable declarations to the main() in <project-root>/agora_rtsa_sdk/example/hello_rtsa/hello_rtsa_multi.c:

    char channel_name[10][512];
    char token[10][512];
    Copy

  2. Set channel name & token for each connection

    Using the sample project, your user can join 10 additional channels. In this page you set channel names and tokens for three connections. To do this, add the following code before the join channel loop, agora_rtc_join_channel:

    // Set channel name & token for each connection
    strcpy(channel_name[0], <channel_name1>);
    strcpy(token[0], <token1>);

    strcpy(channel_name[1], <channel_name2>);
    strcpy(token[1], <token2>);

    strcpy(channel_name[2], <channel_name3>);
    strcpy(token[2], <token3>);
    Copy

  3. Join multiple channels

    To send audio and video streams to multiple channels, join these different channels with different tokens, user and connection Ids. Replace agora_rtc_join_channel in the join channel loop with:

    rval = agora_rtc_join_channel(g_app.conn_ids[i], channel_name[i], config->uid + i + 1, token[i],
    &channel_options);
    Copy

Test your implementation​

To ensure that you have implemented multi-streaming into your app, follow the relevant testing procedure:

Test pushing multiple streams to a single channel​

  1. Generate a temporary token in Agora Console.

  2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

  3. Compile the sample project

    In the terminal, execute the following commands:

    cd <project-root>/agora_rtsa_sdk/example

    ./build-x86_64.sh
    Copy

  4. Run the sample app:

    1. In the terminal, navigate to the output folder:

      cd out/x86_64
      Copy

    2. Launch the compiled app with your app ID, channel ID, and token from Agora Console

      ./hello_rtsa_multi --app-id <your-app-id> --channel-id <your-channel-name> --token <authentication-token>
      Copy

You see 3 video streams with audio are playing in the browser. The terminal output shows all the connections sending streams to same channel:

[INF] [conn-1] on_audio_data, uid 3416170155 sent_ts 39985 data_type 100, len 640
[INF] [conn-3] on_audio_data, uid 3416170155 sent_ts 39985 data_type 100, len 640
[INF] [conn-2] on_audio_data, uid 3416170155 sent_ts 39985 data_type 100, len 640
[INF] [conn-3] on_audio_data, uid 3416170155 sent_ts 40005 data_type 100, len 640
[INF] [conn-2] on_audio_data, uid 3416170155 sent_ts 40005 data_type 100, len 640
[INF] [conn-1] on_audio_data, uid 3416170155 sent_ts 40005 data_type 100, len 640
Copy

Test streaming to multiple channels​​

  1. Generate the following temporary tokens in Agora Console.

    • token1 - using the appId and channelName1.

    • token2 - using the appId and channelName2.

    • token3 - using the appId and channelName3.

  2. In your browser, navigate to the Agora web demo and join a channel using appId, channelName1 and token1.

  3. In another browser tab, join the Agora web demo using appId, channelName2 and token2.

  4. In another browser tab, join the Agora web demo using appId, channelName3 and token3.

  5. Compile the sample project

    In the terminal, execute the following commands:

    cd <project-root>/agora_rtsa_sdk/example

    ./build-x86_64.sh
    Copy

  6. Run the sample app:

    1. In the terminal, navigate to the output folder:

      cd out/x86_64
      Copy

    2. Launch the compiled app using your app ID:

      ./hello_rtsa_multi --app-id <your-app-id>
      Copy

You see that your app joins all 3 Agora web demo channels. The video and audio streams are playing individually. The terminal output shows the behavior:

[INF] [conn-3] Join the channel successfully, uid 3 elapsed 559 ms
[INF] conn_id=3 uid=3 cname=token55
[INF] joined-after: conn_id=1 uid=1 cname=token33
[INF] joined-after: conn_id=2 uid=2 cname=token44
[INF] joined-after: conn_id=3 uid=3 cname=token55
[INF] [conn-1] Join the channel successfully, uid 1 elapsed 2053 ms
[INF] conn_id=1 uid=1 cname=token33
[INF] [conn-2] Join the channel successfully, uid 2 elapsed 2164 ms
[INF] conn_id=2 uid=2 cname=token44
[INF] [conn-1] on_audio_data, uid 3416170155 sent_ts 37125 data_type 100, len 640
[INF] [conn-2] on_audio_data, uid 3416170155 sent_ts 37125 data_type 100, len 640
[INF] [conn-3] on_audio_data, uid 3416170155 sent_ts 37145 data_type 100, len 640
[INF] [conn-2] on_audio_data, uid 3416170155 sent_ts 37145 data_type 100, len 640
[INF] [conn-1] on_audio_data, uid 3416170155 sent_ts 37145 data_type 100, len 640
Copy

Reference​

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Page Content

vundefined