Skip to main content
  • Core Products
  • Add-Ons
  • Extensions Marketplace

    Record captions

    Real-Time STT can record the caption text it produces, generate a .vtt file, and then store it in the cloud storage as configured. Recorded captions are designed for playback with audio and video recordings. The .vtt file can also be used to generate meeting minutes and topic summaries, conduct sentiment analyses and content moderation, and so on. Caption recording entails no additional fees.

    Since cloud recording and transcription tasks are running on different servers, enable the ntpTimestamp parameter for cloud recording to sync the time with transcription.

    This page explains how to record captions and sync them with the audio or video files generated by Cloud Recording.

    Prerequisites​

    To follow this procedure, you must:

    • Have a valid Agora Account.

    • Have a valid Agora project with an app ID and a temporary token or a token server. For details, see Agora account management.

    • Have a computer with access to the internet. If your network has a firewall, follow the steps in Firewall requirements.

    • Join an RTC channel as a host and start streaming.

    • Make sure Real-Time STT is enabled for your app.

    • Enable ntpTimestamp for cloud recording.

    Implementation​

    Record the captions​

    To record the captions, follow the API call sequence from the REST Quickstart and modify the start request to include caption recording parameters as follows:


    _21
    curl --location -g 'https://api.agora.io/v1/projects/{{appId}}/rtsc/speech-to-text/tasks?builderToken={{tokenName}}' \
    _21
    --header 'Content-Type: application/json' \
    _21
    --data '{
    _21
    {{
    _21
    "languages": [
    _21
    "<YourTranscribeLanguages>"
    _21
    ],
    _21
    "maxIdleTime": 50,
    _21
    "rtcConfig": {
    _21
    // No change on rtcConfig.
    _21
    },
    _21
    "captionConfig": { // The caption recording configurations.
    _21
    "storage": {
    _21
    "accessKey": "<YourOssAccessKey>", // The OSS access key.
    _21
    "secretKey": "<YourOssSecretKey>", // The OSS secret key.
    _21
    "bucket": "<YourOssBucketName>", // The storage bucket name.
    _21
    "vendor": <YourOssVendor>, // The OSS vendor.
    _21
    "region": <YourOssRegion>, // The OSS region.
    _21
    "fileNamePrefix": ["<YourOssPrefix>"] // Optional, an array of strings. Sets the path of the recorded files in the third-party cloud storage.
    _21
    }
    _21
    },

    Sync the files​

    The m3u8+vtt file generated by Real-Time STT and the m3u8+ts file generated by Cloud Recording are two independent files, with different time stamps. The Cloud Recording time stamp starts at 0, while Real-Time STT uses the system time stamp. If either process starts abnormally, the media files generated by the two services may be out of sync during playback.

    Agora provides a post-processing script that lets you sync the m3u8+ts and m3u8+vtt files. To run this script, take the following steps:

    1. Unzip the post-processing script to a local folder.

    2. Run the script on your transcription files:


      _2
      1
      _2
      python3 insert_subtitle.py --av audio_dir/audio_ts.m3u8 --subtitle subtitle_dir/subtitle.m3u8 --output output_dir/ --overwrite

      If ffmpeg/ffprob are not in your PATH, use -ffmpeg_path to specify the path.

    3. Play the synchronized files:

      1. Run the following command to start the HTTP server:


        _1
        python3 -m http.server --bind 127.0.0.1 -doutput_dir

      2. In your browser, enter the following URL:


        _1
        http://127.0.0.1:8000/player_demo.html

    Supported OSS vendors​

    vendor: Number. The third-party cloud storage vendor. The following are supported:

    region: Number. The region information specified for the third-party cloud storage. The service only supports regions in the following lists:

    • Amazon S3 (vendor = 1):

      • 0: US_EAST_1
      • 1: US_EAST_2
      • 2: US_WEST_1
      • 3: US_WEST_2
      • 4: EU_WEST_1
      • 5: EU_WEST_2
      • 6: EU_WEST_3
      • 7: EU_CENTRAL_1
      • 8: AP_SOUTHEAST_1
      • 9: AP_SOUTHEAST_2
      • 10: AP_NORTHEAST_1
      • 11: AP_NORTHEAST_2
      • 12: SA_EAST_1
      • 13: CA_CENTRAL_1
      • 14: AP_SOUTH_1
      • 15: CN_NORTH_1
      • 16: CN_NORTHWEST_1
      • 17: US_GOV_WEST_1
      • 20: AP_NORTHEAST_3
      • 21: EU_NORTH_1
      • 22: ME_SOUTH_1
      • 23: US_GOV_EAST_1
      • 24: AP_SOUTHEAST_3
      • 25: EU_SOUTH_1
      • 28: IL_CENTRAL_1

    • Alibaba Cloud (vendor = 2):

      • 0: CN_Hangzhou
      • 1: CN_Shanghai
      • 2: CN_Qingdao
      • 3: CN_Beijing
      • 4: CN_Zhangjiakou
      • 5: CN_Huhehaote
      • 6: CN_Shenzhen
      • 7: CN_Hongkong
      • 8: US_West_1
      • 9: US_East_1
      • 10: AP_Southeast_1
      • 11: AP_Southeast_2
      • 12: AP_Southeast_3
      • 13: AP_Southeast_5
      • 14: AP_Northeast_1
      • 15: AP_South_1
      • 16: EU_Central_1
      • 17: EU_West_1
      • 18: EU_East_1
      • 19: AP_Southeast_6
      • 20: CN_Heyuan
      • 21: CN_Guangzhou
      • 22: CN_Chengdu
      • 23: CN_Nanjing
      • 24: CN_Fuzhou
      • 25: CN_Wulanchabu
      • 26: CN_Northeast_2
      • 27: CN_Southeast_7

      For details, see Alibaba Cloud documentation.

    • Tencent Cloud (vendor = 3):

      • 0: AP_Beijing_1
      • 1: AP_Beijing
      • 2: AP_Shanghai
      • 3: AP_Guangzhou
      • 4: AP_Chengdu
      • 5: AP_Chongqing
      • 6: AP_Shenzhen_FSI
      • 7: AP_Shanghai_FSI
      • 8: AP_Beijing_FSI
      • 9: AP_Hongkong
      • 10: AP_Singapore
      • 11: AP_Mumbai
      • 12: AP_Seoul
      • 13: AP_Bangkok
      • 14: AP_Tokyo
      • 15: NA_Siliconvalley
      • 16: NA_Ashburn
      • 17: NA_Toronto
      • 18: EU_Frankfurt
      • 19: EU_Moscow

    • Kingsoft Cloud (vendor = 4):

      • 0: CN_Hangzhou
      • 1: CN_Shanghai
      • 2: CN_Qingdao
      • 3: CN_Beijing
      • 4: CN_Guangzhou
      • 5: CN_Hongkong
      • 6: JR_Beijing
      • 7: JR_Shanghai
      • 8: NA_Russia_1
      • 9: NA_Singapore_1

    • Microsoft Azure (vendor = 5): The region parameter has no effect, whether set or not.

    • Google Cloud (vendor = 6): The region parameter has no effect, whether set or not.

    • Huawei Cloud (vendor = 7):

      • 0: CN_North_1
      • 1: CN_North_4
      • 2: CN_East_2
      • 3: CN_East_3
      • 4: CN_South_1
      • 5: CN_Southwest_2
      • 6: AP_Southeast_1
      • 7: AP_Southeast_2
      • 8: AP_Southeast_3
      • 9: AF_South_1
      • 10: SA_Argentina_1
      • 11: SA_Peru_1
      • 12: NA_Mexico_1
      • 13: SA_Brazil_1
      • 14: LA_South_2
      • 15: SA_Chile_1
    vundefined