On this, half 2 on how you can cut back latency in media managed by Amazon Kinesis Video Streams (KVS) I describe the methods by which to configure KVS, the media producer and the media gamers for optimum latency settings. Lastly, I introduce the Amazon Kinesis Video Stream Internet Viewer and carry out various experiments on KVS to validate the latency figures you may count on to attain below a wide range of situations.
In half 1, I lined the basics of streaming media, KVS Operation, widespread design patterns and the main contributing elements to latency in streaming media managed by KVS.
Decreasing Latency of KVS Stay Media
The best technique of decreasing latency of streaming media in KVS are the next:
- Optimize KVS offered HLS Playlist settings for reside media
- Cut back the fragment size set by the KVS media producer
- The place supported by the media participant:
- Programmatically cut back fragment buffering by the media participant
- Set the playback pace to barely above x1 (i.e x1.05 / x1.1) to maintain up with the reside fringe of the media always
Remember that adjusting buffer and fragment settings for low-latency usually comes as a commerce off with video high quality and re-buffering, particularly throughout community instability. It’s beneficial to totally check utilized settings in opposition to the vary of anticipated networ.
- For strong reside video high quality with low-latency throughout a wide range of community situations:
- Present 5 – 10 fragments within the HLS Media Playlist and
- Use fragment size of 1 second.
- To aggressively tune for latency (the place community high quality and / or media participant helps):
- Present 3 fragments within the HLS Media Playlist,
- Use fragments size of 0.5 seconds and
- Programmatically restrict the media participant buffer to 2 – 3 fragments relying on community high quality,
- Set the media playback pace to barely above x1 (x1.05 / x1.1) in order to maintain up with the reside fringe of the media even throughout dropped fragments (this may trigger extra frequent re-buffering).
Ref: Latency too excessive between producer and participant
Optimizing KVS HLS Playlist Settings for Stay Media
The KVS Archived Media API helps the GetHLSStreamingSessionURL name to request the HLS Media Playlist and entry media fragments as proven within the following (simplified) sequence:
Determine 1 – KVS Request (HLS) Playlist URL course of stream.
When an utility requests the HLS Playlist URL from KVS, various settings can be found to optimize for low latency streaming media. The under instance is an easy python script that makes a request to the KVS GetHLSStreamingSessionURL API to return the URL of the HLS Media Playlist for the requested stream and settings:
import boto3
KVS_STREAM_NAME = '[ENTER_KVS_STREAM_NAME]'
REGION = '[ENTER_AWS_REGION]'
#=====================================================
# Create the AWS Kinesis Video consumer
kvs = boto3.consumer("kinesisvideo", region_name=REGION)
#=====================================================
# Get the KVS endpoint for this stream
endpoint = kvs.get_data_endpoint(
APIName="GET_HLS_STREAMING_SESSION_URL",
StreamName=KVS_STREAM_NAME)['DataEndpoint']
print(f'nKVS ENDPOINT: {endpoint}')
#=====================================================
# Get the KVS Archive media consumer
kvam = boto3.consumer("kinesis-video-archived-media", endpoint_url=endpoint, region_name=REGION)
# Get LIVE HLS URL
hls_playback_url = kvam.get_hls_streaming_session_url(
StreamName=KVS_STREAM_NAME,
PlaybackMode="LIVE",
MaxMediaPlaylistFragmentResults=3,
Expires=43200)['HLSStreamingSessionURL']
print(f'nLIVE HLS_STREAMING_SESSION_URL: {hls_playback_url}n')
The output of this script is the URL for the KVS Stream HLS Grasp Playlist (.m3u8) file. This URL might be handed immediately as a hyperlink in supporting browsers corresponding to Safari or to desktop media gamers like VLC to play the KVS media stream with the chosen parameters. The HLS URL is returned with a SessionToken that gives time restricted entry to the KVS media so watch out with the way you handle this URL.
Within the above code, there are two parameters specifically to contemplate:
- PlaybackMode: When set to LIVE, the HLS media playlist is frequently up to date with the most recent fragments as they develop into out there. It will usually show the reside notification within the media participant and in some gamers will choose preconfigured fragment buffer settings extra appropriate to low-latency media.
- MaxMediaPlaylistFragmentResults: The utmost variety of fragments which might be returned within the HLS media playlists. Limiting this may additionally restrict fragment buffering by the media participant and cut back latency. When the PlaybackMode is LIVE, the latest fragments are returned as much as this worth. When the PlaybackMode is ON_DEMAND , the oldest fragments out there are returned, as much as this most quantity.
Due to this fact; to cut back latency when requesting the KVS HLS Playlist URL, make sure the PlaybackMode is about to LIVE and it’s beneficial that MaxMediaPlaylistFragmentResults is about to no less than 3 for low latency with strong media high quality.
Cut back the Fragment Size Set by the KVS Media Producer
Fragment size of a media stream impacts latency because it determines the preliminary buffering and fragmentation time on the producer and the buffering time on the media participant. Fragment lengths are set on the KVS producer and are preserved by the KVS service. And so, the fragment size created by the producer is what will likely be obtained and buffered by the media participant or shopper. Under is an instance of how you can set the fragment size utilizing a easy GStreamer media pipeline to KVS.
Utilizing the GStreamer KVS Plugin:
The under GStreamer pipeline preforms a reside display seize on a MacOS machine (avfvideosrc plugin), encodes the uncooked video to H.246 frames (vtenc_h264_hw plugin) and passes the encoded stream to the KvsSink plugin which ingests it to KVS. Within the instance, the requested body fee from the digicam and the key-frame interval on the encoder are each 20 FPS leading to 1 second fragments.
gst-launch-1.0 avfvideosrc capture-screen=true
! videoconvert
! videoscale
! video/x-raw,width=1280,top=720,framerate=20/1
! vtenc_h264_hw allow-frame-reordering=FALSE realtime=TRUE max-keyframe-interval=20
! h264parse
! video/x-h264,stream-format=avc,alignment=au,profile=baseline
! kvssink stream-name=$STREAM_NAME aws-region=$REGION access-key=$ACCESS_KEY secret-key=$SECRET_KEY retention-period=$RETENTION_HRSTo scale back the fragments to 0.5 seconds, modify the key-frame interval to 10 by adjusting the max-keyframe-interval worth. With a key-frame interval of 10 and a frame-rate of 20 fps it will set off 0.5 second fragments by the kvssink factor. It’s additionally legitimate on this instance to extend the frame-rate to 40 fps by adjusting the framerate worth if it’s supported by the digicam however it will have a a lot larger impression on the community bandwidth and KVS consumption prices for the media stream.
Utilizing the KVS Producer Libraries:
If utilizing the KVS Producer Libraries, you because the developer are chargeable for the variety of frames per fragment and the fragment size utilizing the programmatic means out there to regulate key-frame interval and body fee ingested to KVS.
Programmatically Decreasing Media Participant Fragment Buffering
If creating a customized net or cellular utility for consuming KVS media, hottest media participant libraries assist programmatic tuning of settings such because the variety of fragments to buffer. An instance of a KVS built-in net utility is the Amazon Kinesis Video Streams Internet Viewer, an AWS offered code pattern you may host in your personal AWS account to check and eat KVS media.
KVS Internet Viewer Demo Utility:
Determine 2 – KVS Internet Viewer Utility Display screen Shot.
The KVS Internet Viewer makes use of the react-player library which calls on HLS.js library to play HLS primarily based media corresponding to offered from KVS. The KVS Internet Viewer exposes the entire programmatic settings out there within the hlsConfigs.js file. Particularly, the hlsConfig.js parameters liveSyncDurationCount and liveMaxLatencyDurationCount have been decreased from the defaults to optimize for decrease latency.
Utilizing this instance, you may deploy a KVS shopper net utility with HLS and buffer settings uncovered by way of a easy configuration file for check and improvement in your surroundings.
Setting the Media Playback Velocity to Barely Above x1
Media gamers will usually slip behind the reside fringe of the streaming media including to video latency. This may be resulting from slight imperfections in timing between media producer and shoppers or resulting from misplaced seconds throughout media re-buffering or community instability. As a result of this impact is accumulative, the impression is proportional to the size of the playback session and so the longer the media is anticipated to run in reside playback mode the upper the impression. Due to this fact, that is particularly related the place a media supply is displayed completely on reside monitor corresponding to for a safety digicam or comparable.
One approach to cut back latency being launched resulting from media participant falling behind the reside edge is to set the media playback pace to barely above x1 (i.e: x1.05 / 1.1 relying on media participant assist). This places the media participant into ‘Catch-Up’ mode that’s always attempting to out run the reside edge. This retains the media participant as near the reside edge as attainable however will in some instances trigger the media to overrun the present fragment which is able to end result within the media participant being uneven. As a result of if this, it’s solely beneficial to make use of this system for lengthy working or completely monitored media sources.
The KVS Internet Viewer mentioned helps each x1.05 and x1.1 playback speeds, use this as a method to check the elevated playback pace approach in your surroundings.
Validating KVS Latency Values
On this part, I’ll check the described KVS HLS settings, fragment lengths, the customized KVS Internet Viewer in comparison with direct viewing of a KVS stream within the Safari net browser and throughout world networks to validate latency values achievable below these particular situations.
Take a look at Set-Up:
This check makes use of the above described GStreamer pipeline to ingest a reside display seize from a MacOS machine to KVS. The Mac display shows a millisecond accuracy stopwatch and the identical KVS stream is consumed on a second monitor for comparability. The check is being carried out from Melbourne, Australia utilizing the AWS Sydney Area throughout a bodily distance of ~900Kms and subsequently an ~1800 Km spherical journey.
Determine 3 – KVS latency check set-up and distance.
Utilizing KVS Really useful Settings for Strong Video High quality:
HLS PlaybackMode: Stay
HLS MaxMediaPlaylistFragmentResults: 7
Fragment Size: 1 Second
Determine 4 – KVS latency check for strong video high quality outcomes.
Utilizing the KVS Internet Viewer with latency optimized buffer configuration, the beneficial KVS HLS settings for strong video high quality and 1-Second fragment size, latency was measured at 1.95 Seconds.
This precise check was run consuming the KVS HLS Playlist URL immediately within the Safari net browser and measured 4.072 seconds. This distinction is because of the truth that we will’t management the variety of fragments the Safari browser buffers and browsers are usually tuned for Video on Demand with media high quality being a precedence over low latency.
And eventually, this check was run once more with the KVS Internet Viewer however within the N.Virginia area, a spherical journey from Melbourne Australia of over 30,000Kms (20,000 mi) which measured 3.32 seconds finish to finish latency. This offers an approximate scale to the impression of distance on media latency. This may very well be decreased additional through the use of companies just like the AWS World Accelerator.
Utilizing KVS Latency Optimized Settings:
HLS PlaybackMode: Stay
HLS MaxMediaPlaylistFragmentResults: 5
Fragment Size: 0.5 Seconds
Determine 5 – KVS latency check for latency optimised settings outcomes.
By decreasing the Fragment size to 0.5 seconds we had been in a position to obtain an finish to finish latency of 1.412 seconds. Right here, as a result of we’re utilizing the KVS Internet Viewer that permits us to programmatically configure the variety of buffered fragments the MaxMediaPlaylistFragmentResults worth that may in any other case affect media participant buffer settings will not be a number one contributor.
Operating this identical check however consuming the KVS media immediately from the Safari net browser resulted in latency of three.55 seconds.
Conclusion
On this collection on how you can cut back finish to finish latency of media managed by Amazon Kinesis Video Streams, in half 1; you realized the basics of streaming media, KVS operation and customary design patterns and eventually the main contributing elements to latency of streaming media in KVS.
On this, half 2; you realized the means by which to cut back latency of streaming media by decreasing fragment lengths, media participant buffer dimension and how you can choose optimum HLS playlist settings when requesting a HLS URL from the KVS API for low latency media.
Following this, you examined beneficial settings for strong video high quality and latency optimized media streams utilizing the Amazon KVS Internet Viewer. You additionally in contrast Amazon KVS Internet Viewer to viewing the identical media stream immediately within the Safari net browser. These outcomes had been in comparison with geographically close to and distant AWS Areas to additionally validate the impression of distance on media latency.
Via this testing I used to be in a position to exhibit how you can obtain latency on KVS managed media of 1.95 seconds utilizing settings that ship strong video high quality over a wide range of community situations and latency of 1.412 seconds when latency optimized settings had been utilized.
Concerning the writer
 Dean Colcott is a Snr IoT and Robotics Specialist Answer Architect at Amazon Internet Providers (AWS) and a SME for Amazon Kinesis Video Streams. He has areas of depth in distributed purposes and full stack improvement, video and media, video analytics and pc imaginative and prescient, Industrial IoT and Enterprise knowledge platforms. Dean’s focus is to carry relevance, worth and perception from IoT, Industrial knowledge, video and knowledge producing bodily property inside the context of the Enterprise knowledge technique. |
