Figure 1: Low delay encoding inter frame prediction structures using 2 refs (a) without LTR and (b) with LTR Adaptive LTRĪdaptive LTR is a content analysis based intelligent feature which automatically turns on LTR frame prediction structure based on scene characteristics, automatically decides which frames to assign as LTR, and has an advanced bitrate controller to adaptively assign bits & quality across a scene to maximize LTR prediction efficiency.Īdaptive LTR supports different predict structures in Media SDK, the "IPPP." structure we showed in Figure 1 is a example, it is configured by Target Usage 4(TU4, the default setting by MSDK). The prediction structure with LTR has two advantages at the decoder side, first the decoding algorithm always references to LTR to have a better image quality secondly the structure with LTR always references to the previous frame and LTR which saves the memory space for caching the extra frame. Figure 1 below shows low delay encoding inter frame prediction structures with and without using LTR. This is called Adaptive Long Term Reference.īefore we introduce the details, let's look at how LTR affects the prediction structure. Effective use of LTR frames requires detecting such stable content, finding the correct frame for LTR assignment, encoding the LTR frame with high quality and turning off LTR for unsuitable content. A Long-Term Reference allows, for example, encoding scene background with high quality for better motion compensated prediction in many future frames. This is why Intel ® Media SDK chooses LTR as the new feature for its hardware based codec solution. For example, Cisco has been using LTR as the error correction algorithm for their video conference application for a long time. This adds the flexibility to improve the encoding efficiency by combining the other tools to do a better decision of reference frame change according to this, many algorithms with LTR are developed. LTR frame has the advantage of being controlled by the encoding process at the application level. If the video scenario has a stable image scene, the reference frame could be kept longer while decoding more frames, which will help avoid transferring another reference frame in case of the stable scene and hence save the transfer bandwidth. Reference frames encoded with higher quality could improve image quality for the follow up frames. Long-Term Reference (LTR) frames can be saved and referenced until explicitly removed by the application. The standard defines short-term and long-term reference frame. These frames are called reference frames. The algorithm applies a prediction model to generate a new frame by shifting the macro-blocks or sub-blocks from the other frames. H.264 standard offers motion compensation prediction algorithm to reach this goal. To improve encoding capability, the challenge is to increase compression efficiency while preserving the original video quality. We achieved up to 20% improved efficiency by adding long-term reference and increasing I-frame quality for low motion content.
Media SDK introduces the Adaptive Long Term Reference (LTR) in the 2018R1 Windows release, an intelligent encoding feature which significantly improves compression efficiency and video quality for video conferences, surveillance and certain graphics/game streaming applications. Intel® Media SDK has been working hard on improving this capability based on our hardware. In media application, one important feature of the encoder is to lower the bitrate while keeping the high video quality.