Living Room Certification Tests

Changelog

2019-10-11

• Remove WebGL tests from branch 2020.

2019-09-18

• Fork Official 2020 Branch.

2019-06-03

• Move MSE Codec Tests out of MSE Conformance Tests.
• Move MSE Format support tests out of MSE Conformance Tests.
• Add Format Support Tests for AC3 (Dolby Digital) and EAC3 (Dolby Digital Plus) audio.
• Add tests for AV1 to the following:
  • MSE Conformance Tests
  • MSE Codec Tests
  • Format Support Tests
  • Playback Performance Tests (SFR up to 1080p24fps)

2019-05-28

• Change test behavior honoring noVp9 parameter.

2019-05-24

• Update names of manual test cases

2019-05-23

• Add SourceBuffer.changeType MSE Conformance Tests
• Add tests for 10-bit VP9 Profile 2 (HDR) including PQ and HLG

2019-05-17

• Change Cobalt User Agent test to strict build check

2019-05-16

• Break down the Playbackperf tests into smaller test suites.

2019-04-22

• Load scripts as async to prevent failure on pause/suspend

2019-04-18

• Switch the WebP tests from Modernizr to our own implementation.

2019-04-03

• Prevent version increase on subsequent runs of Cobalt User Agent test

2019-03-25

• Branch 2020 is forked. Available at https://ytlr-cert.appspot.com/2020/main.html

2019-03-15

• Add 1.75x Playback Performance Tests

2019-02-28

• Add splash screen to YTS

2019-02-14

• Fetch Cobalt buildid directly from googlesource

2019-02-01

• Update required Cobalt version to 19.lts.194831

2019-01-02

• Fix window.onerror bug

2018-12-20

• Fix test regression

2018-12-11

• Remove manual SSL test

2018-11-26

• Fix the bug of VideoSrcURLLength-mp4 in functional tests

2018-11-13

• Migrated isTypeSupported and Video / Audio tests from Functional Tests to Format Support Tests
>

2018-11-09

• Add automated Cookie tests 'Set Cookie' and 'Set Expired Cookie' in Functional Test
• Add isTypeSupported test for "decode-to-texture" parameter
• Add "decode-to-texture" parameter to Cobalt Spherical Tests
• Host manual Cookie test in YTS

2018-11-05

• Add Manual test links.
• Add YouTube deep links in Manual tests.
• Split the downloadable zip file into Source and Media.
• Bugs fixation.

2018-09-12

• Add responsive design so that the format would match for 720p, 1080p and 4k devices.
• Add font tests, cookie storage tests, User Agent tests to functional testsuite.
• Add Cobalt Spherical testsuite.
• Fix AudioContext.decodeAudioData and VideoSrcUrl bugs.

2018-08-20

• Add DOM CSS Tests, DOM Document Tests, DOM Miscellaneous Tests to branch 2019.

2018-08-18

• Add Functional Tests, CSS Conformance Tests, HTML DOM Element Tests to branch 2019.

2018-08-02

• Create 2019 branch corresponding to 2019 test suite.

2018-06-29

• 2019 tests have been branched from tip.

2018-06-26

• Updated Widevine multi media key test to have 16 key sessions in tip.
• Added test to validate media.currentTime to tip.
• Updated test names for createMediaElementSource test in 2018 and tip.
• Added TotalVideoFrames validation test to Playback Performance Tests in tip.

2018-04-12

• Updated to use a shorter duration media in AppendOpusAudio and AbortOpusAudio on 2018.

2017-09-22

• Added HE-AAC testcases for both explicit and implicit SBR.

2017-08-10

• Added Opus 5.1 testcase to 2018 and tip.

2017-07-20

• Support for video.getVideoPlaybackQuality() is now enabled for all performance tests.

2017-07-12

• 2018 and tip now have an EME test to validate the creation of multiple MediaKeySession objects.

2017-07-06

• Added playback performance tests to 2018 and tip. Tests validate decoder performance for all video formats at different playback rates.
• Added playbackRate timing tests to progressive test suite for 2018 and tip.

2017-05-31

• 2018 tests have been branched from tip.

2017-05-18

• Layout for test pages no longer uses tables.

2017-04-26

• Updated tip tests to resolve a specification change. MediaSource.duration is now not set below the highest presentation timestamp of any buffered coded frames. Frames are now explicitly removed.

2016-12-21

• Added WebGL performance tests to 2017 and tip.

2016-10-31

• Added EME initData validation tests for 2016, 2017, and tip.

2016-09-06

• 2017 tests have been branched from tip.

2016-09-03

• Added MSE Opus tests to tip.
• Added encrypted Opus test to tip.
• EME tests on tip now validate that key status is usable for all keys.

2016-08-31

• Fixed broken progressive tests on tip.
• Tip test AppendH264VideoOutOfOrder is now optional.

2016-08-26

• Fixed EME stream fetching logic for tip. Now we are only fetching 4 segments of each stream for aac, h264, and vp9.

2016-08-08

• Fixed mixed-content license requests issue for 2016 and Tip.

2016-08-02

• Fixed Tip EME bug where the tests were always selecting Widevine key system even when using PlayReady.

2016-07-20

• Reduced the number of audio segments appended to the audio buffer in 2016 and tip EME tests.
• Performance and Endurance tests have been removed for all test versions.
• Tip EME tests have been updated with Widevine and PlayReady tests using EME Spec from 04 February 2016.

2016-06-23

• Added tests for Widevine Aac, Widevine VP9, PlayReady H264, and PlayReady Aac to EME 2016.

2016-06-17

• Added Widevine H264 test to EME 2016.

2016-06-13

• Cleaned up EME Tests for 2016.
• Fixed a webm parser bug that excluded Tracks from the init segment.
• Fixed a bug in the msutil appendAt method that assumed a blocking appendBuffer call.

2016-04-28

• Cleaned up console logging.

2016-04-15

• Updated EventTimestamp test on Tip to be more spec compliant and Chrome compliant.

2016-02-24

• Removed PIFF uuid header from encrypted mp4 files.

2016-01-20

• Clear Key PSSH headers are now added for encrypted MP4 content in cases where the header is missing.

2016-01-13

• Resolved a bug where some tests were failing with VP9 media unexpectedly.

2015-12-11

• Resolved a bug where some tests were not using VP9 media as expected and were instead using H264 media. Tests now use VP9 media as expected.
• Updated DurationAfterAppendAudio and DurationAfterAppendVideo tests on 2015, 2016, and tip. Tests are now more MSE spec compliant and account for timing issues.

2015-10-27

• Update to MediaSource duration test to make it more compatible with webkit based browsers.
• Branched 2016 tests.
• Updated test runner to have more control over the configurations for each test version. Tip will now default to VP9.

2015-10-09

• VideoBufferSize and DASHLatency tests now handle QUOTA_EXCEEDED_ERROR_CODE in 2015 and tip.
• Updated MediaSourceDuration on 2015 and tip to be more MSE specification compliant.

2015-10-06

• Fix for webm parser that caused large segment sizes. Improved logging in webm parser.
• Progressive tests now support VP9.

2015-09-10

• Encrypted media tests now support VP9 on 2015.

2015-08-20

• Fixed PlayReadySupport test on 2015 to check for 'probably' when including the codec type and 'maybe' when excluding the codec type.
• Improvements to 2015 EME tests. Keys and kids are now directly associated with the specific stream definition. Cleaned up EME test code.

2015-08-03

• Fix to MediaSourceDuration test in 2015 and tip to assert that values are within 0.01 of the expected values.
• Improvements to StreamDef to allow tests that to use the format they require. This allows FrameGap and FrameOverlap to be executed in VP9 mode but with still using the H264 assets that it depends on.

2015-07-28

• Fixed a bug that caused a race condition on MediaElementTest on 2015 and tip.
• Clarified test result requirements for PlayReady and WideVine Support tests. It is now documented in 2015 that you must pass one of these tests.

2015-07-09

• Updated WidevineSupport EME test on 2015 and tip to more accurately validate the Widevine support. Now VP9 Widevine support is not tested unless WebM/VP9 is enabled.
• Updated AbortVideo and AbortAudio MSE tests on 2015 and tip to be event driven. This should resolve some timing issues with this test.

2015-06-16

• Updated DelayedAudio and DelayedVideo test cases to account for Chrome allowing 3 seconds of underflow when video starved.

2015-06-11

• Replaced WebM/VP9 Video1MB video with a video encoded with VP9 instead of VP8.
• Removed WebM vorbis audio files. Replaced the files with aac mp4 files.

2015-06-03

• Updated test-video-1MB.mp4 (Video1MB) to have a start time that starts at 0.0.

2015-05-15

• Updated EME tests on 2015 to use explicitly EME 0.1b.
• Removed key system redefining in encryptedMediaPortability.

2015-05-12

• Updated tests label to contain versions of the MSE and EME specs that are being targeted by the tests.

2015-03-31

• Branched EME tests from MSE conformance test suite into its own test suite for 2015 and tip.
• Added more control for the display and linking between test suites (performance, endurance, etc).

2015-03-26

• Update to performance tests on 2015 and tip to the latest MSE.
• Update to endurance tests on 2015 and tip to the latest MSE.

2015-02-13

• Fixed bad conditional check for dual implementations of MSE webkit prefixes on tip and 2015.
• Removed optional parameter from EME tests that have been moved into mandatory on tip.
• Added MSE version to the page heading.

2015-02-04

• Upgraded tip EME tests to latest draft specification.

2014-12-10

• Branched 2015 tests from tip. Tip will continue to be updated to the latest version of MSE and EME.
• File reorganization to make branching new versions easier.

2014-12-04

• Fixed session issue related to mediakey detection for EME tests on tip.
• Fixed race condition in PlaybackState test on tip.

2014-11-25

• WidevineSupport test on tip now validates for VP9.

2014-11-17

• Fixed issue with mediakey detection in EME tests on tip.
• Updated instructions page.

2014-11-12

• It is now easier to select 'webm/vp9'. Replaced dropdown selector with on/off switch. You can also enable webm/vp9 testing by appending '&enablewebm=true' to the url.
• WebMHandling test removed from tip. To validate VP9 playback with MSE run the tests with 'webm/vp9' set to 'on'.

2014-11-07

• Fixed format issue in DASHLatency test on tip. Reduced size of video content that is added to the source buffer.
• Fixed format issues in some tests as a result of VP9 migration. Specifically fixes failures in 'Seek' test on tip.
• Log output is now in reverse order.
• Branched performance tests into 0.5, 0.6, and Tip. Updated tip tests to use 'appendBuffer' instead of 'append'.

2014-10-08

• Tests are now checking for updating source buffers. This fixes issues with tests #42 and #48.

2014-09-04

• Fixed timeout issues that resulted from downloading large video files on slow connections (specifically with test #20).

2014-04-11

• Added initial webm/VP9/vorbis support into tip only, so ignore the switch in the 2013/2014 tests. Not all tests are working (the mp4-specific tests and some of the clearkey tests get a pass). Use the dropdown for webm. Nothing should be changed for legacy or 2014 tests.
• Note that vp9 testing isn't ready for the other tests (Performance, Progressive, Endurance).
• Abstracted out streams so that the tests can be toggled between mp4 and webm.
• Fixed issue with DASHLatency test not being, well, testing latency.
• Moved DASHLatency test below TimestampOffset* tests, as it will most likely fail if TimestampOffsetVideo test fails.
• Known bugs:
  • Existing bug with unprefixed API usage in tip + endurance/performance/progressive tests.
• TODO:
  • Add URL options to start the tests in mp4 or webm mode directly.
  • Fix tests to be more MVC.
  • Branch 2015 tests.

2014-01-27

• Remove revokeObjectURL leak fix for 0.5.

2014-01-16

• Fixed a leak issue involving create/revokeObjectURL.

2013-12-11

• Consolidated and documented EME portability files.
• Unified MSE backend between 0.6 and tip.
• Fixed an issue with shutdown code.
• General style cleanups.

2013-11-14

• Missed a teardown override removal for 0.6.

2013-11-13

• Changed SourceBufferGC to DASHLatency.
• Added watchdog behavior for timeouts. Now test timeout logic will check if there are outstanding XHR requests. If so, it will delay timeout by 15 seconds, and try the same logic again. Hopefully this will help clients with low bandwidth connections.
• Added webkit wrappers for source(open|close|ended) events in 0.6.
• Removed teardown overrides for various tests in 0.6 and tip. This will cause some browsers to fail (e.g. Chrome, but fixed in a later release).
• Renamed tot.html to tip.html, to be more consistent with naming schemes.
• Cosmetic changes for landing page.

2013-11-06

• Updated test 28 (VideoBufferSize) in 0.6/tot to be more relaxed in terms of its requirements.
• Added new test to test of SourceBuffer GC in Tip.
• Reorganized 0.6/tot tests into categories. Test numbers have shuffled and names shortened. It shouldn't be too difficult to match up.
• Reformated test view to be more compact. Color codes:
• Red: Fail
• Green: Pass
• Hollow Red: Optional test that failed
• Yellow: Running test
• Tuning cursor selection with the arrow keys.
• Internal code refactoring.
• Grouping all MSE/EME/Optional tests for better organization.
• Relaxed VideoBufferSize's test requirements to allow for GC at the beginning.

2013-10-30

• General site cleanup (no test changes).
• Fixed broken links.
• Consolidated changes page into main page (and removed old main page).

2013-10-25

• Fixed the links page.

2013-10-23

• Added test to determine if the CDM supports dual video keys. This is meant for LQ/HQ protected playback streams. This has been marked optional for 0.5 and 0.6.

2013-10-15

• Changed ClearKeyCancelKeyRequestPostAdd to listen to keyadded event.
• Changed ClearKeyCancelKeyRequestPostAdd to skip the InvalidStateError test if the CDM doesn't support session IDs.
• Added playback test to CanPlayClearKey, to make sure the browser can actually play ClearKey content.
• Added focus manager to links.html.
• Added ClearKey cancel key tests.
• Fixed some links in the links page.
• Fixed many bugs with tests and infrastructure.

2013-9-30

• Fixed broken tot.html tests.
• Fixed 0.6 tests.
• Added generateKeyRequest and addKey tests.

2013-9-16

• Segregated tests into 0.5 (legacy) and 0.6 (late 2013 and beyond).
• Stripped 0.5 polyfill from 0.6.

2013-3-10

• Add new command "exclude" to exclude particular tests. See instructions for more information.

2013-3-6

• Explicitly set the duration of the stream in seek related tests to ensure that their seek position is valid.

2013-1-9

• Put the tests used to check resource leaks into Endurance Tests.

2013-1-3

• Add a set of Performance Tests to measure the performance of individual implementation.

2012-12-12

• Add TestXHROpenState test to ensure that we can set responseType before open the request.
• Add a switch to enable stop on error during test. Add parameter "stoponfailure" to enable it via url. See instructions for more information.
• Add new command "loop" to turn on loop. See instructions for more information.
• Add new command "run" to run the tests. See instructions for more information. The commands 'runselected' and 'runselectedloop' are deprecated.

2012-12-06

• Make test for media.paused into a separate test instead of putting it inside SourceChain test.
• Add parameter "tests" to select only the tests listed. See instructions for more information.
• Simplify the ReadyState test to only check dimension and rename it to VideoDimension.

2012-12-04

• Add parameter "timeout" to override the default timeout. See instructions for more information.
• Add parameter "disable_log" to disable log output. See instructions for more information.
• Add test CannotPlayWidevine to ensure that no one incorrectly reports to be able to playback media protected by widevine.
• Add WebMHandling test to test if the implementation report its WebM support correctly.