[BUG] Full screen playback is not retained when auto-play to next canvas

[elynema]

Description

As of 7.6 in Avalon if a user started playback on a section on the media item page and entered full-screen mode, the full-screen would be retained when advancing to the next section.

This is no longer happening. Instead, at the end of the section, the browser automatically exits full-screen and playback on the next section begins in the browser page, not in full-screen mode.

To Reproduce

1. Start playback of a multi-section video item
2. Enter full-screen mode
3. Allow the video to automatically advance to the next section

It is expected behavior that the browser should remain in full screen mode.

For QA

• test transitions audio to video and video to audio
• check that all custom components load properly with canvas change
• test transitioning to and from inaccessible items in playlist / media object page
• check sticky quality selector and volume when transitioning between canvases
• test cookbook example with multiple files in one canvas
• test CC value changes between canvases
• Test values in Transcript / Files tabs when switching canvases
• test that zoom scrubber works after transitioning canvases / playlist items
• test these transitions with prev/next canvas button as well as selecting section / timespan in structured navigation
• Test markers and source item display changes when switching playlist items
• test that active segment / section markers transition when changing canvas / playlist item
• Test transition when encountering an inaccessible item in full screen
• Caption button shows in player both in playlist context and in media item (even if the first file doesn't have captions and the second file does)

Question

• when in full screen mode, and playlist reaches an inaccessible item, what should happen? What used to happen?
  Reasonable first pass behavior is for full screen to exit, go back to playlist/media object page, and then advance to next item not in full screen after 10 seconds.

Ideally, we could retain full screen with the error message showing in full screen and a countdown timer.

[joncameron]

This could be a big change because we rely on a lot of communication from Video.js. For this to be implemented we need to retain the Video.js instance; this could break other parts in the process.

[elynema]

@Dananji Jon and I were interested in bringing this into 7.7.1, but we weren't quite clear on the scope of these code changes. If you are reloading the source of the player, rather than rebuilding it, on every canvas (suggested in #408), does that mean it could potentially cause bugs in all the different bits of the interface that update with canvas changes (tabs, add to playlist, create timeline, create thumbnail, etc)? What do you think are the places where the code touches other functionality that could break and would need to be tested? Or is there a different way you would approach this issue?

Do you think this type of infrastructure change would help with the blinking of section titles and the page jumping every time you click on a section? Those are both pretty annoying.

[Dananji]

All the other components except for MetadataDisplay, SupplementalFiles, and AutoAdvanceToggle components get refreshed on Canvas changes. And there are 2 ways for the Canvas to change, via the player (previous/next buttons or playback end event) or via StructuredNavigation component.
When implementing the functionality in this ticket, we will have to check all the components that get updated with the Canvas to ensure they all function as expected.
Hope this helps.

[elynema]

Dananji working on getting custom components and tracks to reload on canvas change. When the player refreshes, it is supposed to take in new info for the build, but we have been injecting the tracks into the HTML, and video.js isn't seeing those injected tracks. Custom components are supposed to load on the player ready event, but need to change listeners to listen for metadata loaded event?

[elynema]

Dananji also testing playlist manifest to see if there is anything else that needs to be done there.

[joncameron]

QA resulted in the expected behavior for all of the cases listed!

I created #501 for the issue of exiting fullscreen when switching to an inaccessible item.

[charumitraravi]

Verified this issue on media page and playlist page on web, android, iphone and iPad (portrait and landscape). Full screen playback is working as expected.While testing playlists, I was no longer able to see the captions icon. In iOS devices, there is a black placeholder for captions which does'nt seem to go away despite turning off native captions [refer screenshot]. This issue is'nt happening on avalon-staging.

[elynema]

@Dananji Charu found a couple of significant issues with testing that are probably unrelated to this issue, but maybe not because this issue had a lot of scope. I'm wondering if the first one should be resolved here and #3 go onto Ramp #473?

1. The CC button is not showing in playlists. The captions themselves are displaying, but no CC button in the player to turn them on/off. It's working fine on media items.

2. I'm not able to replicate the issue with the captions overlay box remaining after captions are turned off on iPhone iOS17. @charumitraravi Do you have any other details about that that might help us replicate?

3. I am seeing oddities with captions in the native player on iOS Safari. Mostly the issues seem to be that sometimes the first selection from the subtitles menu doesn't work and then the clicks after that may be out of sync or not working. This is happening both for media items and playlists. Is this related to Ramp Unusual Behavior For Captions in iOS Native Player #473. Should things related to captions and iOS native player be reported there?

[Dananji]

Yes, the first issue seem to be related to the work done in this ticket. It seems the CC button is not getting re-populated with canvas change in playlist context.
For the other 2 issues, I think we could create new tickets as needed since they seem to be scoped to iOS environment.

[charumitraravi]

@elynema I was not able to find a specific scenario where it happened, since I was testing different playlists, media and orientations and the captions overlay box started appearing on the items after the 4th playlist item in this playlist. I am able to replicate it when I try to move the slider forward in the video. (Although doesn't seem to happen. consistently now)

These are some of the other observations I made: the captions button does seem to appear on the media player on the playlists page once in a while, although I couldn't make out a pattern around it. And the iOS native player seems to be a bit slow to respond, and gets stuck often, specifically during orientation changes and slider movements to advance the video.

[elynema]

@Dananji I will pull this one back to in progress for resolving the issue with the CC button not showing in the playlist context. I will add the other issues to the iOS specific ticket.

It is concerning to hear that iOS native player is struggling again. I know we did some work to improve that significantly in 7.7.0. Is it possible some of those optimizations were lost when consolidating the player listener code in Ramp?

[elynema]

Tested that the captions button is working properly in playlist context on Chrome and Safari in desktop. Need to test in Android and iOS.

[elynema]

Verified on iPhone, iPad, and Android that the advance and captions buttons seem to be working.