Droidkaigi 2019 (part 2)

This is the second part in my series about my talk at DroidKaigi 2019. You can find the rest of them below:

• Part 1
• Part 2 (this post)
• Part 3
• Part 4

2. The proposal

I would dedicate this whole post to share about my proposal about this topic: "ExoPlayer in RecyclerView". The presentation I used at DroidKaigi is below:

2.1 Investigate the challenges

First, let's start by investigating our challenges. I would like to start from the fullscreen playback experience.

In fact, the experience including user open a dedicated playback screen for one item from the list, I would like to call it Single Player instead of Fullscreen.

Let's think about this experience. How many form factor of Single Player we have? Of course, it depends on how you design your App, but roughly thinking, there are 3 forms we can use to represent the Single Player:

• Same orientation Player
• Multi windows state Player
• Landscape Player

There may be more, for example Picture-In-Picture mode. But implement it is not trivial, I would like to spend other post for it.

Opening Single Player and back has the following challenges in term of UX:

- Opening Single Player has many chances for config changes
- Handle config changes manually (= add manifest entry)
  - Pros: no resource reloading, playback continuity done!
  - Cons: no adaptive layout, or manually apply using if/else/while… (not me (>_<))
  - Many config changes patterns = error prone
- Handle config changes automatically (= no manifest entry)
  - Pros: adaptive layout for each config
  - Cons/Unresolved: playback continuity

Let's take an example of Facebook app to see how the experience works in practice:

1. User from homescreen click to an item and open its Single Player in the same orientation.
2. User from homescreen, rotate the phone. It will open the Single Player in landscape mode automatically.
3. User rotate the phone from "same orientation Single Player". It will open the Single Player in landscape mode automatically.

We can see that in 2. and 3., there is chance for configuration change.

Above, we discussed about the form factor of a Single Player. Below is common flow to open one:

- Open single player on new Activity
- Open single player on Dialog (fullscreen, in same Activity)
- Open in Fragment by replacing current one (in same Activity).

Of course, there may be more, but it will end up asking us the same questions:

- How to open Single Player and keep the playback continuity?
- How to come back and keep the playback continuity?

So to summarize, our implementation should ends up answering the following questions:

• How we handle all the changes during opening Single Player and back from it?
• How we ensure playback continuity?

2.2 Principle (strategy) and Components

Next, let me talk about my Proposal, starting from the principles I would follow to design it:

- Balance UX and performance
  - 2019 devices has more RAM than my Mac
  - And more CPU cores too
  - UX gain requries Performance loss
- Avoid resource reloading as much as possible.
  - No reload on config change
- Well lifecycle control does the magic
  - Design lifecycle flow if need

To accomplish those principles, here is how I make it: (1) design a mechanism where an instance of our "playback" could survive across many changes, including configuration changes (the recreation of an Activity), as well as lifecycle change (from one Activity to another, or from a Fragment to another). (2) design a resource management mechanism such that, it would retain just enough amount of resource (eg: ExoPlayer instances). The definition of 'enough amount' would vary, below is mine:

- Max instance number = max number of Videos on screen at once
  - Play then Pause video + still on screen = holds an Player instance.
  - Play then Pause video + scrolled off screen → release Player instance to Global Pool.
  - Release video = instance is stored in Pool for reuse.

So, in this approach, the more videos you have on screen, the less performant the App will be.

One good point of my approach is that, it doesn't force the client to follow this principle always, but it allows for customization that just need to meet some certain requirements.

Below is my explanation for (1): a mechanism to keep the playback continuity.

First, we need some components to make it happens:

• Playable: a piece of resource, can be played. Example: an ExoPlayer instance
• Target: to which a Playable can be played on, the object to present the playback on. Eg: PlayerView (in ExoPlayer library)
• Playback: when a Playable is bound to a Target, it produces a Playback. Playback represents the connection of a Playable and Target

These components form a team and work together:

• Playable ⇆ Target binding is unique, but not required

  • Binding same target will produce same Playback instance.
  • Different Playables bind to one Target: the later wins.
  • Not in bound Playable will be cleaned up eventually.
• Playable must be bound to Target to be “noticed”
• Playback observes Target's behavior and trigger Playable

The inspiration behind this model is the blow line fro ExoPlayer

PlayerView.switchTargetView(player, null, playerView)

Because these components belong to a team, we need something to manage that team. And in our App, there will be many teams (many lifecycles), we also need something to manage those teams:

• Manager: manages Playback instances (also, acknowledge the Playable)

  • Ensure Playback uniqueness
  • One Playable can belong to at-most one Manager

• Global singleton: manages Managers and Playables

  • Also manage low layer resources like ExoPlayer instances, Factories
  • Observe lifecycles and dispatch actions to Managers

I just say that Playable is in a team managed by Manager, but in fact, this guy is like us, developers. We stay in this project and then move to other project on demand. Playable is what we use to create what I called Playback Continuity before. To demonstrate this, let's move to components' lifespan and a figure below:

Compoent's lifespan

• Components stay alive as long as possible

  • Playable: contains only resource for playback, stays in Application lifecycle
  • Target: View, stays in Activity/Fragment lifecycle
  • Playback: min(Target, Playable), stays in Activity/Fragment lifecycle
• Playable survives config changes

• Playable can survive lifecycle changes if need

  • eg: from Activity to Activity

• Manager, Playback: stay in Activity/Fragment lifecycle

  • Do not survive config changes

Component's lifecycle in action

With these components, the way we keep the playback continuity is as follow: by switching the Target of a Playable, we do not need to recreate it or reprepare it. So says our current Target A is in the list inside Fragment A, when we open single player for it in Fragment B (and the playback surface is Target B), we rebind the Playable from Target A to Target B. Done!

But wait, replacing Fragments will involve a lot of changes, including Fragment destroying and creating. So how our Playable survices those changes? Let move to cross-lifecycles.

Cross-lifecycles

Cross-lifecycles is the behavior when something happens in a lifecycle and keeps happening in another lifecycle started by current one. For example: a Video playing inside Activity, if this App using new Activity for single player, the Video should be playing after opening single player Activity from current one.

There are many patterns of cross-lifecycles:

• Starting Activity from current Activity (in this case, old Activity will be stopped).
• Replacing Fragment by another Fragment (in this case, old Fragment will have it View destroyed by default).
• Opening a DialogFragment from current Fragment (in this case, old Fragment is still active).
• Config change is a special scenario of cross-lifecycle, where the same Activity is recreated on demand.

In my approach, keeping Playable survive through cross-lifecycle is equal to keeping playback continuity. We investigate cross-lifecycles patterns below, and ses how our components work with them?

1. Configuration changes

As you can see from the figure: once config change happens, the Activity will tell it to client via the call isChangingConfigurations. Catching that value, we will know if we should keep our Playables survive or release them. In case of config change, if after the creation, the Activity still use the Playable, we keep it in Global cache, and rebind it after the recreation. That way, our Playale will just keep playing.

2. Starting Activity from Activity

There is no isChangingConfigurations signal in this case. Instead, we follow the lifecycle callback of each. Says from Activity A, the client calls startActivity to start single player in Activity B. The figure below shows how we handle this case:

Yeah it is. In this scenario: after calling startActivity from Activity A to open Activity B, first Activity B will be created, then started, after that Activity will be stopped and destroyed. So in Activity B's onStart or onCreate, it just needs to take the Playable from Activity A, this Playable will be unchanged, and our playback will keep playing continously.

3. Replacing Fragment by a Fragment

Consider the case: client open single player in Fragment B, using Activity's FragmentManager to replace current Fragment A with it. Because Fragment represents the same lifecycle sequence with Activity, so we hope the same timeline above will be true for Fragment.

By default, it is not. Fragment A will be destroyed first before the creation of Fragment B. During the gap between a destruction and a creation, we have no clue to rely on, to keep our Playable alive.

But, we are lucky. Latest support Fragment library brings the method setReorderingAllowed to live, which allow Fragment transaction to happen in the same way Activities' lifecycle. So we have the same methodology to keep Playable alive across FragmentTransaction:

4. Opening DialogFragment from Fragment/Activity

There is no destruction in this scenario. We can simple take the Playable to play inside the DialogFragment.

I will explain about taking the Playable in detail later.

Combine those scenarios

Combining those scenarios we discussed above, we could keep a Playable alive through various lifecycle transition, which in turns, keep our playback continuity:

Conclusion

As shown above, by understanding the cross-lifecycles behavior, and use it correctly, we can keep the playback play continuously, across various transition, including configuration changes.

In next part, I would like to show how my approach manage the playback resource, make it work together with the components I define in this part, and finally how we would combine all of them in our final goal: to make ExoPlayer works in RecyclerView with ease.