Android 12 - Letterbox mode

1. Introduction

With the emergence of more and more large screen and folding screen devices, many applications do not adapt the UI of devices of different sizes. At this time, applications choose to display with a specific aspect ratio (although Google does not recommend this, the official still hopes that developers can carry out adaptive layout for different screen sizes ~). When the aspect ratio of applications is incompatible with its container ratio, It opens in Letterbox mode.

In Letterbox mode, the interface will be displayed at the specified scale, and the surrounding blank area can be filled with wallpaper or color. As for the appearance of Letterbox, the following factors can affect it:

  • config_letterboxActivityCornersRadius: interface fillet size
  • config_letterboxBackgroundType: background filling type, including:
    • LETTERBOX_ BACKGROUND_ APP_ COLOR_ Backgroup: the color is affected by android:colorBackground
    • LETTERBOX_BACKGROUND_APP_COLOR_BACKGROUND_FLOATING: the color is affected by android:colorBackgroundFloating
    • LETTERBOX_BACKGROUND_SOLID_COLOR: color is controlled by config_letterboxBackgroundColor effect
    • LETTERBOX_BACKGROUND_WALLPAPER: display wallpaper, this option and flag_ SHOW_ Similar to wallpaper, it will cause the wallpaper window to display
  • config_letterboxBackgroundWallpaperBlurRadius: wallpaper blur
  • config_ Letterbox backgroundwallaberdarkscrim alpha: wallpaper darkening

2. When to trigger

Trigger conditions of Letterbox generally include:

  • When Android: resizableactivity = false and the aspect ratio declared by the application is incompatible with the container (for example, the screen width and height exceed android:maxAspectRatio)
  • setIgnoreOrientationRequest(true) after the system setting ignores the screen direction, open a forced vertical screen interface in horizontal screen mode

3. Implementation plan

The implementation of Letterbox display is not complicated. Android 12 adds LetterboxUiController in ActivityRecord to control the layout and display of Letterbox. Let's take a look at the state of SurfaceFlinger in Letterbox mode:

It can be seen that, compared with the normal situation, except that the size and position of the interface itself are scaled to the specified ratio, there are two layers around, which are hung under the actirecord node. These two layers can be filled with the specified color according to the configuration. If the background is wallpaper, you can also set the dim value and blur degree of the wallpaper, These can be easily implemented through the SurfaceControl interface.

Here is a brief analysis of the code:

// LetterboxUiController.java
void updateLetterboxSurface(WindowState winHint) {
    final WindowState w = mActivityRecord.findMainWindow();
    if (w != winHint && winHint != null && w != null) {
        return;
    }
    // Calculate the position of layers to be displayed around the interface
    layoutLetterbox(winHint);
    if (mLetterbox != null && mLetterbox.needsApplySurfaceChanges()) {
        // Perform creation, parameter setting and other operations on the Surface
        mLetterbox.applySurfaceChanges(mActivityRecord.getSyncTransaction());
    }
}

void layoutLetterbox(WindowState winHint) {
    final WindowState w = mActivityRecord.findMainWindow();
    if (w == null || winHint != null && w != winHint) {
        return;
    }
    updateRoundedCorners(w);
    updateWallpaperForLetterbox(w);
    // Key judgment of whether to enter Letterbox mode
    if (shouldShowLetterboxUi(w)) {
        if (mLetterbox == null) {
            // Delegate specific logic to Letterbox 
            mLetterbox = new Letterbox(() -> mActivityRecord.makeChildSurface(null),
                    mActivityRecord.mWmService.mTransactionFactory,
                    mLetterboxConfiguration::isLetterboxActivityCornersRounded,
                    this::getLetterboxBackgroundColor,
                    this::hasWallpaperBackgroudForLetterbox,
                    this::getLetterboxWallpaperBlurRadius,
                    this::getLetterboxWallpaperDarkScrimAlpha);
            mLetterbox.attachInput(w);
        }
        mActivityRecord.getPosition(mTmpPoint);
        // Get the bounds of the "space-to-fill". The transformed bounds have the highest
        // priority because the activity is launched in a rotated environment. In multi-window
        // mode, the task-level represents this. In fullscreen-mode, the task container does
        // (since the orientation letterbox is also applied to the task).
        final Rect transformedBounds = mActivityRecord.getFixedRotationTransformDisplayBounds();
        final Rect spaceToFill = transformedBounds != null
                ? transformedBounds
                : mActivityRecord.inMultiWindowMode()
                        ? mActivityRecord.getRootTask().getBounds()
                        : mActivityRecord.getRootTask().getParent().getBounds();
        // Location calculation
        mLetterbox.layout(spaceToFill, w.getFrame(), mTmpPoint);
    } else if (mLetterbox != null) {
        mLetterbox.hide();
    }
}
// Letterbox.LetterboxSurface.java
public void applySurfaceChanges(SurfaceControl.Transaction t) {
    if (!needsApplySurfaceChanges()) {
        // Nothing changed.
        return;
    }
    mSurfaceFrameRelative.set(mLayoutFrameRelative);
    if (!mSurfaceFrameRelative.isEmpty()) {
        if (mSurface == null) {
            // Create a Surface hanging under the ActivityRecord node and set it to ColorLayer type
            createSurface(t);
        }
        // Set color, position and clipping
        mColor = mColorSupplier.get();
        t.setColor(mSurface, getRgbColorArray());
        t.setPosition(mSurface, mSurfaceFrameRelative.left, mSurfaceFrameRelative.top);
        t.setWindowCrop(mSurface, mSurfaceFrameRelative.width(),
                mSurfaceFrameRelative.height());

        // Sets the transparency and blur of the wallpaper background
        mHasWallpaperBackground = mHasWallpaperBackgroundSupplier.get();
        updateAlphaAndBlur(t);

        t.show(mSurface);
    } else if (mSurface != null) {
        t.hide(mSurface);
    }
    if (mSurface != null && mInputInterceptor != null) {
        mInputInterceptor.updateTouchableRegion(mSurfaceFrameRelative);
        t.setInputWindowInfo(mSurface, mInputInterceptor.mWindowHandle);
    }
}

4. Summary

This article only briefly analyzes the trigger conditions and display logic of the lower Letterbox mode, but there are many details not covered. For example, for detailed trigger logic judgment, you can view the LetterboxUiController#shouldShowLetterboxUi method

Tags: Android

Posted on Wed, 24 Nov 2021 21:20:35 -0500 by dnszero