The Hardware Composer HAL (HWC) is used by SurfaceFlinger to composite surfaces to the screen. The HWC abstracts objects such as overlays and 2D blitters and helps offload some work that would normally be done with OpenGL.
Android 7.0 includes a new version of HWC (HWC2) used by SurfaceFlinger to talk to specialized window composition hardware. SurfaceFlinger contains a fallback path that uses the 3D graphics processor (GPU) to perform the task of window composition, but this path is not ideal for a couple of reasons:
As the physical display hardware behind the Hardware Composer abstraction layer can vary from device to device, it's difficult to give recommendations on specific features. In general, use the following guidance:
The general recommendation is to implement a non-operational HWC first; after the structure is complete, implement a simple algorithm to delegate composition to the HWC (for example, delegate only the first three or four surfaces to the overlay hardware of the HWC).
Focus on optimization, such as intelligently selecting the surfaces to send to the overlay hardware that maximizes the load taken off of the GPU. Another optimization is to detect whether the screen is updating; if it isn't, delegate composition to OpenGL instead of the HWC to save power. When the screen updates again, continue to offload composition to the HWC.
Prepare for common use cases, such as:
These use cases should address regular, predictable uses rather than edge cases that are rarely encountered (otherwise, optimizations will have little benefit). Implementations must balance two competing goals: animation smoothness and interaction latency.
HWC2 provides a few primitives (layer, display) to represent composition work and its interaction with the display hardware.
A layer is the most important unit of composition; every layer has a set of properties that define how it interacts with other layers. Property categories include the following:
A display is another important unit of composition. Every layer can be present only on one display. A system can have multiple displays, and displays can be added or removed during normal system operations. This addition/removal can come at the request of the HWC device (typically in response to an external display being plugged into or removed from the device, called hotplugging), or at the request of the client, which permits the creation of virtual displays whose contents are rendered into an off-screen buffer instead of to a physical display.
HWC2 provides functions to determine the properties of a given display, to switch between different configurations (e.g., 4k or 1080p resolution) and color modes (e.g., native color or true sRGB), and to turn the display on, off, or into a low-power mode if supported.
In addition to layers and displays, HWC2 also provides control over the hardware vertical sync (VSYNC) signal along with a callback into the client to notify it of when a vsync event has occurred.
In this section and in HWC2 header comments, HWC interface functions are
referred to by lowerCamelCase names that do not actually exist in the interface
as named fields. Instead, almost every function is loaded by requesting a
function pointer using getFunction
provided by
hwc2_device_t
. For example, the function createLayer
is a function pointer of type HWC2_PFN_CREATE_LAYER
, which is
returned when the enumerated value HWC2_FUNCTION_CREATE_LAYER
is
passed into getFunction
.
For detailed documentation on functions (including functions required for every HWC2 implementation), refer to the HWC2 header.
Layers and displays are manipulated by opaque handles.
When SurfaceFlinger wants to create a new layer, it calls the
createLayer
function, which then returns an opaque handle of type
hwc2_layer_t
. From that point on, any time SurfaceFlinger wants to
modify a property of that layer, it passes that hwc2_layer_t
value
into the appropriate modification function, along with any other information
needed to make the modification. The hwc2_layer_t
type was made
large enough to be able to hold either a pointer or an index, and it will be
treated as opaque by SurfaceFlinger to provide HWC implementers maximum
flexibility.
Most of the above also applies to display handles, though handles are created
differently depending on whether they are hotplugged (where the handle is passed
through the hotplug callback) or requested by the client as a virtual display
(where the handle is returned from createVirtualDisplay
).
Once per hardware vsync, SurfaceFlinger wakes if it has new content to composite. This new content could be new image buffers from applications or just a change in the properties of one or more layers. When it wakes, it performs the following steps:
Steps 1 and 2 have some nuances (such as deferred transactions and presentation timestamps) that are outside the scope of this section. However, step 3 involves the HWC interface and is detailed below.
At the beginning of the composition process, SurfaceFlinger will create and
destroy layers or modify layer state as applicable. It will also update the
layers with their current contents, using calls such as
setLayerBuffer
or setLayerColor
. After all layers have
been updated, it will call validateDisplay
, which tells the device
to examine the state of the various layers and determine how composition will
proceed. By default, SurfaceFlinger usually attempts to configure every layer
such that it will be composited by the device, though there may be some
circumstances where it will mandate that it be composited by the client.
After the call to validateDisplay
, SurfaceFlinger will follow up
with a call to getChangedCompositionTypes
to see if the device
wants any of the layers' composition types changed before performing the actual
composition. SurfaceFlinger may choose to:
acceptDisplayChanges
, which has the same effect as
changing the composition types as requested by the device and re-validating
without actually having to call validateDisplay
again.In practice, SurfaceFlinger always takes the latter path (calling
acceptDisplayChanges
) though this behavior may change in the
future.
At this point, the behavior differs depending on whether any of the layers
have been marked for client composition. If any (or all) layers have been marked
for client composition, SurfaceFlinger will now composite all of those layers
into the client target buffer. This buffer will be provided to the device using
the setClientTarget
call so that it may be either displayed
directly on the screen or further composited with layers that have not been
marked for client composition. If no layers have been marked for client
composition, then the client composition step is bypassed.
Finally, after all of the state has been validated and client composition has
been performed if needed, SurfaceFlinger will call presentDisplay
.
This is the HWC device's cue to complete the composition process and display the
final result.
While the HWC2 interface is quite flexible when it comes to the number of displays in the system, the rest of the Android framework is not yet as flexible. When designing a HWC2 implementation intended for use on Android 7.0, there are some additional restrictions not present in the HWC definition itself:
While the SurfaceFlinger operations described above are performed per-display (eventual goal is to be able to composite displays independently of each other), they are currently performed sequentially for all active displays, even if only the contents of one display are updated.
For example, if only the external display is updated, the sequence is:
// Update state for internal display // Update state for external display validateDisplay(<internal display>) validateDisplay(<external display>) presentDisplay(<internal display>) presentDisplay(<external display>)
Synchronization (sync) fences are a crucial aspect of the Android graphics system. Fences allow CPU work to proceed independently from concurrent GPU work, blocking only when there is a true dependency.
For example, when an application submits a buffer that is being produced on the GPU, it will also submit a fence object; this fence signals only when the GPU has finished writing into the buffer. Since the only part of the system that truly needs the GPU write to have finished is the display hardware (the hardware abstracted by the HWC HAL), the graphics pipeline is able to pass this fence along with the buffer through SurfaceFlinger to the HWC device. Only immediately before that buffer would be displayed does the device need to actually check that the fence has signaled.
Sync fences are integrated tightly into HWC2 and organized in the following categories:
setLayerBuffer
and setClientTarget
calls. These
represent a pending write into the buffer and must signal before the HWC client
or device attempts to read from the associated buffer to perform composition.
presentDisplay
using the getReleaseFences
call and are passed back to the
application along with buffers that will be replaced during the next
composition. These represent a pending read from the buffer, and must signal
before the application attempts to write new contents into the buffer.presentDisplay
and represent when the composition of this frame
has completed, or alternately, when the composition result of the prior frame is
no longer needed. For physical displays, this is when the current frame appears
on the screen and can also be interpreted as the time after which it is safe to
write to the client target buffer again (if applicable). For virtual displays,
this is the time when it is safe to read from the output buffer.The meaning of sync fences in HWC 2.0 has changed significantly relative to previous versions of the HAL.
In HWC v1.x, the release and retire fences were speculative. A release fence for a buffer or a retire fence for the display retrieved in frame N would not signal any sooner than frame N + 1. In other words, the meaning of the fence was "the content of the buffer you provided for frame N is no longer needed." This is speculative because in theory SurfaceFlinger may not run again after frame N for an indeterminate period of time, which would leave those fences unsignaled for the same period.
In HWC 2.0, release and retire fences are non-speculative. A release or
retire fence retrieved in frame N will signal as soon as the content of the
associated buffers replaces the contents of the buffers from frame N - 1, or in
other words, the meaning of the fence is "the content of the buffer you provided
for frame N has now replaced the previous content." This is non-speculative,
since this fence should signal shortly after presentDisplay
is
called as soon as the hardware presents this frame's content.
For implementation details, refer to the HWC2 header.