Detailed analysis of flutter window initialization and drawing process

Preface

Environment: flutter sdk v1.7. 8+hotfix.3 @ stable

Corresponding to flutter engine: 54ad777f

What we focus on here is the drawing process at C + + level, how the platform drives and responds to the drawing and rendering process, not the rendering of Dart part.

Combined with the previous analysis, an important method is called in the constructor of the virtual machine instance DartUI::InitForGlobal() , call the process and list it again under 1:

DartVMRef::Create
 DartVMRef::DartVMRef
 DartVM::Create
 DartVMData::Create
 DartVM::DartVM
  DartUI::InitForGlobal()

The implementation body is very clear, and the methods of various class objects are registered, that is to say, these classes that inherit NativeFieldWrapperClass2 in dart language have one implementation in C + + layer, which also shows how DartSDK provides interface binding with C + + layer implementation, which is equivalent to jni in java language.

There is also initialization for Isolate, but only one path that can be import is set, which is not important:

DartIsolate::CreateRootIsolate
 DartIsolate::CreateDartVMAndEmbedderObjectPair
 DartIsolate::LoadLibraries
 DartUI::InitForIsolate
 Dart_SetNativeResolver

Viewport setting

We know that RuntimeController holds an instance of Window. See what happened after the Window instance was created:

RuntimeController::RuntimeController
 Window::Window
 DartIsolate::CreateRootIsolate
 DartIsolate::DartIsolate
 DartIsolate::SetWindow => UIDartState::SetWindow
  WindowClient::UpdateIsolateDescription => RuntimeController::UpdateIsolateDescription
  RuntimeDelegate::UpdateIsolateDescription => Shell::UpdateIsolateDescription
   ServiceProtocol::SetHandlerDescription
 Window::DidCreateIsolate
 library_.Set("dart:ui")
 RuntimeController::FlushRuntimeStateToIsolate
 RuntimeController::SetViewportMetrics
  Window::UpdateWindowMetrics
  library_, _updateWindowMetrics

The operation is directly transferred from the innermost Window1 to Shell. The most important function is to initialize ViewPort (viewport: used as size information such as canvas size and resolution), and then how to set ViewPort after being initialized:

FlutterView.onSizeChanged
 FlutterView.updateViewportMetrics
 FlutterJNI.setViewportMetrics
  FlutterJNI.nativeSetViewportMetrics
  ::SetViewportMetrics
  AndroidShellHolder::SetViewportMetrics
   [async:ui]Engine::SetViewportMetrics
   RuntimeController::SetViewportMetrics
    Window::UpdateWindowMetrics
   Engine::ScheduleFrame

From Java to C + +, FlutterView.onSizeChanged This operation was called by the system after the creation of the FlutterView instance (while the creation of FlutterView occurred at the time of Activity. onCreate), which obviously responded to the notification from the platform layer, which was in line with our cognitive expectations, because the size of the canvas may change due to user actions, and the dart layer needs to respond passively.

Note that in response to onSizeChanged on the Platform thread, call Engine::SetViewportMetrics Cut to the UI thread, bearing in mind that all operations of Engine are on the UI thread.

Start drawing frames

Engine calls another important method, ScheduleFrame, after setting the size of the window through RuntimeController, so look at its implementation:

Engine::ScheduleFrame
 Animator::RequestFrame
 [async:ui]Animator::AwaitVSync
  VsyncWaiter::AsyncWaitForVsync
  callback_= {Animator::BeginFrame}
  VsyncWaiter::AwaitVSync => VsyncWaiterAndroid::AwaitVSync
   [async:platform]FlutterJNI.asyncWaitForVsync
   AsyncWaitForVsyncDelegate.asyncWaitForVsync => VsyncWaiter.asyncWaitForVsyncDelegate
    Choreographer.getInstance().postFrameCallback
  Delegate::OnAnimatorNotifyIdle => Shell::OnAnimatorNotifyIdle
  Engine::NotifyIdle

Notification VSync

The operation here is a bit messy. First, cut to the UI thread, and then cut to the Platform thread. In fact, it is to call the platform interface and find out the ultimate goal.
Finally, the key classes Animator and VSyncWaiter needed to draw images are involved:

The UI thread waits for the VSync signal, indicating that it executes after the signal arrives Animator::BeginFrame Methods; How to set the VSync signal? By calling the platform interface, all platform operations must be in the Platform thread, so cut from the UI thread to the Platform thread in order to call the android Choreographer.postFrameCallback Which performs another string of tuning from C + + to java.

Response VSync

Because the VSync callback is called at the java layer, it can only be responded at the Java layer first, so there are:

FrameCallback.doFrame <= VsyncWaiter.asyncWaitForVsyncDelegate
 FlutterJNI.nativeOnVsync
 VsyncWaiterAndroid::OnNativeVsync
  VsyncWaiterAndroid::ConsumePendingCallback
 VsyncWaiter::FireCallback
  [async:ui]callback() => Animator::BeginFrame

After the VSync signal arrives, the Animator:: BeginFrame thread finally responds at UI and looks at its implementation:

Animator::BeginFrame
 Animator::Delegate::OnAnimatorBeginFrame => Shell::
 Engine::BeginFrame
  Window::BeginFrame
  library_."_beginFrame" => hooks.dart:_beginFrame
   UIDartState::FlushMicrotasksNow
   tonic::DartMicrotaskQueue::RunMicrotasks
  library_."_drawFrame" => hooks.dart:_drawFrame

Finally, we return to the dart layer and call its two important methods, _ beginFrame and _ drawFrame, to complete the frame rendering.

VSync Creation

In addition, the creation timing of VSyncWaiter under 1 is listed:

Shell::CreateShellOnPlatformThread
 PlatformView::CreateVSyncWaiter => PlatformViewAndroid::CreateVSyncWaiter
 VsyncWaiterAndroid()
 Animator::Animator
 Engine::Engine

It is the same time as creating Shell, that is, when the Platform thread is created by PlatformView::CreateVSyncWaiter Created and held by Animator, and Animator is held by Engine. VSyncWaiter is like Engine1, and all operations must be performed in UI threads

Window rendering

The rendering of the window is done by Window of Dart layer, which actually calls the implementation of C + + layer:

("Window_render", Render)
 Render() (window.cc:30)
 Scene=
 WindowClient::Render
  Scene::takeLayerTree
  RuntimeDelegate::Render => Engine::Render
  ProducerContinuation::Complete(layer_tree)
  Animator::Delegate::OnAnimatorDraw => Shell::OnAnimatorDraw(layer_tree_pipeline_)
  [async:gpu]Rasterizer::Draw => android_shell_holder.cc:76
   Rasterizer::DoDraw
   Rasterizer::DrawToSurface
    Surface::AcquireFrame
    ExternalViewEmbedder::BeginFrame
    CompositorContext::AcquireFrame
    ScopedFrame::Raster
    SurfaceFrame::Submit
    ExternalViewEmbedder::SubmitFrame
    FireNextFrameCallbackIfPresent
   Rasterizer::Delegate::OnFrameRasterized

"Window_scheduleFrame", ScheduleFrame

There are more objects involved here, and they are closely related to the rendering and rendering mechanism of Dart layer. It is worth noting that the specific drawing operation (rasterization) is performed on the GPU thread.

In addition, Window of Dart layer also needs active scheduling frames, so ScheduleFrame method is also bound.

Summarize