Tuesday, May 28, 2013

How Gpu process draws contents on the surface of Browser process's window in chromium.

This article explains how Gpu process draws in linux platform.
At the end of article, I'll mention briefly about aura and Android.

What do three processes do?

Gpu process draws contents on the surface of Browser process's window. Briefly, following steps achieve it.
  1. Browser process creates RenderWidgetHostView (GtkWidget)
  2. Browser process sends Window id of backend view (of RenderWidgetHostView)  to Gpu Process  
  3. Gpu Process creates GL context and GLX Surface bound in Window id. (Gpu Process will draw everything on the surface)
  4. Render Process paints GraphicsLayer tree on GraphicsContext3D.
  5. All gl commends that cc calls in Render process are replayed on GLX Surface bound in Window id in Gpu Process.
  6. Browser process draws non-client widgets (e.g. button, url field and tab button).

Initialize: how Gpu process knows render widget

FYI, surface id is just number of GtkWidget so that Browser process tracks surface id and compositing surface  (= GLSurfaceHandle including Window XID in X or EGLNativeWindowType in EGL) pair.

In Browser process, RenderViewHostImpl::CreateRenderView() sends surface_id to Render process

bool RenderViewHostImpl::CreateRenderView(
    const string16& frame_name,
    int opener_route_id,
    int32 max_page_id) {
  ...
  GpuSurfaceTracker::Get()->SetSurfaceHandle(
      surface_id(), GetCompositingSurface());
  ...
  params.surface_id = surface_id();
  ...
  Send(new ViewMsg_New(params));
}

In Render process, when cc is created, notify Browser process with surface id.

int32 RenderThreadImpl::CreateViewCommandBuffer(
      int32 surface_id, const GPUCreateCommandBufferConfig& init_params) {
  ...
  IPC::Message* message = new GpuHostMsg_CreateViewCommandBuffer(
      surface_id,
      init_params,
      &route_id);
  ...
}

In Browser process (IO thread), request GL context creation for render view to Gpu process.

Browser process sends XID of Window to Gpu process.
void GpuProcessHost::CreateViewCommandBuffer(
    const gfx::GLSurfaceHandle& compositing_surface,
    int surface_id,
    int client_id,
    const GPUCreateCommandBufferConfig& init_params,
    const CreateCommandBufferCallback& callback) {
  ...
  if (!compositing_surface.is_null() &&
      Send(new GpuMsg_CreateViewCommandBuffer(
          compositing_surface, surface_id, client_id, init_params))) {
 ...
}

content::GpuProcessHost::CreateViewCommandBuffer() at gpu_process_host.cc:735 0x602fe6 
content::GpuMessageFilter::OnCreateViewCommandBuffer() at gpu_message_filter.cc:222 0x881773 
DispatchToMethod{content::GpuMessageFilter, void () at tuple.h:738 0x883ed5 
IPC::SyncMessageSchema{Tuple2{int, GPUCreateCommandBufferConfig}, Tuple1{int&} }::DispatchDelayReplyWithSendParams{content::GpuMessageFilter, void () at ipc_message_utils.h:834 0x882f4d 
GpuHostMsg_CreateViewCommandBuffer::DispatchDelayReply{content::GpuMessageFilter, void () at gpu_messages.h:310 0x88215b 
content::GpuMessageFilter::OnMessageReceived() at gpu_message_filter.cc:91 0x880b3c 
content::BrowserMessageFilter::DispatchMessage() at browser_message_filter.cc:136 0x5888c6 
content::BrowserMessageFilter::OnMessageReceived() at browser_message_filter.cc:52 0x5883f6 
IPC::ChannelProxy::Context::TryFilters() at ipc_channel_proxy.cc:79 0x1a9555b 
IPC::ChannelProxy::Context::OnMessageReceived() at ipc_channel_proxy.cc:93 0x1a955df

In Gpu process, GLXSurface is created by binding Window XID.

  1. GpuChannelManager::OnCreateViewCommandBuffer() deals with GpuMsg_CreateViewCommandBuffer via GpuChannelManager::OnMessageReceived()
  2. GpuChannel creates and owns GpuCommandBufferStub including Window XID.
  3. Send GpuHostMsg_CommandBufferCreated to Browser process
Via complicated not-important process, Render Process initializes CommandBufferProxyImpl.
  1. WebGraphicsContext3DCommandBufferImpl::InitializeCommandBuffer() calls CommandBufferProxyImpl::Initialize()
  2. CommandBufferProxyImpl::Initialize() send GpuCommandBufferMsg_Initialize to Gpu process.
Time to create GLX surface in Gpu process.
GpuCommandBufferStub::OnInitialize() is called.
  1. CommandBufferService, GLES2Decoder and GpuScheduler are created.
  2. ImageTransportSurface::CreateSurface() creates GLX Surface via binding Window XID.
  3. GL Context is created

How sync between Browser process and Gpu process.

Browser process draws non-client windows and Gpu process draws client windows. How chromium sync? Even more, when you click mouse right button, a context menu floats above client window. of course, Browser process draws a context menu. How it possible?

The secret is behind GpuProcessHostUIShim::OnAcceleratedSurfaceBuffersSwapped().

Gpu process draws something directly on X Window's buffer. Window performs double-buffering. It means Gpu process draws on back buffer and swaps. And then Browser process blits front buffer to screen. Chromium must prevent Browser process from bliting when Gpu process is swapping. How do that?

When Gpu process swap buffers, Gpu process sends GpuHostMsg_AcceleratedSurfaceBuffersSwapped to Browser process. When Browser process receives this msg, Browser process sleep a moment to guarantee data race for front buffer of Window.

Gpu process sends GpuHostMsg_AcceleratedSurfaceBuffersSwapped to Browser process.
void ImageTransportHelper::SendAcceleratedSurfaceBuffersSwapped(
    GpuHostMsg_AcceleratedSurfaceBuffersSwapped_Params params) {
  ...
  manager_->Send(new GpuHostMsg_AcceleratedSurfaceBuffersSwapped(params));
}


UI thread of Browser process deals with this msg. Sleep and send ack msg to Gpu process.
void GpuProcessHostUIShim::OnAcceleratedSurfaceBuffersSwapped(
    const GpuHostMsg_AcceleratedSurfaceBuffersSwapped_Params& params) {
  ...
  static const base::TimeDelta swap_delay = GetSwapDelay();
  if (swap_delay.ToInternalValue())
    base::PlatformThread::Sleep(swap_delay);

  // View must send ACK message after next composite.
  view->AcceleratedSurfaceBuffersSwapped(params, host_id_);
}

content::RenderWidgetHostViewGtk::AcceleratedSurfaceBuffersSwapped() at render_widget_host_view_gtk.cc:1,073 0x6b8a1b 
content::GpuProcessHostUIShim::OnAcceleratedSurfaceBuffersSwapped() at gpu_process_host_ui_shim.cc:343 0x6172ae 
DispatchToMethod{content::GpuProcessHostUIShim, void () at tuple.h:546 0x6192ae 
GpuHostMsg_AcceleratedSurfaceBuffersSwapped::Dispatch{content::GpuProcessHostUIShim, content::GpuProcessHostUIShim, void () at gpu_messages.h:376 0x6184ca 
content::GpuProcessHostUIShim::OnControlMessageReceived() at gpu_process_host_ui_shim.cc:198 0x6166f6 
content::GpuProcessHostUIShim::OnMessageReceived() at gpu_process_host_ui_shim.cc:170 0x616232 
content::RouteToGpuProcessHostUIShimTask() at gpu_process_host_ui_shim.cc:106 0x6159e2

To be honest, I wonder that it really guarantees data race of front buffer. or I perhaps don't find a REAL secret yet. If I know more, I'll update.

When turning on aura, rendering mechanism is quite different.

When turning on aura, GPU process draws almost everything. It means that animation of url field or tab button is also accelerated via GPU.

When it comes to aura, Window term is different on gtk platform.
In Gtk, Window is GtkWindow.
In aura, Window is aura::Window based on XWindow
  1. Browser process creates RootWindowHost
  2. Browser process sends surface id (just number) of RootWindowHost to Gpu Process.
  3. Gpu Process creates GL context and off-screen surface (a.k.a content texture).  (Gpu Process will draw everything on the surface)
  4. Browser process paints some off-client windows (e.g. url field, tab button) on their own backing store.
  5. However, Browser process paints and drawsother off-client windows (e.g. context menu) by itself.
  6. Browser process sends bitmaps (a.k.a backing store of window) to Gpu Process.
    (Or using skia ganesh(opengl backend) directly)
  7. Render Process paints GraphicsLayer tree on GraphicsContext3D.
  8. Gpu Process draws each Render Process's all layers on content texture.
  9. Gpu Process draws some non-client window's textures and content textures on screen.

Glossary

client window means something like web content and dialog
non-client window means something like button, url field and tab button.
paint means skia paints something on bitmap.
draw means compositor composite all layers on screen.

In Android, What Browser process finally draws contents in Window.

In Android, GLSurfaceHandle (a.k.a compositing surface) is different to GLX's. It is because Android does not allow non Activity process change contents of View in Activity process.

In render_widget_host_view_gtk.cc
gfx::GLSurfaceHandle RenderWidgetHostViewGtk::GetCompositingSurface() {
  ..
  return gfx::GLSurfaceHandle(compositing_surface_, gfx::NATIVE_TRANSPORT);
}

In render_widget_host_view_android.cc 
gfx::GLSurfaceHandle RenderWidgetHostViewAndroid::GetCompositingSurface() {
  ...
    return gfx::GLSurfaceHandle(gfx::kNullPluginWindow, gfx::TEXTURE_TRANSPORT);
}
  1. Gpu process send GpuHostMsg_AcceleratedSurfaceBuffersSwapped to browser process in ImageTransportHelper::SendAcceleratedSurfaceBuffersSwapped().
  2. GpuProcessHostUIShim::OnAcceleratedSurfaceBuffersSwapped() in Browser process blit buffer.

How to draw contents on Window in Software path.

Software path does not need Gpu process. Render process draws contents on root backing store and send it to Browser process. Browser process just blits it on screen. 
  1. Render process send ViewHostMsg_UpdateRect (including bitmap) to Browser process.
  2. IO thread of Browser process dispatchs the event to UI thread in RenderWidgetHelper::DidReceiveBackingStoreMsg()
  3. RenderWidgetHostImpl::OnUpdateRect() is called in UI thread of Browser process.
  4. RenderWidgetHostImpl::OnUpdateRect() updates a backing store and async call DidUpdateBackingStore(). (XSyncHandler::PushPaintCounter() send the event in event queue using XSync)
  5. RenderWidgetHostImpl::DidUpdateBackingStore() is called. At this time, all widgets are updated.
Posted by at
Labels: , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)