Thursday, May 22, 2014

Build Chromium for Chromium OS and Deploy to real device.

Origin: https://github.com/ds-hwang/wiki/wiki/Build-Chromium-for-Chromium-OS-and-Deploy-to-real-device

I want to test my media change on Acer C720P. Life is more bitter than I expected.

  • My opinion: It's so burdensome. After this course, you might like Tizen more...
    • Major reasons:
    • You can not launch your custom chrome in command line. What you can do is to reboot everytime.
    • Emerge, chromium os and Gentoo build tool, is so slow to build & package.
  • Target listener: chromium dev to want to test their changes on real device.
  • Why this wiki is needed?: Build Chromium for Chromium OS covers only generic x86/amd64/arm, not real device. and there is no document about it.
  • I explain for peppy (Acer C720/C720P) but it can be applied for all devices.
  • I made it on Ubuntu 14.04. I guess it could work fine on 12.04

Build image and flash to the device

# make your image
$ cros_sdk 
Now you're in the chroot environment.

chroot$ export BOARD=peppy
./setup_board --board=${BOARD}
./set_shared_user_password.sh
./build_packages --board=${BOARD} --nowithdebug
./build_image --board=${BOARD} --noenable_rootfs_verification test

# optional: check if the image is good
./get_latest_image.sh --board=${BOARD}
./mount_gpt_image.sh --board=${BOARD} -f $(./get_latest_image.sh --board=${BOARD})
./mount_gpt_image.sh --board=${BOARD} -u

# optional: make qemu image to run it on your pc.
./image_to_vm.sh --board=${BOARD}
./cros_start_vm --image_path /home/dshwang/chromiumos/src/build/images/peppy/R37-5875.0.2014_05_21_1624-a1/chromiumos_qemu_image.bin

# make booting usb
cros flash usb:// $(./get_latest_image.sh --board=${BOARD})/chromiumos_test_image.bin
  • for more overlays, refer to ./src/overlays
    • personally I'm interested in amd64-genericamd64-generic_freonpeppy,peppy_freon

run image on kvm

  • See Running a Chromium OS image under KVM
  • In summary
    • Enabling native virtualization in BIOS
    • sudo apt-get install qemu-kvm libvirt-bin virt-manager virtinst python-libvirt
    • ./bin/cros_start_vm --image_path ../build/images/peppy/latest/chromiumos_qemu_image.bin
  • trouble shooting
    • I encountered following error: "Could not initialize SDL(No available video device) - exiting"
    • The solution is to add "Defaults env_keep+="DISPLAY XAUTHORITY" after executing 'sudo visudo'.
    • Both env variables should be passed to sudo. I don't know why.

after repo sync

  • You don't need to run setup_board again, although it's really small part.
~/trunk/src/scripts/update_chroot
export BOARD=peppy
./build_packages --board=peppy --nowithdebug
...

Build Chromium and deploy

I guess there is a better way but it is the best way among what I found.

Build

Background

  • chrome os build system does not build chromium actually. It just downloads pre-built chromium from gs server during ./build_package.
    • this file include your chrome.~/chromiumos/chroot/build/peppy/packages/chromeos-base/chromeos-chrome-37.0.2006.4_rc-r1.tbz2
    • If wanna know more, see Portage Build FAQ
  • If you want to build your own change, you need to touch cros_workon, because it requires to know how Gentoo distribution is packaged.
    • However, chromium provides convenient hack for you to not need to make your own branch.

Build

Standard way

  • Chromium ebuild defines CHROME_ORIGIN for chromium dev.
    • See ~/src/third_party/chromiumos-overlay/chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild
    • Enter chroot with --chrome_root option to inform the location of your chromium checkout.
$ cd $HOME/cros
$ cros_sdk --enter --chrome_root=$HOME/chromium
  • Now that you are inside the chroot. Create chromium os image frombuild_packages step. CHROME_ORIGIN=LOCAL_SOURCE inform chromium os build system to build chromium using your chromium checkout.
$ export CHROME_ORIGIN=LOCAL_SOURCE
$ export BOARD=peppy  # or amd64-generic_freon or peppy_freon
$ ./build_packages --board=${BOARD} --nowithdebug
$ ./build_image --board=${BOARD} --noenable_rootfs_verification test
// use test instead of dev to deploy via ssh
... // more
  • Install & reboot chromebook to test
  • You need additional 16GB
    • out_peppy ~13G: ~/chromiumos/chroot/var/cache/chromeos-chrome/chrome-src/src/out_peppy
    • debug symbols ~2G:~/chromiumos/chroot/build/peppy/usr/lib/debug/opt/google/chrome

package build

  • If you want to build a single package, instead of running buid_packages you can manually run emerge. For instance the chromium package can be built with:
$ export CHROME_ORIGIN=LOCAL_SOURCE
$ emerge-peppy chromeos-base/chromeos-chrome
  • However, above command rollback your local source to release branch :( Don't do that.
  • It takes 15 min when you change one line in chromium source code. :(

Build only chromium

  • If you don't add/remove any file, you can
# inside chroot
~/trunk/src/scripts $ cd ~/chrome_root/src/
~/chrome_root/src $ ninja -C c/Release chrome

Build Simple chromium

  • you have an image now. time to follow Build Simple Chromium if you build amd64-generic or amd64-generic_freon
    • you can build it outside chroot after gyp_chromium once, which means you can use icecc.
    • don't forget --nogoma
    • Currently (7/Apr/15), it works only for amd64-generic. Unfortunately, amd64-generic_freon doesn't work :(
> cros chrome-sdk --board=amd64-generic --nogoma

Booting and getting to a command prompt

  • Plug the USB stick into the machine and reboot.
  • At the dev-mode warning screen, press Ctrl-U to boot from the USB.
  • Switch to terminal, Ctrl-Alt-F2 [ Ctrl ] [ Alt ] [ => ]
    • the [ => ] key is the right-arrow key just above the number 3 on your keyboard.
  • Login as chronos, password test0000.
  • Get back to the browser. Just press: [ Ctrl ] [ Alt ] [ <= ]

SSH

> ssh chronos@10.10.15.136
or > ssh root@10.10.15.136
Password: test0000

LOG

  • log is stored in
# /var/log/chrome/chrome # what you want
# /home/chronos/user/log/chrome # some browser process log. why?
$ tail -f /home/chronos/user/log/chrome /var/log/chrome/chrome

Deploy (generic)

# in host. e.g. deploy to peppy_freon
> cd ~/chromiumos/chroot/var/cache/chromeos-chrome/chrome-src/src
> ~/chromiumos/chromite/bin/deploy_chrome --board=peppy --build-dir=out_peppy_freon/Release --to=10.10.15.136
  • It works only for amd64-generic :(

Deploy (hack)

  • remount and prepare in device
> ssh root@10.10.15.136

# remount your root for rw
localhost $ mount -o remount,rw /dev/root /

# backup
localhost $ cd /opt/google/chrome/
localhost $ mv chrome chrome_bak

  • copy via scp
> cd ~/chromiumos/chroot/var/cache/chromeos-chrome/chrome-src/src

# before scp, you might want to strip to reduce size 2G -> 0.2G
> strip out_peppy_freon/Release/chrome

> scp out_peppy_freon/Release/chrome root@192.168.43.175:/opt/google/chrome/chrome
  • restart chrome
# Restart the UI via:
localhost $ restart ui

# however killing chrome is faster. watchdog restarts chrome automatically
localhost $ killall -9 chrome
  • command line
# change following file (read the comments in the file for more details)
localhost $ vi /etc/chrome_dev.conf

# killall is not sufficient
localhost $ restart ui
  • Trouble shooting: after gclient sync your chromium, I recommend to bake image again. otherwise, you might waste your several hours like me :(

Faster build hack

Plz

Send dongseong.hwang(at)intel.com mail if you know a better way.

Reference

Friday, November 8, 2013

Clank (a.k.a chrome for android) vs Android Webview based on Chromium vs Aura

original doc : https://github.com/ds-hwang/wiki/wiki/Clank-(a.k.a-chrome-for-android)-vs-Android-Webview-based-on-Chromium-vs-Aura


I was very confused when I read Android chromium code, because there are two path; single process hack and similar to Aura.

In conclusion

  • Clank is very similar to Aura
    • Clank runs GPU thread, not GPU Process. Refer to InProcessGPU switch.
    • ContentViewRenderView in Clank has similar role to RootWindow in Aura. Those own Compositor!!
  • Android Webview has wholly different.
    • It is single process
    • Impl thread for compositor is the same to UI thread.
    • So there is no latency to send touch events to Impl thread.
    • It uses ContextProviderInProcess instead of ContextProviderCommandBuffer. It means that GPU calls are executed in UI thread (= Impl thread).
    • It uses SynchronousBlahBlah terms. e.g. SynchronousRendererCompositor, SynchronousCompositor, etc..
  • This doc has prerequisites

Clank (a.k.a chrome for android)

Which cpp instances does Clank need?

  • remember ContentViewRenderView is needed.
    // in browser_jni_registrar.cc
    base::android::RegistrationMethod kContentRegisteredMethods[] = {
        {"AndroidLocationApiAdapter",
         content::AndroidLocationApiAdapter::RegisterGeolocationService},
        {"BrowserAccessibilityManager",
         content::RegisterBrowserAccessibilityManager},
        {"BrowserStartupController", content::RegisterBrowserStartupController},
        {"ChildProcessLauncher", content::RegisterChildProcessLauncher},
        {"ContentSettings", content::ContentSettings::RegisterContentSettings},
        {"ContentViewRenderView",
         content::ContentViewRenderView::RegisterContentViewRenderView},
        {"ContentVideoView", content::ContentVideoView::RegisterContentVideoView},
        {"ContentViewCore", content::RegisterContentViewCore},
        {"DataFetcherImplAndroid", content::DataFetcherImplAndroid::Register},
        {"DateTimePickerAndroid", content::RegisterDateTimeChooserAndroid},
        {"DownloadControllerAndroidImpl",
         content::DownloadControllerAndroidImpl::RegisterDownloadController},
        {"InterstitialPageDelegateAndroid",
         content::InterstitialPageDelegateAndroid::
             RegisterInterstitialPageDelegateAndroid},
        {"MediaDrmCredentialManager",
         content::MediaDrmCredentialManager::RegisterMediaDrmCredentialManager},
        {"MediaResourceGetterImpl",
         content::MediaResourceGetterImpl::RegisterMediaResourceGetter},
        {"LoadUrlParams", content::RegisterLoadUrlParams},
        {"PowerSaveBlock", content::RegisterPowerSaveBlocker},
        {"RegisterImeAdapter", content::RegisterImeAdapter},
        {"SpeechRecognizerImplAndroid",
         content::SpeechRecognizerImplAndroid::RegisterSpeechRecognizer},
        {"TouchPoint", content::RegisterTouchPoint},
        {"TracingControllerAndroid", content::RegisterTracingControllerAndroid},
        {"VibrationMessageFilter", content::VibrationMessageFilter::Register},
        {"WebContentsObserverAndroid", content::RegisterWebContentsObserverAndroid},
        {"WebViewStatics", content::RegisterWebViewStatics}, };
    

What is ContentViewRenderView?

  • ShellManager.java and Shell.java use ContentViewRenderView.
  • Shell looks like Browser in chrome aura.
  • ContentViewRenderView seems to have similar role to RootWindow that DesktopRootWindowHostX11 owns.
// Callstack how to make DesktopRootWindowHostX11
views::DesktopRootWindowHostX11::Init() at desktop_root_window_host_x11.cc:343 0x555559e94c8f 
views::DesktopNativeWidgetAura::InitNativeWidget() at desktop_native_widget_aura.cc:240 0x555559e8f775 
DesktopBrowserFrameAura::InitNativeWidget() at desktop_browser_frame_aura.cc:54 0x55555904bb69 
views::Widget::Init() at widget.cc:374 0x555559ea73e1 
BrowserFrame::InitBrowserFrame() at browser_frame.cc:78 0x555559049423 
BrowserWindow::CreateBrowserWindow() at browser_view.cc:2,618 0x555558e9df6d 
() at browser.cc:235 0x555558db983c 
Browser::Browser() at browser.cc:402 0x555558dba8a8 

Anything interesting

  • There is not huge difference. Aura and Clank are very same.

There is some adjustment for resources.

  • GPU resources adjustment.
    static scoped_ptr<WebGraphicsContext3DCommandBufferImpl>
    CreateGpuProcessViewContext(
        const blink::WebGraphicsContext3D::Attributes attributes,
        int surface_id) {
      ...
      static const size_t kBytesPerPixel = 4;
      gfx::DeviceDisplayInfo display_info;
      size_t full_screen_texture_size_in_bytes =
          display_info.GetDisplayHeight() *
          display_info.GetDisplayWidth() *
          kBytesPerPixel;
      WebGraphicsContext3DCommandBufferImpl::SharedMemoryLimits limits;
      limits.command_buffer_size = 64 * 1024;
      limits.start_transfer_buffer_size = 64 * 1024;
      limits.min_transfer_buffer_size = 64 * 1024;
      limits.max_transfer_buffer_size = std::min(
          3 * full_screen_texture_size_in_bytes, kDefaultMaxTransferBufferSize);
      limits.mapped_memory_reclaim_limit = 2 * 1024 * 1024;
      bool use_echo_for_swap_ack = true;
      return make_scoped_ptr(
          new WebGraphicsContext3DCommandBufferImpl(surface_id,
                                                    url,
                                                    gpu_channel_host.get(),
                                                    use_echo_for_swap_ack,
                                                    attributes,
                                                    false,
                                                    limits));
    }
    

Clank don't use threaded-compositor in Browser side.

void CompositorImpl::SetVisible(bool visible) {
  if (!visible) {
    ui_resource_map_.clear();
    host_.reset();
    client_->UIResourcesAreInvalid();
  } else if (!host_) {
    cc::LayerTreeSettings settings;
    settings.refresh_rate = 60.0;
    settings.impl_side_painting = false;
    settings.allow_antialiasing = false;
    settings.calculate_top_controls_position = false;
    settings.top_controls_height = 0.f;
    settings.use_memory_management = false;
    settings.highp_threshold_min = 2048;

    host_ = cc::LayerTreeHost::CreateSingleThreaded(this, this, NULL, settings);
    ...
  }
}

Others

  • Clank runs GPU thread, not GPU Process. Refer to InProcessGPU switch.
  • Clank enables impl-side-painting in renderer side.
  • Browser side HW Surface is gfx::GLSurfaceHandle(gfx::kNullPluginWindow, gfx::NATIVE_DIRECT). same to Aura.

Android Webview

  • It's single process hacking.

Class diagram

// in android_webview_jni_registrar.cc
static base::android::RegistrationMethod kWebViewRegisteredMethods[] = {
  // Register JNI for android_webview classes.
  { "AndroidProtocolHandler", RegisterAndroidProtocolHandler },
  { "AwAutofillManagerDelegate", RegisterAwAutofillManagerDelegate },
  { "AwContents", RegisterAwContents },
  { "AwContentsClientBridge", RegisterAwContentsClientBridge },
  { "AwContentsIoThreadClientImpl", RegisterAwContentsIoThreadClientImpl },
  { "AwDevToolsServer", RegisterAwDevToolsServer },
  { "AwFormDatabase", RegisterAwFormDatabase },
  { "AwPicture", RegisterAwPicture },
  { "AwSettings", RegisterAwSettings },
  { "AwHttpAuthHandler", RegisterAwHttpAuthHandler },
  { "AwQuotaManagerBridge", RegisterAwQuotaManagerBridge },
  { "AwResource", AwResource::RegisterAwResource },
  { "AwWebContentsDelegate", RegisterAwWebContentsDelegate },
  { "CookieManager", RegisterCookieManager },
  { "InterceptedRequestDataImpl", RegisterInterceptedRequestData },
  { "InputStream", RegisterInputStream },
  { "JavaBrowserViewRendererHelper", RegisterJavaBrowserViewRendererHelper },
};

How about Compositor or something.

  • In conclusion, WebView has no browser side compositor.
  • WebView has only one WebContents instance.
  • InProcessViewRenderer can draw by HW and SW. SW is needed for thumbnail (block API backward compatibility).
  • Important class relationship
    • AwContents owns WebContents
    • AwContents -> InProcessViewRenderer -> SynchronousCompositor -> SynchronousCompositorOutputSurface
  • SynchronousCompositorOutputSurface is back door that WebView gets contents image.
    • SynchronousCompositorOutputSurface is created in render thread but mainly used in UI thread.
    • because UI thread is impl thread!!
    • RenderWidget -> RenderWidgetCompositor -> LayerTreeHost -> ThreadProxy
  • In Impl thread,
    • ThreadProxy -> LayerTreeHostImpl -> SynchronousCompositorOutputSurface

Some code

you can feel darn.
AwContents::AwContents(scoped_ptr<WebContents> web_contents)
    : web_contents_(web_contents.Pass()),
      browser_view_renderer_(
          new InProcessViewRenderer(this, java_renderer_helper(),
                                    web_contents_.get())) {
  ...
}

void InProcessViewRenderer::DidInitializeCompositor(
    content::SynchronousCompositor* compositor) {
  TRACE_EVENT0("android_webview",
               "InProcessViewRenderer::DidInitializeCompositor");
  DCHECK(compositor && compositor_ == NULL);
  compositor_ = compositor;
  hardware_initialized_ = false;
  hardware_failed_ = false;
}

scoped_ptr<cc::OutputSurface> RenderWidget::CreateOutputSurface(bool fallback) {
#if defined(OS_ANDROID)
  if (SynchronousCompositorFactory* factory =
      SynchronousCompositorFactory::GetInstance()) {
    return factory->CreateOutputSurface(routing_id());
  }
#endif
  ...
}

scoped_refptr<cc::ContextProvider>
RenderThreadImpl::OffscreenCompositorContextProvider() {
  DCHECK(IsMainThread());

#if defined(OS_ANDROID)
  if (SynchronousCompositorFactory* factory =
      SynchronousCompositorFactory::GetInstance()) {
    if (compositor_message_loop_proxy_)
      return factory->GetOffscreenContextProviderForCompositorThread();
    return factory->GetOffscreenContextProviderForMainThread();
  }
#endif
  ...
}

Renference

Thursday, November 7, 2013

GLSurface for both render and browser process

original doc : https://github.com/ds-hwang/wiki/wiki/GLSurface-for-both-render-and-browser-process#who-why-how-create-hw-surface

Who, why, how create HW Surface

Where to create HW Surface and How to use it.

Where need it

Render process

  • Where create surface_id()?
    scoped_ptr<WebGraphicsContext3DCommandBufferImpl>
    RenderWidget::CreateGraphicsContext3D(
        const WebKit::WebGraphicsContext3D::Attributes& attributes) {
      ...
      scoped_ptr<WebGraphicsContext3DCommandBufferImpl> context(
          new WebGraphicsContext3DCommandBufferImpl(
              surface_id(), // <- HERE
              GetURLForGraphicsContext3D(),
              gpu_channel_host.get(),
              swap_client,
              attributes,
              false /* bind generates resources */,
              limits));
      return context.Pass();
    }
    

Browser process

  • Where create data->surface_id?
    scoped_ptr<cc::OutputSurface> GpuProcessTransportFactory::CreateOutputSurface(
        ui::Compositor* compositor) {
      PerCompositorData* data = per_compositor_data_[compositor];
      ...
      if (!command_line->HasSwitch(switches::kUIEnableSoftwareCompositing)) {
        context_provider = ContextProviderCommandBuffer::Create(
            GpuProcessTransportFactory::CreateContextCommon(
                swap_client_weak_ptr,
                data->surface_id), // <- HERE
                "Compositor");
      }
      ...
    }
    

Where create surface_id?

render process

// surface_id and routing_id are just seq.
RenderWidgetHostImpl::RenderWidgetHostImpl(RenderWidgetHostDelegate* delegate,
                                           RenderProcessHost* process,
                                           int routing_id,
                                           bool hidden)
    : view_(NULL),
      ...
      last_input_number_(0) {
  if (routing_id_ == MSG_ROUTING_NONE) {
    routing_id_ = process_->GetNextRoutingID();
    surface_id_ = GpuSurfaceTracker::Get()->AddSurfaceForRenderer(
        process_->GetID(),
        routing_id_);
  } else {
  ...
}

// Create HW surface and map to surface_id
void RenderWidgetHostImpl::Init() {
  ...
  GpuSurfaceTracker::Get()->SetSurfaceHandle(
      surface_id_, GetCompositingSurface());
  ...
}

gfx::GLSurfaceHandle RenderWidgetHostImpl::GetCompositingSurface() {
  if (view_)
    return view_->GetCompositingSurface();
  return gfx::GLSurfaceHandle();
}

with Aura
gfx::GLSurfaceHandle RenderWidgetHostViewAura::GetCompositingSurface() {
  if (shared_surface_handle_.is_null()) {
    ImageTransportFactory* factory = ImageTransportFactory::GetInstance();
    shared_surface_handle_ = factory->CreateSharedSurfaceHandle();
    if (!shared_surface_handle_.is_null())
      factory->AddObserver(this);
  }
  return shared_surface_handle_;
}

// Make GLSurfaceHandle with gfx::kNullPluginWindow & gfx::TEXTURE_TRANSPORT
gfx::GLSurfaceHandle GpuProcessTransportFactory::CreateSharedSurfaceHandle() {
  scoped_refptr<cc::ContextProvider> provider =
      SharedMainThreadContextProvider();
  if (!provider.get())
    return gfx::GLSurfaceHandle();
  typedef WebGraphicsContext3DCommandBufferImpl WGC3DCBI;
  WGC3DCBI* context = static_cast<WGC3DCBI*>(provider->Context3d());
  gfx::GLSurfaceHandle handle = gfx::GLSurfaceHandle(
      gfx::kNullPluginWindow, gfx::TEXTURE_TRANSPORT);
  handle.parent_gpu_process_id = context->GetGPUProcessID();
  handle.parent_client_id = context->GetChannelID();
  return handle;
}

With GTK_TOOLKIT
gfx::GLSurfaceHandle RenderWidgetHostViewGtk::GetCompositingSurface() {
  if (compositing_surface_ == gfx::kNullPluginWindow) {
    GtkNativeViewManager* manager = GtkNativeViewManager::GetInstance();
    gfx::NativeViewId view_id = GetNativeViewId();

    if (!manager->GetPermanentXIDForId(&compositing_surface_, view_id)) {
      DLOG(ERROR) << "Can't find XID for view id " << view_id;
    }
  }
  return gfx::GLSurfaceHandle(compositing_surface_, gfx::NATIVE_TRANSPORT);
}

// Time to create RenderView in render process
bool RenderViewHostImpl::CreateRenderView(
    const string16& frame_name,
    int opener_route_id,
    int32 max_page_id) {
  ...

  ViewMsg_New_Params params;
  ...
  params.view_id = GetRoutingID();
  params.surface_id = surface_id();
  ...
  Send(new ViewMsg_New(params));
  ...
  return true;
}

In render process. Now RenderViewImpl : RenderWidget knows surface_id.
void RenderThreadImpl::OnCreateNewView(const ViewMsg_New_Params& params) {
  EnsureWebKitInitialized();
  // When bringing in render_view, also bring in webkit's glue and jsbindings.
  RenderViewImpl::Create(
      params.opener_route_id,
      params.renderer_preferences,
      params.web_preferences,
      params.view_id,
      params.main_frame_routing_id,
      params.surface_id,
      ...
      params.allow_partial_swap);
}
  • NOTE
    • AURA : HW surface of RenderWidget is just texture
    • GTK : HW surface of RenderWidget is XWindow
  • QUESTION: How Render process create GLSurface by just sequence surface_id
    • A: It is because Render process always create CommandBufferProxyImpl via render process -> browser process -> gpu process.
    • A: In more detail, see below How WebGraphicsContext3DCommandBufferImpl connects GPU Process.

Browser process

  • Only Aura makes HW surface for browser process
    // if aura content_shell, (don't explain how chromium work because it's more complex)
    void DesktopNativeWidgetAura::InitNativeWidget(
        const Widget::InitParams& params) {
      ...
      aura::RootWindow::CreateParams rw_params(params.bounds);
      desktop_root_window_host_->Init(content_window_, params, &rw_params);
    
      root_window_.reset(new aura::RootWindow(rw_params));
      ...
    }
    
    RootWindow::RootWindow(const CreateParams& params)
        : Window(NULL),
          host_(CreateHost(this, params)),
          ...
          held_event_factory_(this) {
      compositor_.reset(new ui::Compositor(host_->GetAcceleratedWidget()));
      ...
    }
    
    // if aura chrome
    gfx::AcceleratedWidget RootWindowHostX11::GetAcceleratedWidget() {
      return xwindow_;
    }
    //or if aura content_shell
    gfx::AcceleratedWidget DesktopRootWindowHostX11::GetAcceleratedWidget() {
      return xwindow_;
    }
    
    Compositor::Compositor(gfx::AcceleratedWidget widget)
        : root_layer_(NULL),
          widget_(widget),
          ...
          schedule_draw_factory_(this) {
      ...
    }
    
    // Time to create OutputSurface
    scoped_ptr<cc::OutputSurface> GpuProcessTransportFactory::CreateOutputSurface(
        ui::Compositor* compositor) {
      PerCompositorData* data = per_compositor_data_[compositor];
      if (!data)
        data = CreatePerCompositorData(compositor);
      ...
      CommandLine* command_line = CommandLine::ForCurrentProcess();
      if (!command_line->HasSwitch(switches::kUIEnableSoftwareCompositing)) {
        context_provider = ContextProviderCommandBuffer::Create(
            GpuProcessTransportFactory::CreateContextCommon(
                swap_client_weak_ptr,
                data->surface_id),
                "Compositor");
      }
      ..
    }
    
    GpuProcessTransportFactory::PerCompositorData*
    GpuProcessTransportFactory::CreatePerCompositorData(
        ui::Compositor* compositor) {
      ...
      gfx::AcceleratedWidget widget = compositor->widget();
      GpuSurfaceTracker* tracker = GpuSurfaceTracker::Get();
    
      PerCompositorData* data = new PerCompositorData;
      data->surface_id = tracker->AddSurfaceForNativeWidget(widget);
      ...
      tracker->SetSurfaceHandle(
          data->surface_id,
          gfx::GLSurfaceHandle(widget, gfx::NATIVE_DIRECT));
    
      per_compositor_data_[compositor] = data;
    
      return data;
    }
    
  • NOTE: HW Surface of aura browser process is XWindow

Summary

  • GTK Render Process : XWindow with gfx::NATIVE_TRANSPORT
  • Aura Render Process : nothing with gfx::TEXTURE_TRANSPORT
  • Aura Browser Process: XWindow with gfx::NATIVE_DIRECT

What's happen near GPU side

How WebGraphicsContext3DCommandBufferImpl connects GPU Process

  • Where/How to create GLSurface from HW Surface.
  • As a results
    • Browser process send the HW Surface handle to GPU process
    • GPU process creates GLSurface from HW Surface.

Common part

bool WebGraphicsContext3DCommandBufferImpl::InitializeCommandBuffer(
    bool onscreen) {
  ...
  // Create a proxy to a command buffer in the GPU process.
  if (onscreen) {
    command_buffer_.reset(host_->CreateViewCommandBuffer(
        surface_id_,
        share_group,
        attribs,
        active_url_,
        gpu_preference_));
  } else {
    command_buffer_.reset(host_->CreateOffscreenCommandBuffer(
        gfx::Size(1, 1),
        share_group,
        attribs,
        active_url_,
        gpu_preference_));
  }
  ...
  // Initialize the command buffer.
  return command_buffer_->Initialize();
}

CommandBufferProxyImpl* GpuChannelHost::CreateViewCommandBuffer(
    int32 surface_id,
    CommandBufferProxyImpl* share_group,
    const std::vector<int32>& attribs,
    const GURL& active_url,
    gfx::GpuPreference gpu_preference) {
  ...
  // connect gpu process and get route_id. This impl is different between browser and render process
  int32 route_id = factory_->CreateViewCommandBuffer(surface_id, init_params);
  if (route_id == MSG_ROUTING_NONE)
    return NULL;

  // make CommandBufferProxyImpl using route_id
  CommandBufferProxyImpl* command_buffer =
      new CommandBufferProxyImpl(this, route_id);
  ...
}

// Offscreen logic is the same between browser and render process.
CommandBufferProxyImpl* GpuChannelHost::CreateOffscreenCommandBuffer(
    const gfx::Size& size,
    CommandBufferProxyImpl* share_group,
    const std::vector<int32>& attribs,
    const GURL& active_url,
    gfx::GpuPreference gpu_preference) {
  ...
  int32 route_id;
  if (!Send(new GpuChannelMsg_CreateOffscreenCommandBuffer(size,
                                                           init_params,
                                                           &route_id))) {
    return NULL;
  }
  ...
  CommandBufferProxyImpl* command_buffer =
      new CommandBufferProxyImpl(this, route_id);
  ...
}

in GPU process for offscreen
void GpuChannel::OnCreateOffscreenCommandBuffer(
    const gfx::Size& size,
    const GPUCreateCommandBufferConfig& init_params,
    int32* route_id) {
  ...
  *route_id = GenerateRouteID();

  scoped_ptr<GpuCommandBufferStub> stub(new GpuCommandBufferStub(
      this,
      share_group,
      gfx::GLSurfaceHandle(),
      ...
      *route_id,
      ...
      init_params.active_url));
  ...
  router_.AddRoute(*route_id, stub.get());
  stubs_.AddWithID(stub.release(), *route_id);
  ...
}

Render process

Render process need to pass through browser process because render process does not know HW Surface handle.
// in render process
int32 RenderThreadImpl::CreateViewCommandBuffer(
      int32 surface_id, const GPUCreateCommandBufferConfig& init_params) {
  ...
  int32 route_id = MSG_ROUTING_NONE;
  IPC::Message* message = new GpuHostMsg_CreateViewCommandBuffer(
      surface_id,
      init_params,
      &route_id);

  // Allow calling this from the compositor thread.
  thread_safe_sender()->Send(message);

  return route_id;
}

// in io thread in browser process
void GpuMessageFilter::OnCreateViewCommandBuffer(
    int32 surface_id,
    const GPUCreateCommandBufferConfig& init_params,
    IPC::Message* reply) {
  ...
  gfx::GLSurfaceHandle compositing_surface;

  int renderer_id = 0;
  int render_widget_id = 0;
  bool result = surface_tracker->GetRenderWidgetIDForSurface(
      surface_id, &renderer_id, &render_widget_id);
  ...
  // Get HW Surface via surface_id
    compositing_surface = surface_tracker->GetSurfaceHandle(surface_id);

  ... // error handling for gpu process die

  host->CreateViewCommandBuffer(
      compositing_surface,
      surface_id,
      render_process_id_,
      init_params,
      base::Bind(&GpuMessageFilter::CreateCommandBufferCallback,
                 weak_ptr_factory_.GetWeakPtr(),
                 reply));
}

void GpuProcessHost::CreateViewCommandBuffer(
    const gfx::GLSurfaceHandle& compositing_surface,
    int surface_id,
    int client_id,
    const GPUCreateCommandBufferConfig& init_params,
    const CreateCommandBufferCallback& callback) {
  ..
  // Request GPU process to make route_id for GPUChannel. send surface_id and HW Surface handle!!
  // NOTE!! How to receive? this is on io thread. io thread MUST not be blocked. GPU process will send ACK msg.
  // At that time, callback will run to notify render process.
      Send(new GpuMsg_CreateViewCommandBuffer(
          compositing_surface, surface_id, client_id, init_params))) {
    create_command_buffer_requests_.push(callback);
    surface_refs_.insert(std::make_pair(surface_id,
        GpuSurfaceTracker::GetInstance()->GetSurfaceRefForSurface(surface_id)));
  ..
}

// in GPU process
void GpuChannelManager::OnCreateViewCommandBuffer(
    const gfx::GLSurfaceHandle& window,
    int32 surface_id,
    int32 client_id,
    const GPUCreateCommandBufferConfig& init_params) {
  DCHECK(surface_id);
  int32 route_id = MSG_ROUTING_NONE;

  // NOTE!!: get gpu_channel using render_process_id. render process uses only one gpu_channel_host.
  GpuChannelMap::const_iterator iter = gpu_channels_.find(client_id);
  if (iter != gpu_channels_.end()) {
    iter->second->CreateViewCommandBuffer(
        window, surface_id, init_params, &route_id);
  }

  // NOTE!!: and Send route_id to browser process. remember browser process is waiting for it.
  Send(new GpuHostMsg_CommandBufferCreated(route_id));
}

void GpuChannel::CreateViewCommandBuffer(
    const gfx::GLSurfaceHandle& window,
    int32 surface_id,
    const GPUCreateCommandBufferConfig& init_params,
    int32* route_id) {
  ...
  // Similar to GpuChannel::OnCreateOffscreenCommandBuffer()
  *route_id = GenerateRouteID();
  scoped_ptr<GpuCommandBufferStub> stub(
      new GpuCommandBufferStub(this,
                               share_group,
                               window,
                               ...
                               *route_id,
                               surface_id,
                               ...
                               init_params.active_url));
  ...
  router_.AddRoute(*route_id, stub.get());
  stubs_.AddWithID(stub.release(), *route_id);
}

// in io thread in browser process
void GpuProcessHost::OnCommandBufferCreated(const int32 route_id) {
  ...
  CreateCommandBufferCallback callback =
      create_command_buffer_requests_.front();
  create_command_buffer_requests_.pop();
  callback.Run(route_id);
}

void GpuMessageFilter::CreateCommandBufferCallback(
    IPC::Message* reply, int32 route_id) {
  DCHECK(BrowserThread::CurrentlyOn(BrowserThread::IO));
  GpuHostMsg_CreateViewCommandBuffer::WriteReplyParams(reply, route_id);
  Send(reply);
}

// in render process
int32 RenderThreadImpl::CreateViewCommandBuffer(
      int32 surface_id, const GPUCreateCommandBufferConfig& init_params) {
  // NOTE!!: sync GpuHostMsg_CreateViewCommandBuffer msg return!!
  int32 route_id = MSG_ROUTING_NONE;
  IPC::Message* message = new GpuHostMsg_CreateViewCommandBuffer(
      surface_id,
      init_params,
      &route_id);

  // Allow calling this from the compositor thread.
  thread_safe_sender()->Send(message);

  return route_id;
}

Browser process

// in main thread of browser process
int32 BrowserGpuChannelHostFactory::CreateViewCommandBuffer(
      int32 surface_id,
      const GPUCreateCommandBufferConfig& init_params) {
  CreateRequest request;
  // Request IO thread and wait.
  GetIOLoopProxy()->PostTask(FROM_HERE, base::Bind(
        &BrowserGpuChannelHostFactory::CreateViewCommandBufferOnIO,
        base::Unretained(this),
        &request,
        surface_id,
        init_params));
  ...
  request.event.Wait();
  return request.route_id;
}

// in io thread
void BrowserGpuChannelHostFactory::CreateViewCommandBufferOnIO(
    CreateRequest* request,
    int32 surface_id,
    const GPUCreateCommandBufferConfig& init_params) {
  GpuProcessHost* host = GpuProcessHost::FromID(gpu_host_id_);
  ...
  gfx::GLSurfaceHandle surface =
      GpuSurfaceTracker::Get()->GetSurfaceHandle(surface_id);
  // gpu_client_id_ is browser process id (seq.)
  host->CreateViewCommandBuffer(
      surface,
      surface_id,
      gpu_client_id_,
      init_params,
      base::Bind(&BrowserGpuChannelHostFactory::CommandBufferCreatedOnIO,
                 request));  // wow. partial binding!!
}

/*
 * Same to render process 
 * from GpuProcessHost::CreateViewCommandBuffer() to gpu process to receive ACK msg
 */

// in io thread in browser process
void GpuProcessHost::OnCommandBufferCreated(const int32 route_id) {
  ...
  CreateCommandBufferCallback callback =
      create_command_buffer_requests_.front();
  create_command_buffer_requests_.pop();
  callback.Run(route_id);
}

void BrowserGpuChannelHostFactory::CommandBufferCreatedOnIO(
    CreateRequest* request, int32 route_id) {
  request->route_id = route_id;
  request->event.Signal();
}

GpuCommandBufferStub

  • represent GraphicsContext3D in gpu process
  • See How Gpu process draws contents on the surface of Browser process's window in chromium.
  • See GPU Command Buffer
    gpu::gles2::GLES2DecoderImpl::DoBindTexture()
    gpu::gles2::GLES2DecoderImpl::HandleBindTexture()
    gpu::gles2::GLES2DecoderImpl::DoCommand()
    gpu::CommandParser::ProcessCommand()
    gpu::GpuScheduler::PutChanged()
    gpu::CommandBufferService::Flush()
    content::GpuCommandBufferStub::OnAsyncFlush()
    _ZN30GpuCommandBufferMsg_AsyncFlush8DispatchIN7content20GpuCommandBufferStubES2_MS2_FvijEEEbPKN3IPC7MessageEPT_PT0_T1_.isra.51
    content::GpuCommandBufferStub::OnMessageReceived()
    content::MessageRouter::RouteMessage()
    content::GpuChannel::HandleMessage()
    ...
    

How to create GLSurface

void GpuCommandBufferStub::OnInitialize(
    base::SharedMemoryHandle shared_state_handle,
    IPC::Message* reply_message) {
  ...
  if (!handle_.is_null()) {
    ...
    surface_ = ImageTransportSurface::CreateSurface(  // CreateViewCommandBuffer!!!
        channel_->gpu_channel_manager(),
        this,
        handle_);
  } else {
    GpuChannelManager* manager = channel_->gpu_channel_manager();
    surface_ = manager->GetDefaultOffscreenSurface();  // CreateOffscreenCommandBuffer!!!
  }
  ...
}

scoped_refptr<gfx::GLSurface> ImageTransportSurface::CreateSurface(
    GpuChannelManager* manager,
    GpuCommandBufferStub* stub,
    const gfx::GLSurfaceHandle& handle) {
  scoped_refptr<gfx::GLSurface> surface;
  if (handle.transport_type == gfx::TEXTURE_TRANSPORT)
    surface = new TextureImageTransportSurface(manager, stub, handle); // AURA RENDER PROCESS!!!!! do you remember?
  else
    surface = CreateNativeSurface(manager, stub, handle);  // Aura browser process, GTK Render process!!!

  if (!surface.get() || !surface->Initialize())
    return NULL;
  return surface;
}

scoped_refptr<gfx::GLSurface> ImageTransportSurface::CreateNativeSurface(
    GpuChannelManager* manager,
    GpuCommandBufferStub* stub,
    const gfx::GLSurfaceHandle& handle) {
  DCHECK(handle.handle);
  DCHECK(handle.transport_type == gfx::NATIVE_DIRECT ||   // Aura browser process!!!
         handle.transport_type == gfx::NATIVE_TRANSPORT);  // GTK render process!!!
  scoped_refptr<gfx::GLSurface> surface =
      gfx::GLSurface::CreateViewGLSurface(handle.handle);
  if (!surface.get())
    return surface;
  return scoped_refptr<gfx::GLSurface>(new PassThroughImageTransportSurface(
      manager, stub, surface.get(), handle.is_transport()));
}
FIN.