[meta-freescale] Chromium acceleration

[Carlos Rafael Giani]

Hi Eric and Christian,

I wanted to avoid _any_ copy involving the CPU, including any texture 
upload. The Vivante GLES direct texture extension can color-convert, 
yes, but it can also read pixels directly from a DMA buffer, without any 
copy. This is what I use for the GLES-based sink in my GStreamer 
plugins, and allows me to play 1080p videos with less than 6% CPU usage.
v4lsink would be another idea, but I am a bit skeptical, because it 
wouldn't play well with any X11 or Wayland compositor, since it would 
essentially act like an overlay. This is even worse with the isink (if 
the regular, non-overlay layer is used), because then, the sink simply 
overwrites pixels drawn by X11. What the isink does is to make sure the 
pixels are blit in the area of its output window, so it does not 
overwrite anything outside. But again, with desktop compositing this 
would break down. (Also, the isink is quite slow.)

I would consider this one texture upload only if absolutely necessary.

My idea was to actually produce the texture in the decoder myself, and 
just send the texture GL name around to the renderer. This would require 
that everything is drawn with GLES of course - what would a software 
renderer do with a GL name?

As for the decoder itself: I implemented it in the Chromium media 
framework, in media/ . I simply took the vpx decoder code, copied it, 
and modified it to use the VPU. I had VP8, MPEG2, MPEG4, and h264 
decoding working. It wasnt much code, but unfortunately, the interfaces 
tend to change between versions, so the code would have to be updated 
for the newest Chromium version. It is not much code, I will try to 
clean it up a bit and post it. In addition, I will ask the colleagues 
for the other patches for accelerating the 2D and WebGL rendering.

Carlos


On 2014-03-20 03:50, Eric Nelson wrote:
> Thanks Christian,
>
> On 03/19/2014 06:29 PM, Christian Betz wrote:
>>     The Vivante libraries can map the physical buffers produced by
>>     the VPU directly, so they could do format conversion on their
>>     way to the graphics stack if the Chromium bindings have access
>>     to that (the "single copy" I referred to above).
>>
>>
>> in my experience "zero-copy" really means zero memcpy() (or similar
>> system call, or really anything that involves the cpu to move around
>> image data. /metadata /such as buffer pointers is generally allowed). in
>> other words having the GPU do format conversion isn't considered a copy.
>>
>> here's a great article on this general technique implemented in
>> webkit/gtk+:
>> http://blogs.igalia.com/vjaquez/2013/07/26/composited-video-support-in-webkitgtk/ 
>>
>>
>
> That's a nice article, and if Webkit (Blink) retains the same mapping,
> video buffers from the VPU would end up hitting a RenderObject and
> the only "copy" would be in rendering that to the Framebuffer.
>
> Since we're on the fat end of the pipe here, that copy can be expensive
> and might limit things to 1x1080P decode, but I'd be happy with that.
>
> In the mfw_v4lsink case at least, this can be bypassed by having
> the V4L2 buffer filled by the VPU and also displayed by the IPU
> (such that copy), but aiming there would keep the video out of the
> rest of the RenderLayer tree (I think).
>
> This is what I was so awkwardly trying to suggest.
>
>> moving beyond semantics... let me tell you what i know about this
>> particular issue:
>>
>> there is source code in chromium to accomplish a similar goal.
>> specifically there is a "generic" android driver, which works out of the
>> box with i.mx6 because freescale implements the standard android video
>> decode interface that allows VPU -> GPU texture uploading (don't quote
>> me on this, i'm just reading the source code:
>> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/android_video_decode_accelerator.cc) 
>>
>>
>> sadly there is no such standard interface on GNU/Linux. outside of
>> android (i.e. chromeos), there are drivers for particular chipsets, like
>> samsung exynos. here is the source for that:
>> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc 
>>
>>
>> the group should also be aware of the current situation regarding these
>> modules /only/ working when certain flags to enable "chromeos" are used
>> (explained here:
>> https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode). in other
>> words: this stuff doesn't work on your x86 with the chrome you download
>> from google.
>>
>
> Again, thanks for the info.
>
> Regards,
>
>
> Eric