[meta-freescale] QtMultimedia on i.MX6

[Thomas Senyk]

Hi,

I've started to investigate to see the QtMultimedia-gstreamer integration on 
the i.MX6 could be optimized.

I'm writing this mail to sync with others who might already done some work or 
investigations on this matter, to get hints where to start and what's worth to 
investigate.



State right now:

The HW-decoders are used (according to gstreamer log).
That's a very good start/out of the box experience! Good job..freescale I 
guess?

There is still a lot of cpu load:
   - 720p: smooth playback but~18% complete cpu load => ~70% of one core
   - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one core 
+ some additional cpu load

The obvious reason for this cpu load is some sort of memcopy and/or 
conversion.
  ... gstreamer doesn't render into gpu-memory directly.
  ... or (if you want it the other way around) the gstreamer (/cpu) memory 
isn't mapped into gpu-memory (which is possibly on the imx6)




Started to do some digging:

One of the first things I tried was to compile gst-plugin-gl (which is part of 
the meta-fsl-arm layer via a .bbappend).
The current recipe is driver-version-1.1.0 based and is not building out of 
the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz (based 
on 4.0.0) building.**

So, after I got it deployed I hoped for magic to happen and everything is 
perfect now ... no big surprise: it's not :) nothing changed, possibly the 
gst-pipline build by qtmultimedia is the ame.



The reason for this mail / some questions:

So I though, I write this mail before I waste a lot of time.
What does people think is the best way forward?
Try to convince QtMultimedia to use glsink?
Try to use a fakesink/memory-based-sink and try to map the cpu memory into gpu 
memory via vivante APIs?

Does anyone have c/c++ code I could try?
Ideally for direct framebuffer/EGL use (I'm not using X11)
I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
 .. but it's crashing rather early and I'm not sure the if the time is well 
spend fix this code? Anyone used it before?
Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?


Greet
Thomas

** some more info in gst-plugin-gl:
It's 'detected' on the board:
root at nitrogen-1:~# gst-inspect |grep gl
opengl:  glupload: OpenGL upload
opengl:  gldownload: OpenGL video maker
opengl:  glimagesink: OpenGL video sink
opengl:  glfiltercube: OpenGL cube filter
opengl:  gleffects: Gstreamer OpenGL Effects

... if someone is interested on how I got it building, let me know. If I got 
something useful out of it (read: "it's tested") I'll start a discussion on 
how to upstream it properly into meta-fsl-arm (by the way: I find the current 
setup of how to bbappend and it's patch looks like very weird)