[meta-freescale] QtMultimedia on i.MX6

[Sébastien Taylor]

How would one gain access (read-only) to this repository?

On 2013-07-04, at 3:42 PM, Otavio Salvador <otavio at ossystems.com.br> wrote:

> On Thu, Jul 4, 2013 at 12:14 PM, Thomas Senyk
> <thomas.senyk at pelagicore.com> wrote:
>> On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote:
>>> Hi,
>>> 
>>> I've started to investigate to see the QtMultimedia-gstreamer integration on
>>> the i.MX6 could be optimized.
>>> 
>>> I'm writing this mail to sync with others who might already done some work
>>> or investigations on this matter, to get hints where to start and what's
>>> worth to investigate.
>>> 
>>> 
>>> 
>>> State right now:
>>> 
>>> The HW-decoders are used (according to gstreamer log).
>>> That's a very good start/out of the box experience! Good job..freescale I
>>> guess?
>>> 
>>> There is still a lot of cpu load:
>>>   - 720p: smooth playback but~18% complete cpu load => ~70% of one core
>>>   - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one
>>> core + some additional cpu load
>>> 
>>> The obvious reason for this cpu load is some sort of memcopy and/or
>>> conversion.
>>>  ... gstreamer doesn't render into gpu-memory directly.
>>>  ... or (if you want it the other way around) the gstreamer (/cpu) memory
>>> isn't mapped into gpu-memory (which is possibly on the imx6)
>>> 
>>> 
>>> 
>>> 
>>> Started to do some digging:
>>> 
>>> One of the first things I tried was to compile gst-plugin-gl (which is part
>>> of the meta-fsl-arm layer via a .bbappend).
>>> The current recipe is driver-version-1.1.0 based and is not building out of
>>> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz
>>> (based on 4.0.0) building.**
>>> 
>>> So, after I got it deployed I hoped for magic to happen and everything is
>>> perfect now ... no big surprise: it's not :) nothing changed, possibly the
>>> gst-pipline build by qtmultimedia is the ame.
>>> 
>>> 
>>> 
>>> The reason for this mail / some questions:
>>> 
>>> So I though, I write this mail before I waste a lot of time.
>>> What does people think is the best way forward?
>>> Try to convince QtMultimedia to use glsink?
>>> Try to use a fakesink/memory-based-sink and try to map the cpu memory into
>>> gpu memory via vivante APIs?
>>> 
>>> Does anyone have c/c++ code I could try?
>>> Ideally for direct framebuffer/EGL use (I'm not using X11)
>>> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
>>> .. but it's crashing rather early and I'm not sure the if the time is well
>>> spend fix this code? Anyone used it before?
>>> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?
>>> 
>>> 
>>> Greet
>>> Thomas
>> 
>> Ok ... what is it with me and sending mails 1minute to early all the time ...
>> After I pressed "send", I had a very simple and obvious idea .. why not just
>> simply try glimagesink instead of mfw_v4lsink:
>> 
>> gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux !
>> vpudec !  glimagesink
>> 
>> ... long shot, but:
>> It's actually working!
>> It just opens /dev/fb0 and renders the video into it.
>> 
>> 1080p with 0%(!!) cpu load!
>> 
>> I think that pretty much answers all questions I had ;)
>> The way forward: patch QtMultimedia to use glimagesink
>> ... I hope it's going to be trivial to just give him a active FBO or texture.
>> 
>> 
>> Sorry to everybody who read my way to long mail ;)
> 
> It was very good read. By the way, Rogerio (add in Cc) started to try
> to get the gst-plugin-gl code more manageble in bitbucket
> (https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/all)
> but it was not finished. Maybe you could help on this and we could try
> to merge the Vivante backend upstream? This would allow us to drop the
> bbappend completely.
> 
>>> ** some more info in gst-plugin-gl:
>>> It's 'detected' on the board:
>>> root at nitrogen-1:~# gst-inspect |grep gl
>>> opengl:  glupload: OpenGL upload
>>> opengl:  gldownload: OpenGL video maker
>>> opengl:  glimagesink: OpenGL video sink
>>> opengl:  glfiltercube: OpenGL cube filter
>>> opengl:  gleffects: Gstreamer OpenGL Effects
>>> 
>>> ... if someone is interested on how I got it building, let me know. If I got
>>> something useful out of it (read: "it's tested") I'll start a discussion on
>>> how to upstream it properly into meta-fsl-arm (by the way: I find the
>>> current setup of how to bbappend and it's patch looks like very weird)
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale at yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale
> 
> 
> 
> --
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://projetos.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
> _______________________________________________
> meta-freescale mailing list
> meta-freescale at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale