[meta-freescale] QtMultimedia on i.MX6

Wed Jul 10 03:41:23 PDT 2013

On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> On Wed, Jul 10, 2013 at 7:52 PM, Thomas Senyk
> 
> <thomas.senyk at pelagicore.com>wrote:
> > On Wednesday, 10 July, 2013 19:13:57 Philip Craig wrote:
> > > On Wed, Jul 10, 2013 at 6:26 PM, Thomas Senyk
> > > 
> > > <thomas.senyk at pelagicore.com>wrote:
> > > > On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> > > > > On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes at gmail.com>
> > > > 
> > > > wrote:
> > > > > > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <
> > > > 
> > > > thomas.senyk at pelagicore.com>
> > > > 
> > > > > > wrote:
> > > > > > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > > > > > >> Repository is public now.
> > > > > > >> I only have the code from BSP 1.1.0 there yet, but I'll add
> > 
> > 4.0.0
> > 
> > > > this
> > > > 
> > > > > > >> afternoon.
> > > > > > >> 
> > > > > > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > > > > > >> meta-fsl-arm layer to point to this repo. It's cleaner than the
> > > > 
> > > > patch
> > > > 
> > > > > > >> we currently have in the recipe.
> > > > > > > 
> > > > > > > Sounds wonderful!
> > > > > > > If something is ready to test, let me know.
> > > > > > 
> > > > > > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > 
> > > > > > package:
> > https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > 
> > > > > > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch
> > 
> > to
> > 
> > > > the
> > > > 
> > > > > > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > > > > > 
> > > > > > In the bitbucket repo, branch fsl-0.10 has commits that reflect
> > > > > > "as
> > > > > > is"
> > > > > > packages from the BSP.
> > > > > > My idea now is to work on branch 0.10 to generate a set of patches
> > > > 
> > > > that we
> > > > 
> > > > > > can upstream.
> > > > > > I would appreciate any help.
> > > > > 
> > > > > I can't speak for upstream, but do note that gstreamer 0.10 is no
> > 
> > longer
> > 
> > > > > maintained, and gst-plugins-gl has recently been updated to 1.0, so
> > 
> > it
> > 
> > > > > might be worth checking if upstream will accept 0.10 patches before
> > > > 
> > > > working
> > > > 
> > > > > on them.
> > > >  
> > > >  ... again ... a rather long mail ;)
> > > > 
> > > > I kind a gave up on 'glupload' (and therefor on gst-plugins-gl)
> > 
> > yesterday
> > 
> > > > evening.
> > > > It sounded very good at first, but after a while I realized it can't
> > 
> > work
> > 
> > > > in
> > > > it's current implementation/API ... it least not without X (which is a
> > 
> > no
> > 
> > > > go
> > > > for my in any case)
> > > > 
> > > > What I wanted to achieve:
> > > > Being able to use textures 'filled' by gst-plugins-gl within Qt's
> > 
> > OpenGL
> > 
> > > > SceneGraph.
> > > > 
> > > > Why IMO it will not work:
> > > > glupload has to create it's own EGLContext, for that one needs
> > > > displays
> > > > and
> > > > window.
> > > > The problem is that the current implementation does that be using the
> > > > vivante-
> > > > linuxfb-API which results in a separated, independent "connection" to
> > 
> > the
> > 
> > > > display.
> > > > This means glupload's "external-opengl-context" is not usable as any
> > > > provided
> > > > EGLContext will be bound to another display connection (I tried and
> > > > failed, if
> > > > it's supposed to work but I'm doing something wrong, I'm happy to try
> > > > again
> > > > with help)
> > > > 
> > > > This means you'll never be able to use the generated textures outside
> > 
> > of
> > 
> > > > gstreamer. ... I think ;)
> > > > The only use-case left is to use glimagesink to render to framebuffer
> > > > directly,
> > > > maybe adding some gstreamer-gl-effects or something.
> > > > 
> > > > This is not what I indented to achieve (although I've already marked
> > 
> > it as
> > 
> > > > my
> > > > back-up plan and doing it in QtMultimedia is trivial, if someone wants
> > 
> > to
> > 
> > > > try
> > > > that solution let me know and I tell you what to change)
> > > > 
> > > > To achieve what I want I thing the only viable solution would be to
> > > > replace
> > > > the "external-opengl-context" with a 'makeGLContextCurrent'-callback
> > 
> > ...so
> > 
> > > > that the EGLContext can be created and managed by the UI-framework
> > 
> > rather
> > 
> > > > then
> > > > the multimedia-framework... but I'm not eager to fiddle around with
> > > > gst-API ;)
> > > > 
> > > > 
> > > > 
> > > > If I'm wrong with any part and it's actually possible to use textures
> > > > generated by gst-plugins-gst outside of gstreamer (without X/other
> > > > 'windowing
> > > > manager'), I happy to try again if someone has a hint/idea how the
> > 
> > stack
> > 
> > > > should look like.
> > > > 
> > > > 
> > > > 
> > > > In the mean while I have an alternative plan:
> > > > The important part the thing is replacing the cpu based texture upload
> > > > (copy)
> > > > with a vivante-based memory-mapping ('no-copy'/'zero-copy')
> > > > Today I'm going try to implement this (using the gst-plugins-gl code
> > > > as
> > > > reference) within QtMultimedia.
> > > 
> > > I'm very new to opengl, but your analysis sounds correct to me.
> > > 
> > > You might be interested in looking at the qt-gstreamer package. It has a
> > > qtglvideosink plugin which uses the context you give it, rather than
> > > creating its own display etc. I did try using it, but I was trying to
> > 
> > use a
> > 
> > > QGLWidget to get the context, and it seems to be incompatible with
> > > eglfs.
> > > It displays fine on my laptop with xcb. Maybe you know an alternative
> > > way
> > > of getting a context when using eglfs.
> > 
> > But this is not imx6 specific, is it?
> > If it's not: we can't get to a optimized stack without using the vivante-
> > opengl ui
> > If it is: I'll take a look (where to find?)
> 
> I'm not sure why it needs to be imx6 specific. Doesn't the opengl API make
> it so that it doesn't need to be specific? The Qt eglfs platform is using
> the vivante opengl libraries. As I understand it, the parts that need to be
> imx6 specific are confined to the eglfs platform plugin.
> 
> qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer

No. There is no vendor-independent way to map cpu/vpu based memory into gpu 
(/opengl) memory.
Without 'mapping' one needs to copy/upload via glTexImage2D.
This one single line(!) consumes >100% cpu of one iMX6 core  for 1080p.
If you 'map' the cpu load is very easily <10%

Greets
Thomas