[meta-freescale] Gstreamer pipeline problem

[Thomas Senyk]

On Friday, 12 July, 2013 17:35:01 Chris Tapp wrote:
> Hi Thomas,
> 
> Some more info below:
> 
> On 11 Jul 2013, at 09:39, Thomas Senyk wrote:
> > On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
> >> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
> >>> I've got an application which uses playbin2 to capture video. The
> >>> pipeline
> >>> is of the form:
> >>> 
> >>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
> >>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
> >>> fakesink"
> >>> 
> >>> I then get the "frame" property from the pipeline and use this to grab
> >>> the
> >>> latest frame.
> >>> 
> >>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
> >>> Yocto system, but the pipeline fails on the Wandboard Quad. I think this
> >>> is related to:
> >>> 
> >>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
> >>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linu
> >>> x
> >>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetra
> >>> ns
> >>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
> >>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
> >>> framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
> >>> in anything we support
> >>> 
> >>> I added an ffmpegcolorspace element betwween the queue2 and the
> >>> videoscale
> >>> to get round this and the pipeline now builds, but only a few frames are
> >>> captured. There are different diagnostics showing:
> >>> 
> >>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
> >>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
> >>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
> >>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
> >>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no
> >>> frame
> >>> buffer message, return 0x89, 8 frames in displaying queue!!
> >>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x89,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> <lots of repeats>
> >>> 
> >>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
> >>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x88,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> <lots of repeats>
> >>> 
> >>> Message Callback : Element playbin0x250aa0 changed state from PAUSED to
> >>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x88,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
> >>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
> >>> ret 2 No such file or directory queued 0
> >>> 
> >>> 
> >>> The "Message Callback" events are my own logging to try and see what's
> >>> happening in my app.
> >>> 
> >>> Is this something I'm doing wrong, or are these messages a real issue
> >>> somewhere?
> >> 
> >> This is when playing a .webm. The results for an .flv are as expected.
> > 
> > is it the same when you use v4l2 sink instead of fakesink?
> > Is playbin (without defining the pipeline) behaving the same?
> 
> I've run some tests using pipelines of the form:
> 
>    gst-launch playbin2=file://trailer.webm video-sink="<a-sink-to-test>"
> 
> GST_DEBUG was set to "*:2"
> 
> 1) When the sink is set to "mfw_4vlsink" I get full screen video playback;
> 2) When the sink is set to "fakesink" I get nothing on the screen (as
> expected), and no debug of interest; 3) When the sink is set to "queue2 !
> mfw_4vlsink" I get very choppy full screen video playback (which eventually
> stalls) and debug output as originally reported; 4) When the sink is set to
> "queue2 ! fakesink" I get nothing on the screen (as expected), and no debug
> of interest; 5) When the sink is set to "queue2 ! fakesink sync=1" I get
> nothing on the screen and debug output as originally reported.
> 
> These were all run from the command line, no 'X' running.
> 
> It looks like queue2 + sync=1 (default for v4lsink) combination causes the
> problem... Setting sync=0 for v4lsink makes no difference and looks to be
> ignored.

Do you actually need queue2 or sync=1?

Your end goal is to get everything into your opengl context and render it, 
right?
Shouldn't you be good with setting up a minimal gstreamer pipeline and let 
gstreamer handle the rest?


Don't get me wrong. I'm not an gstreamer expert :)
I'm just trying to give any (possibly) useful hint I have :)


> 
> Things also 'get confused' at times and 1) doesn't work anymore without a
> restart.

I've seen this as well .. the framebuffer gets black and nothing seems to be 
able to render to it anymore ... I think for me it's something with vpu vs. 
gpu memory (which is the same 'allocation' on the unified-memory)

See:
https://community.freescale.com/docs/DOC-93591

Do you see any allocation errors?
I'm going to play with 'gpumem' today to see if I can get better/stable 
results.

> 
> Does any of that help? ;-)


Another thing I spotted:
I just looked at your original mail and from the work I've done last week I 
can tell that I've never seen* or therefor used  x-raw-rgb
* 'seen' as in: source file having rgb format

For me there was no need ... I'm having a sink (Qt/c++ code) which takes x-
raw-yuv ... this code give the frame over to the opengl-scenegraph and then I 
map this memory with code like:

void *bits = (void*)mFrame.bits();
GLuint physical = ~0U;	
glTexDirectVIVMap_LOCALE(GL_TEXTURE_2D, w, h, GL_VIV_YV12, &bits, &physical);


This is then directly used in the shaders without any YUV->RGB code. The 
texture units knows YUV and does the texture-coordinate/color-lookup 
accordingly.
 ... what I'm trying to say: you don't have to convert to GL_RGB(A).



> 
> Chris Tapp
> 
> opensource at keylevel.com
> www.keylevel.com