[meta-freescale] [meta-fsl-arm] Using gstreamer on Nitrogen6x/SabreLite

[Gary Thomas]

On 2014-03-05 11:06, Carlos Rafael Giani wrote:
> On 2014-03-05 18:59, Eric Nelson wrote:
>> Hi Gary,
>>
>> On 03/05/2014 10:52 AM, Gary Thomas wrote:
>>> On 2014-03-05 10:44, Eric Nelson wrote:
>>>> Hi Gary,
>>>>
>>>> On 03/04/2014 08:03 AM, Gary Thomas wrote:
>>>>> I have a SabreLite with the OV5642 camera.  I'd like to capture
>>>>> some video and display it on the screen.  Here's my gstreamer
>>>>> pipeline:
>>>>>
>>>>> gst-launch -e -vvv mfw_v4lsrc device=/dev/video0 num-buffers=100 typefind=true \
>>>>>      ! "video/x-raw-yuv, format=(fourcc)I420, width=640, height=480, framerate=(fraction)30/1" \
>>>>>      ! ffmpegcolorspace \
>>>>>      ! ximagesink
>>>>>
>>>>> What I don't understand is why the format from mfw_v4lsrc has to
>>>>> be I420 when the OV5642 [kernel] driver seems to only support YUYV
>>>>> (yuyv422)?
>>>>>
>>>>> To further confuse, I can grab a frame like this:
>>>>>     yavta -fYUYV -s640x480 -F -c1 /dev/video0
>>>>> and the V4l2 subsystem tells me this sensor is YUYV:
>>>>>     root at nitrogen6x:~# v4l2-ctl -d /dev/video0 --list-formats
>>>>>     ioctl: VIDIOC_ENUM_FMT
>>>>>           Index       : 0
>>>>>           Type        : Video Capture
>>>>>           Pixel Format: 'YUYV'
>>>>>           Name        :
>>>>
>>>> This appears to be a bug in the video driver (mxc_v4l2_capture) with
>>>> respect to enumeration (not a strong point for the drivers).
>>>>
>>>> The driver (mxc_v4l2_capture) appears to support a wide variety of
>>>> pixel formats through the magic of the IPU.
>>>>
>>>> The mfw_v4lsrc plugin appears to be hard-coded to support only
>>>> UYVY and I420 on i.MX6 though (and NV12 on i.MX51).
>>>>
>>>> Note that your pipeline is very expensive, and would benefit from
>>>> using a sink that can support YUV natively (mfw_isink, mfw_v4lsink)
>>>> or that can do the conversion in hardware (glimagesink).
>>>
>>> Thanks.  I only used ximagesink as an example (one that also
>>> works on the desktop) for others to see.  In the end, the video
>>> will probably be packaged into some container format (.mp4) and/or
>>> streamed, so those other methods don't help much.
>>>
>>> Is it possible to use the v4l2src gstreamer element with this video?
>>> I haven't been able to make that work at all...
>>>
>> Not without some hacking of the kernel driver(s), and after that,
>> you'd need to figure out how to handle allocation of DMA'able
>> buffers, and so on.
>>
>> If you were to undertake that, I'd recommend taking a hard look
>> at Carlos's GStreamer-1.0 in the process, since I understand that
>> the buffer-chaining process is different in 1.0 (and cleaner in 1.0).
> 
> 
> Philip Craig kindly submitted patches to the gstreamer-imx project to add a v4l2src element that uses i.MX6 specifics. And yes, the allocation scheme is much cleaner in GStreamer
> 1.0. In 0.10, it was only possible through hacks. I implemented an imxeglvivsink in gstreamer-imx, which is a video sink that outputs YUV frames located in DMA buffers with OpenGL
> ES and the Vivante GPU drivers' direct texture extension. This extension allows for using DMA buffers as pixel source for OpenGL ES textures. Net result is that the frames are
> shown directly without involving a CPU copy. I usually measure ~7% CPU usage for typical 1080p videos , more if some nontrivial audio encoding has been used (like AC3).

Is this something that I can [easily] test using my Yocto setup?
If so, do you have a pointer to your gstreamer elements (layer)?

> 
> ximagesink should never be used for video (unless it is the only sink that is available). When possible, either use a platform specific sink, like the mfw ones for 0.10 of
> imxeglvivsink in my 1.0 plugins, or at least xvimagesink. (But I realize that XVideo is not exposed in the X11 drivers..)

Understood :-)

Thanks again

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------