[meta-freescale] [meta-fsl-arm] Using gstreamer on Nitrogen6x/SabreLite

[Philip Craig]

On Thu, Mar 6, 2014 at 4:53 AM, Gary Thomas <gary at mlbassoc.com> wrote:
> On 2014-03-05 11:06, Carlos Rafael Giani wrote:
>> On 2014-03-05 18:59, Eric Nelson wrote:
>>> Hi Gary,
>>>
>>> On 03/05/2014 10:52 AM, Gary Thomas wrote:
>>>> On 2014-03-05 10:44, Eric Nelson wrote:
>>>>> Hi Gary,
>>>>>
>>>>> On 03/04/2014 08:03 AM, Gary Thomas wrote:
>>>>>> I have a SabreLite with the OV5642 camera.  I'd like to capture
>>>>>> some video and display it on the screen.  Here's my gstreamer
>>>>>> pipeline:
>>>>>>
>>>>>> gst-launch -e -vvv mfw_v4lsrc device=/dev/video0 num-buffers=100 typefind=true \
>>>>>>      ! "video/x-raw-yuv, format=(fourcc)I420, width=640, height=480, framerate=(fraction)30/1" \
>>>>>>      ! ffmpegcolorspace \
>>>>>>      ! ximagesink
>>>>>>
>>>>>> What I don't understand is why the format from mfw_v4lsrc has to
>>>>>> be I420 when the OV5642 [kernel] driver seems to only support YUYV
>>>>>> (yuyv422)?
>>>>>>
>>>>>> To further confuse, I can grab a frame like this:
>>>>>>     yavta -fYUYV -s640x480 -F -c1 /dev/video0
>>>>>> and the V4l2 subsystem tells me this sensor is YUYV:
>>>>>>     root at nitrogen6x:~# v4l2-ctl -d /dev/video0 --list-formats
>>>>>>     ioctl: VIDIOC_ENUM_FMT
>>>>>>           Index       : 0
>>>>>>           Type        : Video Capture
>>>>>>           Pixel Format: 'YUYV'
>>>>>>           Name        :
>>>>>
>>>>> This appears to be a bug in the video driver (mxc_v4l2_capture) with
>>>>> respect to enumeration (not a strong point for the drivers).
>>>>>
>>>>> The driver (mxc_v4l2_capture) appears to support a wide variety of
>>>>> pixel formats through the magic of the IPU.
>>>>>
>>>>> The mfw_v4lsrc plugin appears to be hard-coded to support only
>>>>> UYVY and I420 on i.MX6 though (and NV12 on i.MX51).
>>>>>
>>>>> Note that your pipeline is very expensive, and would benefit from
>>>>> using a sink that can support YUV natively (mfw_isink, mfw_v4lsink)
>>>>> or that can do the conversion in hardware (glimagesink).
>>>>
>>>> Thanks.  I only used ximagesink as an example (one that also
>>>> works on the desktop) for others to see.  In the end, the video
>>>> will probably be packaged into some container format (.mp4) and/or
>>>> streamed, so those other methods don't help much.
>>>>
>>>> Is it possible to use the v4l2src gstreamer element with this video?
>>>> I haven't been able to make that work at all...
>>>>
>>> Not without some hacking of the kernel driver(s), and after that,
>>> you'd need to figure out how to handle allocation of DMA'able
>>> buffers, and so on.
>>>
>>> If you were to undertake that, I'd recommend taking a hard look
>>> at Carlos's GStreamer-1.0 in the process, since I understand that
>>> the buffer-chaining process is different in 1.0 (and cleaner in 1.0).
>>
>>
>> Philip Craig kindly submitted patches to the gstreamer-imx project to add a v4l2src element that uses i.MX6 specifics. And yes, the allocation scheme is much cleaner in GStreamer
>> 1.0. In 0.10, it was only possible through hacks. I implemented an imxeglvivsink in gstreamer-imx, which is a video sink that outputs YUV frames located in DMA buffers with OpenGL
>> ES and the Vivante GPU drivers' direct texture extension. This extension allows for using DMA buffers as pixel source for OpenGL ES textures. Net result is that the frames are
>> shown directly without involving a CPU copy. I usually measure ~7% CPU usage for typical 1080p videos , more if some nontrivial audio encoding has been used (like AC3).
>
> Is this something that I can [easily] test using my Yocto setup?
> If so, do you have a pointer to your gstreamer elements (layer)?

It should be easy to test. It's already in meta-fsl-arm layer:
http://git.yoctoproject.org/cgit/cgit.cgi/meta-fsl-arm/tree/recipes-multimedia/gstreamer/gstreamer1.0-plugins-imx_0.9.2.bb