[meta-xilinx] AMP using Yocto meta-xilinx

[Edward Wingate]

On Wed, May 20, 2015 at 7:36 AM, Nathan Rossi <nathan at nathanrossi.com> wrote:
> Hi Edward,
>
> Just looking at that stack trace it appears that the remoteproc driver
> is failing to allocate the IPI (the set_ipi_handler is a give away, as
> the other functions above it look debug related) they are also known
> as SGIs in the Zynq TRM.
>
> It is worth checking whether the IPI number setup in your device tree
> is actually free to be allocated for the remoteproc driver. You can
> check what is already allocated by the kernel by looking at
> /proc/interrupts.
>
> $ cat /proc/interrupts
>             CPU0
> <snip>
> IPI1:          0          0  Timer broadcast interrupts
> IPI2:          0          0  Rescheduling interrupts
> IPI3:          0          0  Function call interrupts
> IPI4:          0          0  Single function call interrupts
> IPI5:          0          0  CPU stop interrupts
> IPI6:          0          0  IRQ work interrupts
> IPI7:          0          0  completion interrupts
> Err:          0

My IPIs used are just like what you had:
IPI1:          0         11  Timer broadcast interrupts
IPI2:       1144       1100  Rescheduling interrupts
IPI3:          0          0  Function call interrupts
IPI4:         30         67  Single function call interrupts
IPI5:          0          0  CPU stop interrupts
IPI6:         93         52  IRQ work interrupts
IPI7:          0          0  completion interrupts

I tried ipino of 0, 8 and 15 for zynq_remoteproc, and all behave the
same way, with the same error.  Though despite the error, the IPI
handler does seem to get properly registered no matter which of the 3
ipino I use:
IPI15:          0          0  Firmware kick

The source of the kernel error seems to be some printk rate limiting code:
https://github.com/Xilinx/linux-xlnx/blob/master/lib/smp_processor_id.c#L39
https://github.com/Xilinx/linux-xlnx/blob/master/kernel/printk/printk.c#L2679

Ultimately caused by this:
https://github.com/Xilinx/linux-xlnx/blob/master/lib/ratelimit.c#L28

But maybe more relevant, none of the earlier checks before line 39
caused an exit to a "good" state to occur:
https://github.com/Xilinx/linux-xlnx/blob/master/lib/smp_processor_id.c#L15

This is my first dive into kernel code, but will try to figure out
what's going on.
Being my first dive, any advice would be appreciated!

> The remoteproc driver and some of the other AMP related stuff is all
> in a state of flux (at least with respect to the linux-xlnx vendor
> tree). As Xilinx is moving forward with the OpenAMP project
> (https://github.com/OpenAMP/open-amp), it relies on some of the
> existing kernel drivers/features like remoteproc. Might be worth a
> look, although I am under the impression it is not quite ready just
> yet.

I may give OpenAMP a try later, but want to continue in this vein
since I have it partly working and gotten to understand about 90% of
it.  Don't want to give up just yet.

Also, the Xilinx AMP documents ug978 and xapp1078 propose completely
distinct methods to AMP, is that right?  ug978 uses zynq_remoteproc
and xapp1078 uses a value in a fixed memory location to trigger CPU1
app.  Is one approach recommended over the other?  I suppose OpenAMP
will be THE approach at some point.