[yocto] Conditional patches on kernel depending on board, how to maintain?

[Daniel Hilst Selli]

I have a SoM which will be used on several boards, this SoM has a base 
kernel for it, with its board-*.c file. In each board I may have 
different peripherals, so I have to patch the same board-*.c file 
depending on my target board, and that patches may be conflicting one 
each other. For example, I could have a RF on first SPI bus on one 
board, and on another board a SD card on same first SPI bus.

So basically I will have a different kernel(uImage) and rootfs (with 
kernel modules) for each board.

I think to create a layer for each target board, with the 
linux-SoM.bbappend including the patches for that board..., so I enable 
the layer depending on target board I'm creating, but is too much file 
editions, or have a build directory for each target board, enabling the 
right layer on each local.conf, but this means mantaining build 
directories, or at last local.conf, which doesn't seem a good idea for me...

Would be possible to do this relying only new layers and its configurations?

Thanks in advance
Cheers!