[linux-yocto] [yocto-kernel-cache][PATCH] features/systemtap/systemtap.cfg: enable CONFIG_KERNEL_DEBUG

[Victor Kamensky]

On Wed, 14 Mar 2018, Bruce Ashfield wrote:

> On 2018-03-14 11:54 AM, Victor Kamensky wrote:
>> 
>> 
>> On Wed, 14 Mar 2018, Bruce Ashfield wrote:
>> 
>>> On 2018-03-14 10:41 AM, Victor Kamensky wrote:
>>>> Hi Bruce,
>>>> 
>>>> On Wed, 14 Mar 2018, Bruce Ashfield wrote:
>>>> 
>>>>> As it turns out, Ross is reporting a big performance degradation in the 
>>>>> kernel
>>>>> in footprint, packaging and build time.
>>>>> 
>>>>> It completely slipped my mind when looking at this commit, that we went 
>>>>> through
>>>>> a split two years ago for that very same issue.
>>>>> 
>>>>> For now, I'm going to revert this,
>>>> 
>>>> Please note that after revert in your regular build kernels will not
>>>> be debuggable, because it is missing symbols. For example for 
>>>> investigation like [1] where kernel hangs where one can pick at
>>>> hangged kernel quickly after 'runnqemu qemuparam="-S -s"'. It is
>>>> not possible without symbols.
>>> 
>>> That's fully understood. But we cannot have that config on for
>>> the standard kernel build. That's what the developer ktype is for.
>>> 
>>>> 
>>>>> and then work systemtap properly into the
>>>>> developer kernel type. The issue is that the standard kernel includes 
>>>>> systemtap
>>>>> by default (it probably shouldn't), and hence is picking up the 
>>>>> DEBUG_KERNEL
>>>>> option .. and that's causing the issues.
>>>> 
>>>> Even if you revert commit please do not remove systemtap fragment
>>>> from regular kernel. I said that without CONFIG_KERNEL_DEBUG
>>>> kernel tracing does not make sense. But besides kernel systemtap
>>>> can do user-land process tracing, this one works OK even if symbols
>>>> are not in kernel.
>>>> 
>>>> Instead of removing systemtap.cfg from regular kernel, it would be
>>>> better to add separate debuginfo.cfg that would enable debug
>>>> symbols kernle and have some quick way through which one can add
>>>> symbols through simple local.conf setting. So if run into situation
>>>> like [1], and need to look at kernel with gdb or run systemtap
>>>> against it, one can just add line to local.conf and rebuild kernel.
>>> 
>>> There's already a fragment for that purpose and it is what the
>>> developer kernel type uses. Anyone can include that fragment via
>>> a bbappend.
>>> 
>>> The developer kernel type includes the standard kernel, so it is
>>> a superset .. so yes, I can leave systemtap in place and the developer
>>> kernel type just finishes the enablement of it.
>> 
>> I will look at kernel ktype mechanism. But if in developper
>> fragment there are some other options, it may not be suitable
>> for use cases that I have in mind. Suppose one runs into the
>
> We can always make the fragments more granular, what is there is
> just the first pass.
>
>> problem with kenel and would like to debug it with gdb or write
>> systemtap script to investigate it. There should be a
>> way just to add symbols to it and leave all other configs intact,
>> because who knows maybe those config will change kernel behavior
>> and issue won't be reprodicible anymore. So I think there should
>> be a way to enable DEBUG_INFO symbols only quickly for any given
>> config. I'll try to look at how to do it, and will post some
>> RFC patches for it.
>
> Sounds good. Let me know if you need any help on my end.

Just to close on this:

I looked closely and experimented with it. It seems that
everything I need is already in place. If I add

KERNEL_EXTRA_FEATURES_append = " features/debug/debug-kernel.scc"

to my conf/local.conf or conf/site.conf it will do the trick:
kernel will be built with symbols. debug-kernel.cfg contains
DEBUG_PREEMPT besided DEBUG_INFO, but it looks like it does not
make sense to move just DEBUG_INFO out because DEBUG_PREEMPT
default is 'y' and it will be enabled as soon as KERNEL_DEBUG
enabled. If debug-kernel.cfg will ever pick up more content,
maybe factoring out just DEBUG_INFO would make some sense.

Thanks,
Victor

> Bruce
>
>> 
>> Thanks,
>> Victor
>> 
>>> Bruce
>>> 
>>>> 
>>>> [1] 
>>>> http://lists.openembedded.org/pipermail/openembedded-core/2018-March/148587.html 
>>>> Thanks,
>>>> Victor
>>>> 
>>>>> For now, you can build the developer kernel type, or add the developer 
>>>>> kernel
>>>>> configuration fragment via a bbappend, and you'll have a functional 
>>>>> systemtap.
>>>>> 
>>>>> I can move systemtap out of the standard kernel and only into the 
>>>>> developer
>>>>> kernel, since as you say, it isn't functional without this option.
>>>>> But, that ktype
>>>>> hasn't gotten a lot of mileage (despite being around for two years),
>>>>> so I'll need
>>>>> to run some tests before I can truly move the fragment over and confirm 
>>>>> that
>>>>> it works.
>>>>> 
>>>>> So again, for now, I'll revert this, but will work on getting the
>>>>> ktypes fixed up
>>>>> as well.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Bruce
>>>>> 
>>>>> 
>>>>> commit a7b57bb54cace686911faae9a8eace3b9cbf8145
>>>>> Author: California Sullivan <california.l.sullivan at intel.com>
>>>>> Date:   Wed Feb 17 16:47:10 2016 -0800
>>>>> 
>>>>>    ktypes: add developer ktype
>>>>> 
>>>>>    The developer ktype enables EMBEDDED, EXPERT, and DEBUG_KERNEL,
>>>>>    opening up more kernel options and setting some defaults.
>>>>> 
>>>>>    Signed-off-by: California Sullivan <california.l.sullivan at intel.com>
>>>>>    Signed-off-by: Bruce Ashfield <bruce.ashfield at windriver.com>
>>>>> 
>>>>> :000000 100644 000000000000... ee04ec720188... A
>>>>> ktypes/developer/developer.cfg
>>>>> :000000 100644 000000000000... cf2c0e6519d5... A
>>>>> ktypes/developer/developer.scc
>>>>> 
>>>>> 
>>>>> 
>>>>> On Sat, Mar 3, 2018 at 9:03 PM, Victor Kamensky <kamensky at cisco.com> 
>>>>> wrote:
>>>>>> Need to enable CONFIG_KERNEL_DEBUG in systemtap feature, otherwise
>>>>>> CONFIG_DEBUG_INFO set by systemtap.cfg does not have any effect since
>>>>>> it depends on CONFIG_KERNEL_DEBUG and is set off in resulting config.
>>>>>> So systemtap cannot find DWARF information in resulting kernel.
>>>>>> 
>>>>>> Signed-off-by: Victor Kamensky <kamensky at cisco.com>
>>>>>> ---
>>>>>>  features/systemtap/systemtap.cfg | 1 +
>>>>>>  1 file changed, 1 insertion(+)
>>>>>> 
>>>>>> diff --git a/features/systemtap/systemtap.cfg 
>>>>>> b/features/systemtap/systemtap.cfg
>>>>>> index acc2405..858447c 100644
>>>>>> --- a/features/systemtap/systemtap.cfg
>>>>>> +++ b/features/systemtap/systemtap.cfg
>>>>>> @@ -1,4 +1,5 @@
>>>>>>  CONFIG_RELAY=y
>>>>>> +CONFIG_DEBUG_KERNEL=y
>>>>>>  CONFIG_DEBUG_INFO=y
>>>>>>  CONFIG_DEBUG_FS=y
>>>>>>  CONFIG_MODULES=y
>>>>>> -- 
>>>>>> 2.7.4
>>>>>> 
>>>>>> -- 
>>>>>> _______________________________________________
>>>>>> linux-yocto mailing list
>>>>>> linux-yocto at yoctoproject.org
>>>>>> https://lists.yoctoproject.org/listinfo/linux-yocto
>>>>> 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> "Thou shalt not follow the NULL pointer, for chaos and madness await
>>>>> thee at its end"
>>>>> 
>>> 
>>> 
>
>