[yocto-ab] Member involvement in the core of the project

[Khem Raj]

> On Sep 30, 2016, at 7:14 AM, Richard Purdie <richard.purdie at linuxfoundation.org> wrote:
> 
> On Fri, 2016-09-30 at 09:31 -0400, Chris Hallinan wrote:
>> My first question is: why don't we do more to enforce member
>> participation?
> 
> I am leading to that question :). Part of the issue is how we'd manage
> it (and who would manage it?) but things like the recipe reporting
> would give some metrics at least. The who part does worry me as I don't
> really need more to do so whilst I could be well qualified, I'm also
> overloaded.
> 
> So how do we make that work?

Current membership seems to not cover the technical contribution
aspects of membership

Every member designates one or more engineering resource as yocto
maintainer for their organization, who would participate in yocto
technical activities like, weekly technical status meeting and
contribute some percentage ( put a number ) of time to doing things
like bug SWAT activities, They may be also interested/assigned
issues that interest their organization more than others. They
also take the new changes during development and leverage their
testing or development resources as needed.

I would think if some members use extra layers beyond OE-Core
then they contribute by bringing these layers to the quality
levels of OE-Core and integrate into the auto builder testing
process, etc. there are these kind of contributions that also
helps the project I believe.

May be the member organizations can replicate the testing
infrastructure for their interesting layers and machines and
provide additional test coverage for more machines which match
the qemu and reference BSP testing that project does on its own


> 
>>   Perhaps it should be a hard requirement of membership and badging.
> 
> It is already a membership requirement.
> 
>> Personally, since 1) I am not a developer and 2) I have many demands
>> for my time, I myself am not familiar with all of the testing
>> infrastructure, and I'd welcome a formal 1-hour webex-style
>> presentation on the subject.
> 
> Its certainly something I think we need to raise awareness of and that
> would be one way to do it. There are others at Mentor who could also
> probably do with being aware of more about it (including Chris
> Larson!).
> 
>> I wasn't aware of that recipe reporting system, is that basically
>> just meta-oe/poky?  Very nice.
> 
> Right now its basically running for oe-core but we maintainer the
> maintainers list in meta-yocto since it is something I consider to be
> "Yocto Project" owned rather than OE owned.
> 
> There is nothing in the system that would stop it being used for other
> layers. The UI isn't really designed to filter to a layer right now so
> would probably need to be a different instance but that is the kind of
> thing which can be fixed if there is demand. All the code powering it
> is on git.yoctoproject.org.
> 
> 
>> I know we at Mentor have never made use of autobuilder, etc.  I've
>> had discussions internally about that, and I'm certain there is value
>> there, but we have our own internal testing infrastructure which as
>> you can imaging, has had quite significant investment, as I'm sure is
>> true for all of the commercial software vendors and likely the semi
>> manufacturers, too.  Maybe part of the discussion is how to get more
>> member participation there.
> 
> I think the autobuilder gets a lot of bad press and there is a lot of
> misunderstanding which leads to that. Our testing strategy has multiple
> different levels:
> 
> a) recipe level ptest packages
>     [equivalent to "make test" in the source]
> b) sanity.bbclass global configuration testing
>     [does networking work, is MACHINE set?]
> c) insane.bbclass build and package level testing
>     [is the binary we just cross compiled of the right arch?]
>     [did configure look in the host system for something(bad)?]
> d) testimage
>     [does the image we just built boot?]
>     [do things work? networking came up? python works? compiler works?]
> e) testsdk
>     [can the sdk we just built compile things?]
> f) test esdk
>     [does the extensible sdk we just built work?]
> g) oe-selftest
>     [if we do X, then Y, the Z does the right things happen, tests
>      specfic configs or combinations of commands]
> h) bitbake-selftest - unitests for bitbake internal APIs
>     [Does the XXX fetcher within bitbake do the right things?]
> 
> All of the above works outside of the autobuilder and tests different
> facets of the project and its technology. The autobuilder (buildbot)
> just makes sure we build a load of different combinations of settings
> and makes sure all the above forms of testing get called (except for
> ptest which we still manually handle right now, much to my
> frustration).
> 
> So you don't have to switch to or use autobuilder(buildbot) to make use
> of other areas of testing within the project. Admittedly most of the
> above is mostly run against qemu, because that is what we have
> available in our public infrastructure. The QA people do run it against
> real hardware too though so it can be done.
> 
> I appreciate others do have their own systems and each has value. I do
> think buildbot on the core does do a phenomenal job when you look at
> the size of the test matrix and the shear number of combinations which
> it does on a comparatively small amount of hardware in a comparatively
> reasonable time. Trying to replace it out for jenkins or move it to the
> cloud or replace it with lava would likely cost us a lot of work and
> not really gain us much. If we could figure out a way to share some of
> the internal well abstracted pieces listed above, we'd likely gain a
> lot more.
> 
>> At a minimum we should put the subject on the agenda for next month's
>> face-to-face.
> 
> Agreed, I mention it now to get people thinking about it.

+1.

> 
> Cheers,
> 
> Richard
> --
> _______________________________________________
> yocto-ab mailing list
> yocto-ab at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto-ab

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 204 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.yoctoproject.org/pipermail/yocto-ab/attachments/20161002/aec4a47a/attachment.pgp>