[meta-freescale] fsl-community-bsp: signing up for testing

[Daiane Angolini]

> In my opinion, the testing needs to be split into two phases:
> 1) help/guidelines/docs on testing
> 2) reporting

Continuous test? Or code freeze followed by test cycle?

>
> I think we can agree that Daiane has done an excellent job on the
> reporting side: to create forms to collect and aggregate the test result
> information.
>
> I think where we could use more work (or maybe this already exists,
> perhaps I'm just not aware of it?) is on the "how to test" side.

It looks like dumb, but "how to test" is always a bottle neck.

I personally does not believe in benchmarking, because they are used
as a global measurement tool, and in my point of view they only
provide limited results. The same way, I believe any test must be tied
with some boundary condition. And this is the definition of "how to
test" and "how to read the result"

Only to be remember. In test cycle from 1.6 it was the very first time
we (Otavio mainly) provided all the pre-built target image to be
tested. And the form has all the command lines to be used, or open
answers.

>
> If we had a set of procedures that were so "dumbed down" that a person
> with little OE experience could follow them, perhaps it would be easier
> to get more members of the community involved in testing?

We provided the pre-built images last time. And Otavio has been
keeping it online since them.

More "no OE experience" than download and burn sdcard is impossible.


>
> Or, even better than documentation, what if we had build targets that
> were functional and ready to run "out of the box" for various tests? Why
> isn't it good enough to simply run "bitbake core-image-weston" or
> "fsl-image-multimedia"? Why are tweaks always required to local.conf
> (i.e. DISTRO_FEATURES, EXTRA_IMAGES, PREFERRED_PROVIDER)? If the
> multimedia target _always_ requires the gstreamer packages to be added,
> shouldn't that happen automatically when someone builds that target?

fsl-image-machine-test was created focusing it.

>
> We should probably also look into those ptests to see if they can't help
> with some of this testing/reporting.


Absolutely agree!





Daiane