[Automated-testing] conventions for test invocation/execution, etc.

[Cyril Hrubis]

Hi!
> > This is just an idea, but why don't we standartize on a test description
> > instead of on a shell script name? If we choose right format, it would
> > be backward compatible and we can add more information there later on...
> 
> I think that's a great idea!  In fact, I a variation of that idea independently
> when I woke up this morning.  (I think my sub-conscious is actually smarter
> than I am.  It figures stuff out while I'm sleeping.  :-)

:-)

I'm suspecting that's why I have a real need to procrastinate from time
to time, which is just a result of several background tasks on my
mind. Or at least that's how I excuse it...

> In any event, I'll write up some of my concepts, which includes having the
> test description indicate the test program or test script name, and put them
> on the list for discussion (as well as discuss them with people next week
> at Plumbers and the CKI hackfest).
> 
> Just a quick summary - I think it would be nice to have a common mechanism
> to query a test for:
>  - dependencies
>  - variant name and mechanisms
>  - the test program name (or invocation method)
>  - result format
>  - maybe other info (like that used by LTP to indicate relevance to commit ID, CVEs,
>    how to interpret results, etc.)
> 
> Despite how heavyweight this sounds, I think it could be done more easily
> than one would think.

Looks like we are on the same page here. If we make most of the fields
optional it would be fairly easy to implement this in incremental steps.

And maybe in a future we can get to a point where we can have a library
for schedulling test in parallel across different projects, which would
be really cool.

> I think that what LTP is doing with your dependency and test information
> tooling could be the seed for this type of feature.  A key notion is that this
> information is not carried around independent of the test (like a lot of
> frameworks do today), but that the test or its source could be queried
> (or scanned) for it.
> 
> I'll post more on this later.

Great!

-- 
Cyril Hrubis
chrubis at suse.cz