[Automated-testing] LTP and test metadata

[Tim.Bird at sony.com]

> -----Original Message-----
> From: Cyril Hrubis
> 
> Hi!
> As promised I've continued to work on the test metadata
> export/extraction and apart from hacking on actuall code I've done the
> most important part, wrote a README where I put all my ideas I've come
> to when I was keeping the implementation on the backburner.
> 
> You can see both the new proof of concent along with the documentation
> at:
> 
> https://github.com/metan-ucw/ltp/tree/master/docparse
> 
> Ideally I would love to get some feedback before I return to hacking on
> the code.

Here are some initial impressions, unrelated to the discussion points between you and Daniel.

1) I really like the enumeration in the README.me about the rationale and use cases for this
work.  This is a good articulation of the purpose of this feature and it's possible benefits.

There are a few typos - do you want me to submit a patch to fix them up, or just point them
out?

2) I think it would be nice to add some more explanation of how the dependencies are expressed
in the code, along with some examples, to the README.md.  I looked in some of the source
files but didn't find some clear examples of how to add this to tests.  Maybe some tests already have
this information, and you could reference those?

3) json output seems like a good choice.

It would be nice to see a fragment of the json output, to see how the tst_test data is converted.
Also, you have the all the json for all tests in a single file.  Is there any option to get just the json
for a sub-set of tests, or for an individual test?  Writing a filter from the single global file for individual
test should be trivial, and could be external to your system, I suppose.

3) the notion that the implementation might change as you get actual consumers is a good point.

I'll respond to points on the other thread separately.

I was just talking to Shuah Kahn about dependency data in kselftest.  kselftest is in a unique
position, due to it being inside the kernel source tree, but some of the same issues apply.  Right now
I believe that kselftest only supports kernel config dependencies, but I need to do more research.

If we could get harmony between LTP, kselftest, Fuego, LKFT and 0-day in terms of dependency expressions
for sub-tests, that would be very beneficial.  (I think that might be enough to establish a de-facto
standard in this area.)  I'm planning on working on comparing different dependency formats as part of
phase 1 of my test definition standards research.

Thanks,
 -- Tim