[Automated-testing] Opentest's test results

[Hernandez, Carlos]

Per discussions today, I'm sharing the way Opentest handles test results.
There is a service called tlwriter that abstracts the communication with the test management system or backend repository, see service command help below.

$ staf local tlwriter help
Response
--------
TestlinkClient Service Help

UPDATE PLANID <plan_id> BUILDID <build_id> TESTCASE <testcase_id> STATUS <p|f|x> [PLATFORMID <platform_id>] [NOTES <notes>] [PERF <name:units:min:max:s0:s1:s2,...>] [DEVKEY <user devkey>] [EXECTIME <sec>]
Where:
STATUS = p|f|x. p for pass, f for fail and x means a skip test (e.g. required config not available, setup problem detected, test script exception, etc.)
NOTES = any optional test result comments and a link to the test logs
PERF s0= number of samples, s1= sum(samples), s2=sum(samples^2). Storing s0, s1 and s2 makes it more efficient to calculate mean and stddev because you don't have to store all individual samples but only those 3 values.
EXECTIME = total test execution time in seconds

Example:
staf local tlwriter UPDATE PLANID 4005361 BUILDID 3856064 TESTCASE 2000321 STATUS p NOTES "Test passed. Application exited with zero. <p><a href=\"http://gtsystest.telogy.design.ti.com/vatf/tigtfarm04/tigtfarm04_vatf@dra72x-evm/tigtfarm0410_09_2018_07_43_57/files/session_iteration_1/test_2000321/dev_test_perf_gov_1.html\" target=\"_blank\">LOG PATH</a></p>" PERF "lat_syscall-write:us:0.5286:0.5286:1:0.5286:0.27941795999999997,lat_syscall-read:us:0.9269:0.9269:1:0.9269:0.85914361"

Regards,
Carlos

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/automated-testing/attachments/20181025/a212506e/attachment.html>