Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[analyser] analyse metrics in case of timeout during recording phase (broken bag file) #134

Open
floweisshardt opened this issue Feb 27, 2020 · 3 comments

Comments

@floweisshardt
Copy link
Owner

if the recording gets aborted (equvivalent to CTRL-C)

max time [360.0s] allotted for test [atf_recording_ts0_c0_r0_e0_s0_0] of type [atf_core/sm_test.py]

we get somthing like

 - test 'ts0_c0_r0_e0_s0_0' (cob4-2, ipa-apartment, test_default, testblockset_ipa-apartment): 
   - testblock 'testblock_forward': 
     - metric 'interface': no result
     - metric 'tf_length_translation': no result
     - metric 'tf_length_translation': no result
     - metric 'user_result': no result
     - metric 'time': no result
   - testblock 'testblock_all': 
     - metric 'interface': no result
     - metric 'tf_length_translation': no result
     - metric 'tf_length_translation': no result
     - metric 'user_result': no result
     - metric 'time': no result
   - testblock 'testblock_backward': 
     - metric 'interface': no result
     - metric 'tf_length_translation': no result
     - metric 'tf_length_translation': no result
     - metric 'user_result': no result
     - metric 'time': no result

Nevertheless there should be an (unindexed) bag file which can be used to evaluate metrics like path length, time etc. test result however should be False and error message should be test did not finish corretly

@fmessmer
Copy link
Collaborator

fmessmer commented Feb 27, 2020

here is a good example of what happens:
https://travis-ci.com/mojin-robotics/mojin_atf/jobs/291401418#L6993-L9323

i.e. you can see the "all-no-result" result for the rosunit-analysing.xml test - see above
and you can see the TimeoutException for the individual testblocks:

-------------------------------------------------

Full test results for 'build/mojin_atf_navigation/test_results/mojin_atf_navigation/rostest-test_generated_recording_ts0_c0_r1_e0_s0_0.xml'

-------------------------------------------------

<?xml version="1.0" encoding="utf-8"?>

<testsuite errors="1" failures="0" name="unittest.suite.TestSuite" tests="1" time="392.993"><testcase classname="rostest.runner.RosTest" name="testatf_recording_ts0_c0_r1_e0_s0_0" time="392.9925"><error type="RLTestTimeoutException">max time [360.0s] allotted for test [atf_recording_ts0_c0_r1_e0_s0_0] of type [atf_core/sm_test.py]

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run

    testMethod()

  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rostest/runner.py", line 148, in fn

    self.test_parent.run_test(test)

  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rostest/rostest_parent.py", line 132, in run_test

    return self.runner.run_test(test)

  File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/launch.py", line 684, in run_test

    (test.time_limit, test.test_name, test.package, test.type))

</error></testcase><system-out>&lt;![CDATA[

[ROSTEST]setup[/root/target_ws/build/mojin_atf_navigation/test_generated/recording_ts0_c0_r1_e0_s0_0.test] run_id[347cb7be-590c-11ea-8e32-0242ac110002] starting

&amp;#x001B;[1mstarted roslaunch server http://63dfb92e9205:35971/&amp;#x001B;[0m

@fmessmer
Copy link
Collaborator

also, for the testblocks you see the huge print of all ROS parameters and nodes being started and killed (see below) - would there be a way to not have this printed?

@floweisshardt
Copy link
Owner Author

also, for the testblocks you see the huge print of all ROS parameters and nodes being started and killed (see below) - would there be a way to not have this printed?

this should be tracked in #122

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants