Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adding webots dynamic scenario tests #231

Merged
merged 15 commits into from
Dec 20, 2024

Conversation

abanuelo
Copy link
Contributor

@abanuelo abanuelo commented Apr 9, 2024

Description

Adding a webots dynamic scenario test where an object is created above a plane and then drops until it hits the floor. The test checks the expected start and ending z values for the object's position. Note, that webots runs headlessly via a python subprocess. This subprocess then creates a results.txt file which contains the starting and ending z positions. Below is an illustration of the test case running on our machine:

testing_2.mp4

Issue Link

Checklist

  • I have tested the changes locally via pytest and/or other means
  • I have added or updated relevant documentation
  • I have autoformatted the code with black and isort
  • I have added test cases (if applicable)

Additional Notes

@abanuelo abanuelo requested review from dfremont and Eric-Vin April 9, 2024 22:32
Copy link

codecov bot commented Apr 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.36%. Comparing base (d7679fb) to head (89dd220).
Report is 23 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #231      +/-   ##
==========================================
+ Coverage   91.32%   91.36%   +0.03%     
==========================================
  Files          54       54              
  Lines       13515    13577      +62     
==========================================
+ Hits        12343    12404      +61     
- Misses       1172     1173       +1     

see 15 files with indirect coverage changes

@abanuelo
Copy link
Contributor Author

abanuelo commented Apr 11, 2024

Important!!!!

In order to for dynamic webots tests to work you need a physical or virtual display. If you are:

  1. sshing into the instance and triggering the webots tests manually
  2. OR running the CI feat: adding ci for simulator testing #226 that triggers simulation tests on the instance

it is VERY important you create a virtual screen prior to running the tests!

Xvfb :99 -screen 0 1024x768x16 &
export DISPLAY=:99

If not you will generate the following warning which will automatically fail all webots dynamic scenario tests:

Warning: Could not find the Qt platform plugin "offscreen" in ""
Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.

/usr/local/bin/webots: line 105: 12490 Aborted                 (core dumped) "$webots_home/bin/webots-bin" "$@"

If you are using the DCV client to run the test (as illustrated in the PR description above), the built in DISPLAY port should be fine and no need to run the commands above.

scenario = scenic.scenarioFromFile(path)

scene, _ = scenario.generate()
simulation = simulator.simulate(scene, verbosity=2)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a large upper bound to the time limit so if the test fails, it does so gracefully?

Copy link
Collaborator

@lola831 lola831 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've addressed the comment, added the timeout, and tested the changes locally to ensure everything works as expected.

@lola831 lola831 merged commit 8a2a7e9 into main Dec 20, 2024
21 of 24 checks passed
@lola831 lola831 deleted the abanuelo/add-webots-dynamic-scenario-tests branch December 20, 2024 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants