CI/Simulator: Drive a loop in metadrive

[bongbui321]

Resolve #30694

Currently, the simulator will fail when it tries to cross lane , and can change the track_size for CI testing. can test by ./test_metadrive_bridge.py 30, which would be a fail case on the curve

Try and run test locally with this cmd: pytest tools/sim/tests/test_metadrive_bridge.py --time_done=<seconds>

Depends on: metadriverse/metadrive#710

• deterministic test
  - [ ] update poetry Metadrive maintainer is a while to review, detect out of lane locally for now
  - [ ] maybe more test cases (lead car, etc) traffic is extremely expensive even running locally with decent gpu

[github-actions]

Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:

• Convert your PR to a draft unless it's ready to review
• Read the contributing docs
• Before marking as "ready for review", ensure:
  • the goal is clearly stated in the description
  • all the tests are passing
  • the change is something we merge
  • include a route or your device' dongle ID if relevant

[bongbui321]

Tried running locally with pytest tools/sim/tests/test_metadrive_bridge.py --time_done=600 (10mins). can modify to run the loop as long as you want it to be

[adeebshihadeh]

Took me a few attempts to get this running locally. Seems like there were two main issues:

UserWarning: PyOpenCL compiler caching failed with an exception:
[begin exception]
Traceback (most recent call last):
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pyopencl/cache.py", line 495, in create_built_program_from_source_cached
    _create_built_program_from_source_cached(
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pyopencl/cache.py", line 417, in _create_built_program_from_source_cached
    prg.build(options_bytes, [devices[i] for i in to_be_built_indices])
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pyopencl/__init__.py", line 772, in program_build
    compiler_output("%s succeeded, but resulted in non-empty logs:\n%s"
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/site-packages/pyopencl/__init__.py", line 273, in compiler_output
    warn("Non-empty compiler output encountered. Set the "
pyopencl.CompilerWarning: Non-empty compiler output encountered. Set the environment variable PYOPENCL_COMPILER_OUTPUT=1 to see more.
[end exception]

and

State:
Ignition: True Engaged: False


State:
Ignition: True Engaged: False
    Waiting for CAN messages...
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd
Process bridge:
Traceback (most recent call last):
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/batman/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/dev/shm/openpilot/threepilot/openpilot/tools/sim/bridge/common.py", line 71, in bridge_keep_alive
    self._run(q)
  File "/dev/shm/openpilot/threepilot/openpilot/tools/sim/bridge/common.py", line 197, in _run
    self.print_status()
  File "/dev/shm/openpilot/threepilot/openpilot/tools/sim/bridge/common.py", line 88, in print_status
    print(
BlockingIOError: [Errno 11] write could not complete without blocking

modeld init
setting up CL context
CL context ready; loading model
Onnx available providers:  ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'AzureExecutionProvider', 'CPUExecutionProvider']
Onnx selected provider:  [('CUDAExecutionProvider', {'cudnn_conv_algo_search': 'DEFAULT'})]
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd
Onnx using  ['CUDAExecutionProvider', 'CPUExecutionProvider']
ready to run onnx model {'input_imgs': [1, 12, 128, 256], 'big_input_imgs': [1, 12, 128, 256], 'desire': [1, 100, 8], 'traffic_convention': [1, 2], 'lateral_control_params': [1, 2], 'prev_desired_curv': [1, 100, 1], 'features_buffer': [1, 99, 512]}
models loaded, modeld starting
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd
logcatd proclogd modeld ui soundd locationd calibrationd torqued controlsd deleter navd paramsd plannerd radard thermald statsd

[bongbui321]

BlockingIOError: [Errno 11] write could not complete without blocking

Never encounter this issue, through all my run and just tried 100 run locally and still don't get this, but my best guess is that because printing out to the same stdout (both the metadrive bridge and OP) which causes a small chance for them trying to print at the same time.

3b644dd change makes it don't print the metadrive state during test, which is for interactive purposes in the first place, and the unittest already covered always be able to engage

[adeebshihadeh]

Had no issues running locally now. However, it's easy to make the test pass when it shouldn't with this patch. I think adding a check on distance (based on the time) should be enough. Other than that, seems good to go.

diff --git a/tools/sim/bridge/common.py b/tools/sim/bridge/common.py
index 64407143f..997a2e0cf 100644
--- a/tools/sim/bridge/common.py
+++ b/tools/sim/bridge/common.py
@@ -184,6 +184,7 @@ Ignition: {self.simulator_state.ignition} Engaged: {self.simulator_state.is_enga
       brake_out = brake_op if self.simulator_state.is_engaged else brake_manual
       steer_out = steer_op if self.simulator_state.is_engaged else steer_manual

+      throttle_out = 0.
       self.world.apply_controls(steer_out, throttle_out, brake_out)
       self.world.read_state()
       self.world.read_sensors(self.simulator_state)

[bongbui321]

The current distance vs time check is decent for now. I intend to add fps check and OP time in CI vs real time check in the metadrive GHA bounty. Check for staying still at 5s works locally but needed 30s in CI due it being slower, will improve this for the CI bounty

[adeebshihadeh]

Nice work! Looking forward to getting this in CI.