add container for Opentera webrtc ros #91

wendwosenbb · 2025-01-24T20:49:27Z

adds container for ros2 humble branch of opentera-webrtc-ros.

instruction is provided in the readme file.

mamaheux

Thank you for contributing to the repository. I have left some comments for improving your contribution.

mamaheux · 2025-01-26T16:55:24Z

docker/Dockerfile.opentera_ros2

+    update-ca-certificates
+
+# run bash file
+CMD ["/bin/bash", "-c", "chmod +x ./opentera_teleop_startup.sh && ./opentera_teleop_startup.sh"]


Missing blank line

mamaheux · 2025-01-26T16:56:02Z

docker/Dockerfile.opentera_ros2

+    update-ca-certificates
+
+# run bash file
+CMD ["/bin/bash", "-c", "chmod +x ./opentera_teleop_startup.sh && ./opentera_teleop_startup.sh"]


Maybe put "chmod +x ./opentera_teleop_startup.sh" in a run statement.

docker/README.md

mamaheux · 2025-01-26T17:00:14Z

docker/certificate/cert.pem

@@ -0,0 +1,19 @@
+-----BEGIN CERTIFICATE-----


Can you remove all certificate files from the repository? You can replace them by a bash script for creating them. Make sure you add a .gitignore file containing the certificates. You can add a step in the readme file for generating them.

I would suggest you generate new certificates for your application since these have been exposed on the Internet.

mamaheux · 2025-01-26T17:01:52Z

docker/turtlebot3_house/model.sdf

+    <static>1</static>
+    <allow_auto_disable>1</allow_auto_disable>
+  </model>
+</sdf>


Missing blank line.

mamaheux · 2025-01-26T17:02:51Z

docker/opentera_teleop_startup.sh

+
+echo $ROS_DISTRO
+ros2 launch  startup_opentera_demos.launch.py
+#ros2 run rviz2 rviz2


Add an option to run rviz or remove the line.
Add the missing blank line.

mamaheux · 2025-01-26T17:03:44Z

docker/launch/opentera_demo.launch.xml

+    <!-- Front camera -->
+    <!--group><include file="/opt/ros/OPENTERA_WS/ros_stream_client.launch.xml">
+        <arg name="name" value="topic_streamer1"/>
+        <arg name="is_stand_alone" value="$(var is_stand_alone)"/>
+        <arg name="can_send_audio_stream" value="true"/>
+        <arg name="can_receive_audio_stream" value="false"/>
+        <arg name="can_send_video_stream" value="true"/>
+        <arg name="can_receive_video_stream" value="false"/>
+        <arg name="is_screen_cast" value="false"/>
+        <arg name="needs_denoising" value="false"/>
+        <arg name="server_url" value="http://$(var signaling_server_hostname):$(var signaling_server_port)"/>
+        <arg name="client_name" value="Robot Camera 1"/>
+        <arg name="room_name" value="VideoConf"/>
+        <arg name="room_password" value="$(var signaling_server_password)"/>
+        <arg name="input_camera_topic" value="$(var input_camera_topic_based_on_face_cropping)"/>
+        <arg name="force_gstreamer_video_hardware_acceleration" value="$(var force_gstreamer_video_hardware_acceleration)"/>
+    </include></group-->
+
+    <!-- Bottom camera -->


Remove

Suggested change

mamaheux · 2025-01-26T17:05:02Z

docker/launch/ros_stream_client.launch.xml

+<launch>
+
+    <!-- Arguments -->
+    <arg name="name" default="topic_streamer"/>
+    <arg name="is_stand_alone" default="true"/>
+    <arg name="video_queue_size" default="1"/>
+    <arg name="audio_queue_size" default="1"/>
+
+    <!-- Video -->
+    <arg name="can_send_video_stream" default="false"/>
+    <arg name="can_receive_video_stream" default="false"/>
+    <arg name="is_screen_cast" default="false"/>
+    <arg name="needs_denoising" default="false"/>
+
+    <!-- Codec -->
+    <arg name="forced_video_codecs" default="['']"/>  <!-- "[]" is bugged (https://github.com/ros2/rclcpp/issues/1955), using "['']" and filtering empty strings in the node -->
+    <arg name="force_gstreamer_video_hardware_acceleration" default="false"/>
+    <arg name="use_gstreamer_video_software_encoder_decoder" default="false"/>
+
+    <!-- Audio -->
+    <arg name="can_send_audio_stream" default="false"/>
+    <arg name="can_receive_audio_stream" default="false"/>
+    <arg name="sound_card_total_delay_ms" default="40"/>
+    <arg name="echo_cancellation" default="true"/>
+    <arg name="auto_gain_control" default="true"/>
+    <arg name="noise_suppression" default="true"/>
+    <arg name="high_pass_filter" default="false"/>
+    <arg name="stereo_swapping" default="false"/>
+    <arg name="transient_suppression" default="true"/>
+
+    <!-- Signaling -->
+    <arg name="server_url" default="http://localhost:8080"/>
+    <arg name="client_name" default="streamer"/>
+    <arg name="room_name" default="chat"/>
+    <arg name="room_password" default="abc"/>
+    <arg name="verify_ssl" default="false"/>
+
+    <!-- Topics -->
+    <arg name="input_camera_topic" default="ros_image"/>
+    <arg name="output_webrtc_image_topic" default="webrtc_image"/>
+    <arg name="input_audio_topic" default="audio_in"/>
+    <arg name="output_audio_mixed_topic" default="audio_mixed"/>
+    <arg name="output_webrtc_audio_topic" default="webrtc_audio"/>
+
+    <node name="$(var name)" pkg="opentera_webrtc_ros" exec="topic_streamer" respawn="true"> <!--launch-prefix="xterm -e gdb -ex run args" -->
+        <!-- Enable debug log level -->
+        <param name="log_level" value="debug"/>
+        <param name="is_stand_alone" value="$(var is_stand_alone)"/>
+        <param name="video_queue_size" value="$(var video_queue_size)"/>
+        <param name="audio_queue_size" value="$(var audio_queue_size)"/>
+
+        <param name="video_stream">
+            <param name="can_send_video_stream" value="$(var can_send_video_stream)"/>          <!-- Can send video stream to the signaling server -->
+            <param name="can_receive_video_stream" value="$(var can_receive_video_stream)"/>    <!-- Can receive video stream from the signaling server -->
+            <param name="is_screen_cast" value="$(var is_screen_cast)"/>                        <!-- Is the image source a screen capture? -->
+            <param name="needs_denoising" value="$(var needs_denoising)"/>                      <!-- Does the image source needs denoising? -->
+        </param>
+
+        <param name="video_codecs">
+            <param name="forced_codecs" value="$(var forced_video_codecs)"/>
+            <param name="force_gstreamer_hardware_acceleration" value="$(var force_gstreamer_video_hardware_acceleration)"/>
+            <param name="use_gstreamer_software_encoder_decoder" value="$(var use_gstreamer_video_software_encoder_decoder)"/>
+        </param>
+
+        <param name="audio_stream">
+            <param name="can_send_audio_stream" value="$(var can_send_audio_stream)"/>          <!-- Can send audio stream to the signaling server -->
+            <param name="can_receive_audio_stream" value="$(var can_receive_audio_stream)"/>    <!-- Can receive audio stream from the signaling server -->
+            <param name="sound_card_total_delay_ms" value="$(var sound_card_total_delay_ms)"/>
+            <param name="echo_cancellation" value="$(var echo_cancellation)"/>
+            <param name="auto_gain_control" value="$(var auto_gain_control)"/>
+            <param name="noise_suppression" value="$(var noise_suppression)"/>
+            <param name="high_pass_filter" value="$(var high_pass_filter)"/>
+            <param name="stereo_swapping" value="$(var stereo_swapping)"/>
+            <param name="transient_suppression" value="$(var transient_suppression)"/>
+        </param>
+
+        <param name="signaling">
+            <param name="server_url" value="$(var server_url)"/>         <!-- Signaling server URL used in stand_alone mode only -->
+            <param name="client_name" value="$(var client_name)"/>       <!-- Peer name as which to join the room -->
+            <param name="room_name" value="$(var room_name)"/>           <!-- Room name to join -->
+            <param name="room_password" value="$(var room_password)"/>   <!-- Room password used in stand_alone_mode only -->
+            <param name="verify_ssl" value="$(var verify_ssl)"/>         <!-- SSL peer verification -->
+        </param>
+    <!-- Add GStreamer debug environment variable -->
+        <env name="GST_DEBUG" value="4"/>
+        <remap from="ros_image" to="$(var input_camera_topic)"/>
+        <remap from="webrtc_image" to="$(var output_webrtc_image_topic)"/>
+        <remap from="audio_in" to="$(var input_audio_topic)"/>
+        <remap from="audio_mixed" to="$(var output_audio_mixed_topic)"/>
+        <remap from="webrtc_audio" to="$(var output_webrtc_audio_topic)"/>
+    </node>
+
+</launch>


Why have you duplicated the file?

philippewarren · 2025-01-28T16:43:47Z

Also, don't wory about the CI failing for now: we are still waiting on an update to the tf2 ROS package in the ROS repositories for Ubuntu 22.04, the bug is on their side.

…sary lines and files. added missing blank spaces.

Co-authored-by: Marc-Antoine Maheux <[email protected]>

wendwosenbb · 2025-01-29T18:28:01Z

I have added the changes you requested. @mamaheux

let me know if there is more changes required.

philippewarren · 2025-02-04T15:15:53Z

docker/Dockerfile.opentera_ros2

+RUN apt-get update && apt-get install -y --no-install-recommends \
+  ros-humble-rviz2 \
+  libxcb-cursor0 \
+  libx11-xcb-dev \


Duplicate with line 54?

philippewarren · 2025-02-04T15:17:41Z

docker/Dockerfile.opentera_ros2

+RUN git clone https://github.com/elFarto/nvidia-vaapi-driver.git
+
+# Build and install nvidia-vaapi-driver
+WORKDIR /opt/ros/OPENTERA_WS/nvidia-vaapi-driver


Could colcon later try to build it again since it's in the workspace?

philippewarren · 2025-02-04T15:21:10Z

docker/Dockerfile.opentera_ros2

+# Clone and build other packages
+RUN git clone https://github.com/introlab/audio_utils.git --recurse-submodules -b ros2 && \
+    cd audio_utils && \ 
+    git reset --hard fcc404a84fdec9bcb0c9963f394e4e29d5a9653a && \


git checkout instead of reset --hard seems more appropriate?

philippewarren · 2025-02-04T15:24:30Z

docker/Dockerfile.opentera_ros2

+
+COPY opentera_teleop_startup.sh .
+
+COPY launch .


Should these be placed with other launch files? They will get in the container via the git clone in either case, we don't need to copy them like this. Why is this done?

philippewarren · 2025-02-04T15:27:15Z

docker/opentera_teleop_startup.sh

+export LIBVA_DRIVER_NAME=nvidia
+
+# Add the signaling server to PATH
+export PATH=/opt/ros/OPENTERA_WS/opentera-webrtc-ros/install/opentera_webrtc_ros/local/bin:$PATH


It should be in PATH already after line 3, where we source the workspace setup file.

philippewarren · 2025-02-04T15:37:40Z

docker/opentera_teleop_startup.sh

+export GAZEBO_MODEL_PATH=/opt/ros/OPENTERA_WS/opentera-webrtc-ros/install/opentera_webrtc_demos/share/opentera_webrtc_demos/models:$GAZEBO_MODEL_PATH
+export GAZEBO_MODEL_PATH=/opt/ros/OPENTERA_WS/models:$GAZEBO_MODEL_PATH


Are they not in the MODEL_PATH after sourcing the workspace? We export it here: https://github.com/introlab/opentera-webrtc-ros/blob/ros2/opentera_webrtc_demos/package.xml#L30
If it does not work, this is a bug with the package and should be fixed directly (using ament_environment_hooks in cmake?)

philippewarren · 2025-02-04T15:38:11Z

docker/opentera_teleop_startup.sh

+sudo sysctl -w net.core.rmem_max=2147483647
+
+echo $ROS_DISTRO
+ros2 launch  startup_opentera_demos.launch.py


Double space.
Also, add newline

philippewarren · 2025-02-04T19:59:45Z

docker/docker-compose.yml

@@ -0,0 +1,30 @@
+version: "2.3"


I get WARN: the attribute version is obsolete, it will be ignored, please remove it to avoid potential confusion

philippewarren · 2025-02-04T20:44:56Z

docker/README.md

+Dockerfile.opentera_ros2 contains build docker build instructions. 
+
+## Instructions
+


Add link to install NVidia Container Toolkit for the nvidia container runtime.

philippewarren · 2025-02-05T20:15:32Z

docker/launch/opentera_demo.launch.xml

+        <arg name="can_send_audio_stream" value="true"/>
+        <arg name="can_receive_audio_stream" value="false"/>
+        <arg name="can_send_video_stream" value="true"/>
+        <arg name="can_receive_video_stream" value="false"/>


This needs to be set to "true"

philippewarren · 2025-02-05T20:18:00Z

docker/opentera_teleop_startup.sh

+export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib 
+
+# Set the Video Acceleration API (VA-API) driver to NVIDIA
+export LIBVA_DRIVER_NAME=nvidia


The whole VAAPI stuff was a good idea, but it does not seem to work properly. The default case seems to work, because it chooses VP9, for which we ignore hardware accelerated encoding due to a bug. Testing with VP8 is not working reliably, therefore hardware accelerated support should probably be disabled in docker.

wendwosenbb added 5 commits January 23, 2025 14:13

added docker container for opentera_webrtc_ros

ba0061b

remove test file

ddc19ad

fix container working directory error.

3c3d8ff

fixed docker-host file ownership conflict.

ca75a6c

clean up

068fa5b

wendwosenbb mentioned this pull request Jan 24, 2025

topic_streamer driver issue #89

Closed

mamaheux requested changes Jan 26, 2025

View reviewed changes

wendwosenbb and others added 2 commits January 29, 2025 12:21

certificate generated and installed inside container. clean up uneces…

908359b

…sary lines and files. added missing blank spaces.

Update docker/README.md

66e167b

Co-authored-by: Marc-Antoine Maheux <[email protected]>

philippewarren requested changes Feb 4, 2025

View reviewed changes

philippewarren reviewed Feb 4, 2025

View reviewed changes

philippewarren requested changes Feb 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add container for Opentera webrtc ros #91

add container for Opentera webrtc ros #91

wendwosenbb commented Jan 24, 2025

mamaheux left a comment

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

mamaheux Jan 26, 2025

philippewarren commented Jan 28, 2025

wendwosenbb commented Jan 29, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 4, 2025

philippewarren Feb 5, 2025

philippewarren Feb 5, 2025

		export GAZEBO_MODEL_PATH=/opt/ros/OPENTERA_WS/opentera-webrtc-ros/install/opentera_webrtc_demos/share/opentera_webrtc_demos/models:$GAZEBO_MODEL_PATH
		export GAZEBO_MODEL_PATH=/opt/ros/OPENTERA_WS/models:$GAZEBO_MODEL_PATH

		Dockerfile.opentera_ros2 contains build docker build instructions.

		## Instructions


		COPY opentera_teleop_startup.sh .

		COPY launch .

add container for Opentera webrtc ros #91

Are you sure you want to change the base?

add container for Opentera webrtc ros #91

Conversation

wendwosenbb commented Jan 24, 2025

mamaheux left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philippewarren commented Jan 28, 2025

wendwosenbb commented Jan 29, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment