Updates for kbot walking setup.

[alik-git]

this is an updated setup for kbot walking.

• add a script to convert urdf to usd

updates:

Robot stuff:

• updated usd for updated urdf
• updated joint names for above usd
• no more frozen arms
• updated actuator configs
• updated starting pose

Algorithm Stuff:

• updated entropy coef for the PPO (0.005 -> 0.008)

Termination Stuff:

• this one is still not "stable" we are experimenting with changes
• current setup is episode terminates if anything except the feet detect a force

Randomization Stuff:

• basically no randomization at all

Reward Stuff

• instead of beerkely humanoid definitions, use default isaaclab definitions for following rewards
• feet_air_time_positive_biped, feet_slide, track_lin_vel_xy_yaw_frame_exp, track_ang_vel_z_world_exp
• comment out the beerkley definitions of those rewards in our local config (we might need them later)
• new version of tracking xy vel and z ang vel that accounts for heading
• current reward setup

[INFO] Reward Manager:  <RewardManager> contains 14 active terms.
+----------------------------------------------------+
|                Active Reward Terms                 |
+-------+--------------------------------+-----------+
| Index | Name                           |    Weight |
+-------+--------------------------------+-----------+
|   0   | termination_penalty            |    -200.0 |
|   1   | track_lin_vel_xy_yaw_frame_exp |       1.0 |
|   2   | track_ang_vel_z_world_exp      |       2.0 |
|   3   | lin_vel_z_l2                   |       0.0 |
|   4   | ang_vel_xy_l2                  |     -0.05 |
|   5   | joint_torques_l2               |  -1.5e-07 |
|   6   | dof_acc_l2                     | -1.25e-07 |
|   7   | action_rate_l2                 |    -0.005 |
|   8   | feet_air_time_positive_biped   |      0.25 |
|   9   | feet_slide                     |      -0.1 |
|   10  | dof_pos_limits                 |      -1.0 |
|   11  | joint_deviation_arms           |      -0.1 |
|   12  | joint_deviation_hip            |      -0.1 |
|   13  | flat_orientation_l2            |      -1.0 |
+-------+--------------------------------+-----------+

Observation Stuff

• no more imu
• current obs are as follows

[INFO] Observation Manager: <ObservationManager> contains 2 groups.
+---------------------------------------------------------+
| Active Observation Terms in Group: 'policy' (shape: (66,)) |
+-----------+---------------------------------+-----------+
|   Index   | Name                            |   Shape   |
+-----------+---------------------------------+-----------+
|     0     | velocity_commands               |    (3,)   |
|     1     | projected_gravity               |    (3,)   |
|     2     | joint_pos                       |   (20,)   |
|     3     | joint_vel                       |   (20,)   |
|     4     | actions                         |   (20,)   |
+-----------+---------------------------------+-----------+
+----------------------------------------------------------+
| Active Observation Terms in Group: 'critic' (shape: (259,)) |
+-----------+--------------------------------+-------------+
|   Index   | Name                           |    Shape    |
+-----------+--------------------------------+-------------+
|     0     | base_lin_vel                   |     (3,)    |
|     1     | base_ang_vel                   |     (3,)    |
|     2     | projected_gravity              |     (3,)    |
|     3     | velocity_commands              |     (3,)    |
|     4     | joint_pos                      |    (20,)    |
|     5     | joint_vel                      |    (20,)    |
|     6     | actions                        |    (20,)    |
|     7     | height_scan                    |    (187,)   |
+-----------+--------------------------------+-------------+

[budzianowski]

Unitree's velocity solver has 4 steps fyi.

        articulation_props=sim_utils.ArticulationRootPropertiesCfg(
            enabled_self_collisions=False, solver_position_iteration_count=8, solver_velocity_iteration_count=4
        ),
    ),

[alik-git]

hmm not sure what solver does/is. put a TODO to try 4 steps

[budzianowski]

Increase default ones.

[budzianowski]

remove if unused?

[budzianowski]

Are you adding randomization in the PR?

[alik-git]

No, everything is zeroed out, its a TODO to increase the params to see how robust it can be

[budzianowski]

please make it run through config.

[alik-git]

great feedback thank you, I will get through this after mujoco integration

[WT-MM]

does isaaclab have latency for stuff? experimenting with sensor, motor, and command latency in isaacgym rn

[alik-git]

Not built in, no. I could only find this convo about it

isaac-sim/IsaacLab#431