32a241c28f
Some checks failed
nightly / Test against latest dependencies (py3.10) (push) Has been cancelled
nightly / Test against latest dependencies (py3.13) (push) Has been cancelled
tests / tests (3.13, locked) (push) Has been cancelled
tests / tests (3.13, unlocked) (push) Has been cancelled
tests / pyright (3.10) (push) Has been cancelled
tests / lint-format (push) Has been cancelled
tests / tests (3.10, locked) (push) Has been cancelled
tests / tests (3.11, locked) (push) Has been cancelled
tests / tests (3.12, locked) (push) Has been cancelled
tests / pyright (3.11) (push) Has been cancelled
tests / pyright (3.12) (push) Has been cancelled
tests / pyright (3.13) (push) Has been cancelled
tests / ty-check (3.10) (push) Has been cancelled
tests / ty-check (3.11) (push) Has been cancelled
tests / ty-check (3.12) (push) Has been cancelled
tests / ty-check (3.13) (push) Has been cancelled
tests / stubs (push) Has been cancelled
tests / smoke-test (push) Has been cancelled
Docker / check_paths (push) Has been cancelled
docs / build (push) Has been cancelled
Docker / build (push) Has been cancelled
Upstream: https://github.com/michaelgillett/mjlab Upstream-Commit: c19f713c415a699a79d71cd96aa13c3104a05047 Upstream-Branch: main
66 lines
2.0 KiB
ReStructuredText
66 lines
2.0 KiB
ReStructuredText
.. _motion-imitation:
|
|
|
|
Motion Imitation
|
|
================
|
|
|
|
mjlab can train humanoid policies to imitate reference motions. This page
|
|
covers motion data preprocessing and training.
|
|
|
|
WandB registry setup
|
|
--------------------
|
|
|
|
mjlab uses `Weights & Biases <https://wandb.ai/>`_ to store and load
|
|
reference motions. Before preprocessing any motions, create a WandB registry
|
|
by following the
|
|
`BeyondMimic instructions <https://github.com/HybridRobotics/whole_body_tracking/blob/main/README.md#motion-preprocessing--registry-setup>`_
|
|
(only the registry creation step; skip the ``csv_to_npz.py`` command shown
|
|
there).
|
|
|
|
Motion preprocessing
|
|
--------------------
|
|
|
|
Reference motions are retargeted CSV files in Unitree's generalized
|
|
coordinate convention (base position, base quaternion in xyzw, then joint
|
|
angles).
|
|
|
|
Convert a CSV to the NPZ format mjlab expects:
|
|
|
|
.. code-block:: bash
|
|
|
|
MUJOCO_GL=egl uv run -m mjlab.scripts.csv_to_npz \
|
|
--input-file <PATH_TO_CSV> \
|
|
--output-name <MOTION_NAME> \
|
|
--input-fps 30 \
|
|
--output-fps 50 \
|
|
--render True
|
|
|
|
The script plays the motion through MuJoCo Warp, computes forward kinematics
|
|
for every body, and uploads the resulting NPZ to your WandB registry.
|
|
|
|
.. warning::
|
|
|
|
You **must** use mjlab's converter (``mjlab.scripts.csv_to_npz``).
|
|
Converters from other frameworks such as IsaacLab produce NPZ files with
|
|
incompatible body orderings. The NPZ stores precomputed body positions and
|
|
quaternions indexed by body number, and different physics engines assign
|
|
body indices differently (MuJoCo uses depth first traversal, PhysX uses
|
|
breadth first). A mismatched NPZ will map tracking targets to the wrong
|
|
bodies and training will not converge.
|
|
|
|
Training
|
|
--------
|
|
|
|
.. code-block:: bash
|
|
|
|
uv run train Mjlab-Tracking-Flat-Unitree-G1 \
|
|
--registry-name your-org/motions/motion-name \
|
|
--env.scene.num-envs 4096
|
|
|
|
Evaluation
|
|
----------
|
|
|
|
.. code-block:: bash
|
|
|
|
uv run play Mjlab-Tracking-Flat-Unitree-G1 \
|
|
--wandb-run-path your-org/mjlab/run-id
|