Import upstream snapshot c19f713c415a699a79d71cd96aa13c3104a05047

Upstream: https://github.com/michaelgillett/mjlab
Upstream-Commit: c19f713c415a699a79d71cd96aa13c3104a05047
Upstream-Branch: main

docs/source/training/cloud.rst

@@ -0,0 +1,290 @@
+.. _cloud-training:
+
+Cloud Training
+==============
+
+This guide walks through launching training jobs on
+`Lambda Cloud <https://lambdalabs.com/>`_ using
+`SkyPilot <https://skypilot.readthedocs.io/>`_. SkyPilot provisions a GPU
+instance, syncs your code, runs the job, and tears down the machine when
+it finishes.
+
+Two SkyPilot task files live in ``scripts/cloud/``:
+
+.. list-table::
+   :widths: 30 70
+   :header-rows: 1
+
+   * - File
+     - Description
+   * - ``train.yaml``
+     - Installs mjlab directly with uv.
+   * - ``train-docker.yaml``
+     - Pulls the pre-built Docker image from GHCR for a reproducible
+       environment.
+
+
+Prerequisites
+-------------
+
+**1. Install SkyPilot**
+
+SkyPilot is a local CLI tool, not a project dependency. Install it with:
+
+.. code-block:: bash
+
+   uv tool install "skypilot[lambda]"
+
+**2. Lambda Cloud API key**
+
+Generate a key at `Lambda Cloud API keys
+<https://cloud.lambda.ai/api-keys/cloud-api>`_. Name it after your
+machine (e.g. ``kevins-macbook``) so you can tell keys apart later.
+
+.. code-block:: bash
+
+   mkdir -p ~/.lambda_cloud && chmod 700 ~/.lambda_cloud
+   echo "api_key = <your-api-key>" > ~/.lambda_cloud/lambda_keys
+   chmod 600 ~/.lambda_cloud/lambda_keys
+
+**3. Verify setup**
+
+.. code-block:: bash
+
+   sky check lambda
+
+You should see Lambda listed as an enabled cloud.
+
+**4. W&B credentials** *(optional)*
+
+If you log to Weights & Biases, install the ``wandb`` CLI and log in:
+
+.. code-block:: bash
+
+   uv tool install wandb
+   wandb login
+
+This stores your credentials in ``~/.netrc``. The SkyPilot task files
+mount this file onto the remote instance via ``file_mounts`` so that
+``wandb`` authenticates automatically, no environment variable needed.
+
+
+Quick start
+-----------
+
+From the repo root:
+
+.. code-block:: bash
+
+   sky launch scripts/cloud/train.yaml \
+     --env TASK=Mjlab-Velocity-Flat-Unitree-G1
+
+   # Or with Docker:
+   sky launch scripts/cloud/train-docker.yaml \
+     --env TASK=Mjlab-Velocity-Flat-Unitree-G1
+What happens behind the scenes:
+
+1. SkyPilot finds an available Lambda instance with the requested GPU.
+2. It provisions the instance and uploads your local code via rsync.
+3. The ``setup`` step runs (uv install or Docker pull).
+4. The ``run`` step runs (training).
+5. After 5 minutes of idle time the instance is terminated automatically.
+
+.. warning::
+
+   Lambda instances can only be **launched** or **terminated**. There is
+   no pause or suspend. Do not run ``sudo shutdown`` from inside the
+   instance; it will put the machine in an alert state and billing will
+   continue. Always use ``sky down`` to terminate.
+
+
+Common operations
+-----------------
+
+**List available GPUs**
+
+.. code-block:: bash
+
+   sky show-gpus --infra lambda
+
+**Choose a different GPU**
+
+.. code-block:: bash
+
+   sky launch scripts/cloud/train.yaml --gpus H100:1    # 1x H100
+   sky launch scripts/cloud/train.yaml --gpus A100:8    # 8x A100
+   sky launch scripts/cloud/train.yaml --gpus A10:1     # 1x A10 (cheaper)
+
+.. note::
+
+   Both task files pass ``--gpu-ids all``, so multi-GPU instances
+   automatically use :ref:`distributed training <distributed-training>`.
+   When requesting more than one GPU, consider scaling
+   ``MAX_ITERATIONS`` down proportionally. See
+   :ref:`distributed-training` for details on scaling behavior.
+
+**Override training parameters**
+
+Every variable in the YAML ``envs`` block can be overridden from the
+command line with ``--env``:
+
+.. code-block:: bash
+
+   sky launch scripts/cloud/train.yaml \
+     --env TASK=Mjlab-Velocity-Flat-Unitree-Go1 \
+     --env NUM_ENVS=8192 \
+     --env MAX_ITERATIONS=10000
+**Run your own task**
+
+.. code-block:: bash
+
+   sky launch scripts/cloud/train.yaml \
+     --env TASK=Mjlab-Velocity-Flat-Unitree-Go1
+
+To see all registered tasks:
+
+.. code-block:: bash
+
+   uv run list-envs
+   uv run list-envs --keyword Velocity  # filter by keyword
+
+
+Hyperparameter sweeps
+---------------------
+
+Use `W&B Sweeps <https://docs.wandb.ai/models/sweeps/>`_ with SkyPilot
+to search hyperparameters across a multi-GPU instance. The sweep
+controller lives on the W&B servers; each GPU on the instance runs an
+independent sweep agent that pulls a hyperparameter configuration,
+trains, and reports metrics.
+
+The example uses ``method: random``, where each agent samples
+independently. Bayesian search also works well with parallel agents.
+Agents report results back as they finish and the controller updates its
+model between rounds. If using Bayesian, set ``run_cap`` high enough for
+the optimizer to go through several rounds.
+
+Four files are involved:
+
+.. list-table::
+   :widths: 35 65
+   :header-rows: 1
+
+   * - File
+     - Description
+   * - ``sweep.yaml``
+     - W&B sweep configuration (parameters, search method, metric).
+   * - ``sweep-cluster.yaml``
+     - SkyPilot cluster definition (resources, setup, no run section).
+   * - ``sweep-agent.yaml``
+     - SkyPilot job definition that runs ``wandb agent`` on one GPU.
+   * - ``sweep-launch.sh``
+     - Convenience script that creates the sweep, provisions the
+       cluster, and submits one agent per GPU.
+
+**Quick start**
+
+.. code-block:: bash
+
+   ./scripts/cloud/sweep-launch.sh A100:8   # 8 agents on an 8xA100
+
+This creates a W&B sweep, provisions a cluster, and submits one agent
+per GPU. Each agent runs training with a different set of
+hyperparameters sampled by the sweep controller.
+
+**Manual steps** (if you prefer more control):
+
+.. code-block:: bash
+
+   # 1. Create the sweep (returns a SWEEP_ID).
+   wandb sweep scripts/cloud/sweep.yaml
+
+   # 2. Provision the cluster (runs setup, no agents yet).
+   sky launch scripts/cloud/sweep-cluster.yaml \
+     -c mjlab-sweep --gpus A100:8
+
+   # 3. Submit one agent per GPU.
+   sky exec mjlab-sweep scripts/cloud/sweep-agent.yaml \
+     --gpus A100:1 --env SWEEP_ID=<entity/project/sweep_id> -d
+
+Monitor progress on the W&B dashboard or with ``sky queue mjlab-sweep``.
+When done, tear down the cluster with ``sky down mjlab-sweep``.
+
+
+Monitoring
+----------
+
+Provisioning can take five minutes or more while Lambda allocates the
+instance. Open a second terminal to keep an eye on things:
+
+.. code-block:: bash
+
+   sky status                               # cluster state (INIT, UP, ...)
+   sky logs sky-<cluster-name>              # stream logs in real time
+   sky logs sky-<cluster-name> --no-follow  # print current logs and exit
+   sky queue sky-<cluster-name>             # job queue for the cluster
+
+.. tip::
+
+   If the cluster stays in ``INIT`` for a long time, the GPU type is
+   likely sold out. Cancel with ``sky down`` and try a different GPU, or
+   add ``--retry-until-up`` to let SkyPilot keep polling until capacity
+   opens up.
+
+.. code-block:: bash
+
+   sky down sky-<cluster-name>
+   sky launch scripts/cloud/train.yaml --retry-until-up
+
+
+Iterating on a failed job
+-------------------------
+
+When a job fails the cluster keeps running (and billing). You can fix
+the problem locally and resubmit without waiting for a new instance:
+
+.. code-block:: bash
+
+   sky exec sky-<cluster-name> scripts/cloud/train.yaml
+.. important::
+
+   ``sky exec`` rsyncs your latest code and reruns the ``run`` step
+   only. It does **not** rerun ``setup``. If your fix involves
+   dependency changes, use ``sky launch`` again or SSH in and run the
+   setup commands manually.
+
+Other useful commands:
+
+.. code-block:: bash
+
+   sky down sky-<cluster-name>  # terminate the instance immediately
+   ssh sky-<cluster-name>       # SSH in (SkyPilot configures this for you)
+
+
+Cost management
+---------------
+
+.. warning::
+
+   Always run ``sky status`` after each session to confirm nothing is
+   still running. Forgotten instances are the most common source of
+   unexpected charges. To terminate everything at once: ``sky down -a``.
+
+- Instances auto-terminate after 5 minutes of idle time by default.
+  You can change this in the YAML (``idle_minutes``) or at launch time
+  with ``--idle-minutes-to-autostop``.
+- The ``down: true`` setting in the YAML means the instance is fully
+  terminated when it stops, not just paused. Billing stops completely.
+
+
+Troubleshooting
+---------------
+
+**No instances available**
+
+Lambda GPUs sell out frequently. A few things to try:
+
+- Use ``--retry-until-up`` to poll automatically.
+- Try a different GPU type: ``--gpus A100:1``, ``--gpus A10:1``, etc.
+- If you have credentials for other clouds (GCP, AWS), SkyPilot can fall
+  back to them automatically.

docs/source/training/distributed_training.rst

@@ -0,0 +1,109 @@
+.. _distributed-training:
+
+Distributed Training
+====================
+
+mjlab supports multi-GPU distributed training using
+`torchrunx <https://github.com/apoorvkh/torchrunx>`_. Each GPU runs
+independent rollouts with its own environments, and gradients are
+synchronized during policy updates. Throughput scales nearly linearly with
+GPU count.
+
+
+Usage
+-----
+
+.. code-block:: bash
+
+    # Single GPU (default).
+    uv run train <task-name> --gpu-ids "[0]"
+
+    # Two GPUs.
+    uv run train <task-name> --gpu-ids "[0, 1]"
+
+    # All available GPUs.
+    uv run train <task-name> --gpu-ids all
+
+    # CPU mode.
+    uv run train <task-name> --gpu-ids None
+
+Key points:
+
+- GPU indices are relative to ``CUDA_VISIBLE_DEVICES`` if set. For example,
+  ``CUDA_VISIBLE_DEVICES=2,3 uv run train ... --gpu-ids "[0, 1]"`` uses physical
+  GPUs 2 and 3.
+- Single-GPU and CPU modes run directly without torchrunx.
+
+
+Scaling behavior
+----------------
+
+Multi-GPU training is **data-parallel, not work-splitting**. Each GPU runs
+the full ``num-envs`` count independently, so the total experience collected
+per iteration is:
+
+.. code-block:: text
+
+    experience per iteration = num_envs x num_steps_per_env x num_gpus
+
+Iteration speed stays roughly the same because each GPU does the same amount
+of work. The benefit is that each policy update sees more diverse experience,
+so the policy converges faster in wall-clock time.
+
+.. important::
+
+   Because ``max-iterations`` is not automatically adjusted, training with
+   more GPUs runs for proportionally longer. If you want the same total
+   training time, scale ``max-iterations`` down by the number of GPUs
+   (e.g., halve it when doubling from 1 to 2 GPUs).
+
+
+How it works
+------------
+
+mjlab's role is to **isolate MuJoCo Warp simulations on each GPU** using
+``wp.ScopedDevice``. torchrunx handles the rest.
+
+**Process spawning.** ``torchrunx.Launcher`` spawns one process per GPU and
+sets ``RANK``, ``LOCAL_RANK``, and ``WORLD_SIZE`` to coordinate them. Each
+process executes the training function with its assigned GPU.
+
+**Independent rollouts.** Each process maintains its own:
+
+- Environment instances (with ``num-envs`` parallel environments), isolated
+  on its assigned GPU via ``wp.ScopedDevice``
+- Policy network copy
+- Experience buffer (sized ``num_steps_per_env * num_envs``)
+
+Each process uses ``seed = cfg.seed + local_rank`` to ensure different
+random experiences across GPUs, increasing sample diversity.
+
+**Gradient synchronization.** During the update phase, RSL-RL synchronizes
+gradients after each mini-batch through its ``reduce_parameters()`` method:
+
+1. Each process computes gradients independently on its local mini-batch
+2. All policy gradients are flattened into a single tensor
+3. ``torch.distributed.all_reduce`` averages gradients across all GPUs
+4. Averaged gradients are copied back to each parameter, keeping policies
+   synchronized
+
+**Single-writer I/O.** Only rank 0 writes config files, videos, and W&B
+logs to avoid race conditions.
+
+
+Logging
+-------
+
+By default, torchrunx process logs are saved to ``{log_dir}/torchrunx/``.
+This can be customized:
+
+.. code-block:: bash
+
+    # Disable torchrunx file logging.
+    uv run train <task-name> --gpu-ids "[0, 1]" --torchrunx-log-dir ""
+
+    # Custom log directory.
+    uv run train <task-name> --gpu-ids "[0, 1]" --torchrunx-log-dir /path/to/logs
+
+    # Environment variable (takes precedence over the flag).
+    TORCHRUNX_LOG_DIR=/tmp/logs uv run train <task-name> --gpu-ids "[0, 1]"

docs/source/training/motion_imitation.rst

@@ -0,0 +1,65 @@
+.. _motion-imitation:
+
+Motion Imitation
+================
+
+mjlab can train humanoid policies to imitate reference motions. This page
+covers motion data preprocessing and training.
+
+WandB registry setup
+--------------------
+
+mjlab uses `Weights & Biases <https://wandb.ai/>`_ to store and load
+reference motions. Before preprocessing any motions, create a WandB registry
+by following the
+`BeyondMimic instructions <https://github.com/HybridRobotics/whole_body_tracking/blob/main/README.md#motion-preprocessing--registry-setup>`_
+(only the registry creation step; skip the ``csv_to_npz.py`` command shown
+there).
+
+Motion preprocessing
+--------------------
+
+Reference motions are retargeted CSV files in Unitree's generalized
+coordinate convention (base position, base quaternion in xyzw, then joint
+angles).
+
+Convert a CSV to the NPZ format mjlab expects:
+
+.. code-block:: bash
+
+   MUJOCO_GL=egl uv run -m mjlab.scripts.csv_to_npz \
+       --input-file <PATH_TO_CSV> \
+       --output-name <MOTION_NAME> \
+       --input-fps 30 \
+       --output-fps 50 \
+       --render True
+
+The script plays the motion through MuJoCo Warp, computes forward kinematics
+for every body, and uploads the resulting NPZ to your WandB registry.
+
+.. warning::
+
+   You **must** use mjlab's converter (``mjlab.scripts.csv_to_npz``).
+   Converters from other frameworks such as IsaacLab produce NPZ files with
+   incompatible body orderings. The NPZ stores precomputed body positions and
+   quaternions indexed by body number, and different physics engines assign
+   body indices differently (MuJoCo uses depth first traversal, PhysX uses
+   breadth first). A mismatched NPZ will map tracking targets to the wrong
+   bodies and training will not converge.
+
+Training
+--------
+
+.. code-block:: bash
+
+   uv run train Mjlab-Tracking-Flat-Unitree-G1 \
+       --registry-name your-org/motions/motion-name \
+       --env.scene.num-envs 4096
+
+Evaluation
+----------
+
+.. code-block:: bash
+
+   uv run play Mjlab-Tracking-Flat-Unitree-G1 \
+       --wandb-run-path your-org/mjlab/run-id

docs/source/training/rsl_rl.rst

@@ -0,0 +1,277 @@
+.. _rsl_rl:
+
+Training with RSL-RL
+====================
+
+mjlab uses `RSL-RL <https://github.com/leggedrobotics/rsl_rl>`_ for on-policy
+reinforcement learning. The integration has three parts: a **task registry**
+that bundles environment and training configs under a single name, a
+**VecEnv wrapper** that adapts mjlab environments to the interface RSL-RL
+expects, and a set of **configuration dataclasses** that control the training
+run.
+
+
+Task registry
+-------------
+
+Every task in mjlab is a pair: an environment configuration
+(``ManagerBasedRlEnvCfg``) and a training configuration
+(``RslRlOnPolicyRunnerCfg``). The task registry maps a string name to this
+pair so that training can be launched by name from the CLI.
+
+Tasks are registered by calling ``register_mjlab_task`` in the task's
+``__init__.py``:
+
+.. code-block:: python
+
+    from mjlab.tasks.registry import register_mjlab_task
+    from mjlab.tasks.velocity.rl import VelocityOnPolicyRunner
+
+    from .env_cfgs import unitree_g1_rough_env_cfg, unitree_g1_flat_env_cfg
+    from .rl_cfg import unitree_g1_ppo_runner_cfg
+
+    register_mjlab_task(
+      task_id="Mjlab-Velocity-Rough-Unitree-G1",
+      env_cfg=unitree_g1_rough_env_cfg(),
+      play_env_cfg=unitree_g1_rough_env_cfg(play=True),
+      rl_cfg=unitree_g1_ppo_runner_cfg(),
+      runner_cls=VelocityOnPolicyRunner,
+    )
+
+Each registration takes:
+
+- ``task_id``: a unique name following the convention
+  ``Mjlab-{Category}-{Terrain}-{Robot}``
+- ``env_cfg``: the ``ManagerBasedRlEnvCfg`` used for training
+- ``play_env_cfg``: a variant with randomization disabled and episode length
+  set to infinity, used for evaluation
+- ``rl_cfg``: the ``RslRlOnPolicyRunnerCfg`` with PPO hyperparameters and
+  network architecture
+- ``runner_cls``: an optional custom runner class (defaults to
+  ``MjlabOnPolicyRunner``)
+
+All task packages under ``src/mjlab/tasks/`` are auto-discovered at import
+time, so adding a new task only requires creating the config package and
+calling ``register_mjlab_task``.
+
+
+Training and playback
+---------------------
+
+**Launching a training run:**
+
+.. code-block:: bash
+
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 --num-envs 4096
+
+The task name is the first positional argument. The entire configuration
+hierarchy (environment, scene, rewards, PPO hyperparameters, etc.) is
+exposed as CLI flags through `tyro <https://brentyi.github.io/tyro/>`_.
+Every field in ``ManagerBasedRlEnvCfg`` and ``RslRlOnPolicyRunnerCfg`` can
+be overridden from the command line using dot-separated paths:
+
+.. code-block:: bash
+
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 \
+        --num-envs 4096 \
+        --agent.max-iterations 10000 \
+        --agent.algorithm.learning-rate 3e-4 \
+        --env.decimation 2
+
+.. important::
+
+   - **Hyphens, not underscores**: Python field names use underscores
+     (``num_envs``), but CLI flags use POSIX-style hyphens (``--num-envs``).
+   - **Explicit booleans**: boolean flags require an explicit ``True`` or
+     ``False`` value (e.g., ``--agent.resume True``, not ``--agent.resume``).
+     This is intentional for compatibility with W&B sweep configs.
+
+To discover available flags, use ``--help`` and pipe through ``grep``:
+
+.. code-block:: bash
+
+    # See all flags.
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 --help
+
+    # Search for a specific field.
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 --help | grep learning-rate
+
+Some commonly used top-level flags:
+
+``--num-envs``
+    Number of parallel simulation environments.
+
+``--gpu-ids``
+    GPU indices to use. Pass multiple indices for multi-GPU training (see
+    :ref:`distributed-training`), or ``None`` for CPU mode.
+
+``--video``
+    Record training rollout videos to ``{log_dir}/videos/train/``.
+
+``--enable-nan-guard``
+    Enable NaN detection and state capture (see :ref:`nan-guard`).
+
+
+**Playing back a trained policy:**
+
+.. code-block:: bash
+
+    # From W&B.
+    uv run play Mjlab-Velocity-Flat-Unitree-G1 \
+        --wandb-run-path your-entity/mjlab/run-id
+
+    # From a local checkpoint.
+    uv run play Mjlab-Velocity-Flat-Unitree-G1 \
+        --checkpoint-file logs/rsl_rl/g1_velocity/2025-01-27_14-30-00/model_1000.pt
+
+Key ``play`` arguments:
+
+``--agent``
+    Policy mode: ``"trained"`` (default), ``"zero"`` (zero actions), or
+    ``"random"`` (uniform random).
+
+``--viewer``
+    Viewer backend: ``"native"`` (MuJoCo viewer) or ``"viser"``
+    (browser-based).
+
+``--no-terminations``
+    Disable termination conditions so the policy runs indefinitely.
+
+
+VecEnv wrapper
+--------------
+
+``RslRlVecEnvWrapper`` adapts a ``ManagerBasedRlEnv`` to RSL-RL's ``VecEnv``
+interface. It handles three things:
+
+1. **Observation format**: translates observation dictionaries into the
+   ``TensorDict`` format RSL-RL expects.
+2. **Done signal**: merges ``terminated`` and ``truncated`` into a single
+   ``dones`` tensor and passes ``time_outs`` through ``extras`` so RSL-RL can
+   bootstrap correctly on truncated episodes.
+3. **Action clipping**: applies optional action clipping when ``clip_actions``
+   is set in the runner config.
+
+The wrapper also calls ``env.reset()`` during construction because RSL-RL does
+not call reset before beginning rollout collection.
+
+In normal usage you do not interact with the wrapper directly. The training
+script handles wrapping automatically.
+
+
+Configuration
+-------------
+
+``RslRlOnPolicyRunnerCfg`` is the top-level training configuration. It groups
+runner settings, network architecture (``RslRlModelCfg``), and PPO
+hyperparameters (``RslRlPpoAlgorithmCfg``). The following example from the
+Unitree G1 velocity task shows a typical configuration:
+
+.. code-block:: python
+
+    from mjlab.rl import (
+        RslRlModelCfg,
+        RslRlOnPolicyRunnerCfg,
+        RslRlPpoAlgorithmCfg,
+    )
+
+    def unitree_g1_ppo_runner_cfg() -> RslRlOnPolicyRunnerCfg:
+        return RslRlOnPolicyRunnerCfg(
+            actor=RslRlModelCfg(
+                hidden_dims=(512, 256, 128),
+                activation="elu",
+                obs_normalization=True,
+            ),
+            critic=RslRlModelCfg(
+                hidden_dims=(512, 256, 128),
+                activation="elu",
+                obs_normalization=True,
+            ),
+            algorithm=RslRlPpoAlgorithmCfg(
+                value_loss_coef=1.0,
+                use_clipped_value_loss=True,
+                clip_param=0.2,
+                entropy_coef=0.01,
+                num_learning_epochs=5,
+                num_mini_batches=4,
+                learning_rate=1.0e-3,
+                schedule="adaptive",
+                gamma=0.99,
+                lam=0.95,
+                desired_kl=0.01,
+                max_grad_norm=1.0,
+            ),
+            experiment_name="g1_velocity",
+            save_interval=50,
+            num_steps_per_env=24,
+            max_iterations=30_000,
+        )
+
+All fields have sensible defaults and can be overridden from the command line
+(e.g., ``--agent.algorithm.learning-rate 3e-4``). Use ``--help`` to see the
+full list of available fields and their defaults.
+
+
+Checkpoints and logging
+-----------------------
+
+Training artifacts are written to:
+
+.. code-block:: text
+
+    logs/rsl_rl/{experiment_name}/{timestamp}/
+        model_{iteration}.pt      # policy checkpoints
+        params/
+            env.yaml              # full environment config
+            agent.yaml            # full runner config
+
+Checkpoints are saved every ``save_interval`` iterations and uploaded to W&B
+as model artifacts by default. Set ``upload_model=False`` in the runner
+config to disable uploads while keeping metric logging.
+
+.. rubric:: Resuming from a checkpoint
+
+.. code-block:: bash
+
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 \
+        --num-envs 4096 \
+        --agent.resume True
+
+The runner searches for the most recent run directory under
+``logs/rsl_rl/{experiment_name}/`` and loads the highest-numbered checkpoint.
+Narrow the search with ``--agent.load-run`` (regex on directory names) and
+``--agent.load-checkpoint`` (regex on checkpoint filenames).
+
+``--agent.max-iterations`` controls how many *additional* iterations to run
+from the checkpoint. If you are resuming from iteration 11500 with
+``--agent.max-iterations 300`` (the default), training will run iterations
+11500 through 11800. Set this to the number of new iterations you want.
+
+To resume from a W&B run:
+
+.. code-block:: bash
+
+    uv run train Mjlab-Velocity-Flat-Unitree-G1 \
+        --num-envs 4096 \
+        --agent.resume True \
+        --wandb-run-path your-entity/mjlab/run-id
+
+
+Citation
+--------
+
+If you use RSL-RL in your research, consider citing:
+
+.. code-block:: bibtex
+
+    @article{schwarke2025rslrl,
+        title={RSL-RL: A Learning Library for Robotics Research},
+        author={Schwarke, Clemens and Mittal, Mayank and Rudin, Nikita and Hoeller, David and Hutter, Marco},
+        journal={arXiv preprint arXiv:2509.10771},
+        year={2025}
+    }
+
+.. toctree::
+   :maxdepth: 1
+
+   motion_imitation