a9a839f274
webrtcsink's codec discovery builds its encoding chain with no output caps, so `force_profile` is
true and it inserts a capsfilter demanding `profile=constrained-baseline` — WebRTC's interoperable
floor. h264parse strips `alignment`, `stream-format` and `parsed` from a caps query but not
`profile`, so that demand reaches mpph264enc's src pad, whose template listed only
`{ baseline, main, high }`. Empty intersection, GstVideoEncoder's sink getcaps returns nothing,
and the failure surfaces upstream as videorate reporting it "could not transform NV12 ... in
anything we support". Discovery then drops H.264 with a warning nobody was reading, VP8 wins by
default, and the session dies in rtpvp8pay.
The element could always produce constrained-baseline; only its static template denied it.
Measured on an RK3566: `mpph264enc profile=baseline ! h264parse` negotiates
`profile=(string)constrained-baseline` on the parser's src pad, because baseline mode turns CABAC
and 8x8 transform off and MPP emits no FMO, ASO or redundant slices. So this widens the template
by one word rather than claiming something new.
Patches now live in a directory per upstream, applied by an `apply_patches` helper and recorded in
the MANIFEST as `patch <project>/<file>`. Two reasons: a patch aimed at the wrong tree failed the
same way a stale one does, and the MANIFEST's flat `patch <file>` line landed next to whichever
.so happened to precede it, which read as provenance for the wrong plugin.
Assisted-by: Claude:claude-opus-5[1m]
129 lines
6.8 KiB
Markdown
129 lines
6.8 KiB
Markdown
# microduck-gst-plugins
|
|
|
|
Prebuilt aarch64 GStreamer plugins for the micro duck robot, built in CI from pinned upstream
|
|
sources and attached to a release.
|
|
|
|
Two plugins, for two unrelated reasons. Neither is packaged anywhere we can install from.
|
|
|
|
| plugin | provides | why it is here |
|
|
|---|---|---|
|
|
| `libgstrockchipmpp.so` | `mpph264enc`, `mpph265enc`, `mppjpegenc`, `mppvp8enc`, `mppvideodec`, `mppjpegdec` | Debian ships no Rockchip encoder in any suite. Radxa's own `gstreamer1.0-rockchip1_1.14-4` does contain them, so this build is about the pin, dropping `libx11-6`, and riding along with the plugin below — see [below](#the-permission-trap-that-hid-all-of-this). Patched; see [`patches/`](patches/). |
|
|
| `libgstrswebrtc.so`, `libgstrsrtp.so` | `webrtcsink`, `webrtcsrc`, `rsrtp*` | `gstreamer1.0-plugins-rs` does not exist in **any** Debian suite — not trixie, backports, sid or experimental. Patched; see [`patches/`](patches/). |
|
|
|
|
`webrtcbin` is **not** here: it comes from `gstreamer1.0-plugins-bad` in Debian and needs no
|
|
build.
|
|
|
|
## Why a repository of its own
|
|
|
|
The robot's daemon is cross-compiled from a developer's machine with `cargo-zigbuild`, and its one
|
|
C dependency is already the documented cost of doing that. GStreamer would be a much larger second
|
|
one — a cross sysroot or x86 multiarch, either of which links against an approximation of the
|
|
target.
|
|
|
|
So these are built **natively on an arm64 runner, in a `debian:trixie` container**, which is the
|
|
robot's own userland. Nothing is cross-compiled and nothing is approximated. arm64 runners are
|
|
free on public repositories, which is one reason this repository is public.
|
|
|
|
The other reason matters more: a release asset here is fetched by a robot during provisioning and
|
|
by the updater's `preinstall` hook, and that hook runs with a cleared environment and no token. A
|
|
private repository would break it. This is the same arrangement the daemon already relies on for
|
|
ONNX Runtime, which comes from a public `microsoft/onnxruntime` release.
|
|
|
|
Building rather than taking a third-party binary also buys one concrete thing beyond provenance:
|
|
`rkximage` and `kmssrc`, the X11 and KMS *sinks* in the same source tree, are **disabled**. A
|
|
headless robot has no use for either, and they are why the prebuilt Radxa deb depends on
|
|
`libx11-6`.
|
|
|
|
## The permission trap that hid all of this
|
|
|
|
`/dev/mpp_service` arrives as `0600 root:root`, and **an MPP GStreamer plugin registers its
|
|
decoders unconditionally but probes MPP before registering its encoders.** With the node
|
|
unreadable the probe fails and the encoders are silently omitted — no error, no log line.
|
|
|
|
That one cause produced four separate misleading results while this was being worked out:
|
|
|
|
- `mpi_enc_test` wrote an empty file and **exited 0**.
|
|
- Radxa's `1.14-4` looked decode-only. It is not; `strings` on its `.so` lists every encoder.
|
|
- A third-party `1.14-8` deb installed cleanly and still showed no `mpph264enc`.
|
|
- This repository's own CI build shows only `mppjpegdec` and `mppvideodec`, because a container
|
|
has no `/dev/mpp_service` either. **That is expected, not a failed build.**
|
|
|
|
So: a plugin that lists only decoders is evidence about the *device node*, not about the plugin. A
|
|
non-root process needs a udev rule giving the node a group — mode `0660`, group `video` — and only
|
|
then does `gst-inspect-1.0 mpph264enc` mean anything.
|
|
|
|
## Consuming a release
|
|
|
|
```
|
|
tar -xzf microduck-gst-plugins-<version>-aarch64.tar.gz
|
|
```
|
|
|
|
Put the `.so` files anywhere and point `GST_PLUGIN_PATH` at it — `/usr/local/lib/gstreamer-1.0`
|
|
on a robot, which is deliberately **not** the distro's plugin directory, so an `apt` operation can
|
|
never quietly replace or remove them.
|
|
|
|
```
|
|
GST_PLUGIN_PATH=/usr/local/lib/gstreamer-1.0 gst-inspect-1.0 mpph264enc
|
|
```
|
|
|
|
Verify the tarball against the `.sha256` beside it before unpacking. **Pin a version**; do not
|
|
follow "latest". Two provisioning runs a day apart that produce different plugins, with nothing
|
|
recording which, is an unreproducible media bug waiting to happen.
|
|
|
|
### Runtime dependencies
|
|
|
|
The plugins link against libraries a robot needs installed:
|
|
|
|
- `librockchip-mpp1` and `librga2` — from Radxa's pool, at the versions in
|
|
[`pins.env`](pins.env). Not in Debian.
|
|
- `libgstreamer1.0-0`, `libgstreamer-plugins-base1.0-0`, `libglib2.0-0`, `libdrm2` — Debian.
|
|
|
|
`mpph264enc` also needs **read/write access to `/dev/mpp_service`** — see
|
|
[the permission trap](#the-permission-trap-that-hid-all-of-this), which is the single most
|
|
misleading thing about this stack.
|
|
|
|
## Bumping a pin
|
|
|
|
Edit [`pins.env`](pins.env), commit, tag `vN`, push the tag. The release workflow builds and
|
|
attaches the tarball, with the manifest as the release notes so a release always says which
|
|
upstream commits it came from.
|
|
|
|
`workflow_dispatch` builds without cutting a release — worth using, because a workflow that only
|
|
ever runs on a tag is one you discover is broken at the moment you need it.
|
|
|
|
## Licences and source
|
|
|
|
These are binaries built from other people's source, so where that source is matters:
|
|
|
|
- **`gstreamer-rockchip`** is LGPL. Built from
|
|
[`JeffyCN/mirrors`](https://github.com/JeffyCN/mirrors) on the `gstreamer-rockchip` branch, at
|
|
the commit in `pins.env` and recorded in every release's `MANIFEST`.
|
|
`rockchip-linux/gstreamer-rockchip`, which every published deb names as its homepage, is a 404;
|
|
`JeffyCN/mirrors` is the live mirror under the same maintainer.
|
|
- **`gst-plugins-rs`** is MPL-2.0. Built from
|
|
[the upstream repository](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs) at the tag in
|
|
`pins.env`.
|
|
|
|
**Both upstreams are patched, and that matters for more than tidiness.** MPL-2.0 asks that
|
|
modifications be identifiable, so they are stated here, listed in [`patches/`](patches/) with what
|
|
each one buys and how it ends, and recorded in every release's `MANIFEST` — a release names each
|
|
upstream ref *and* every patch applied over it.
|
|
|
|
Two patches today, both about getting hardware H.264 to a browser:
|
|
|
|
- **`gst-plugins-rs`** — `webrtcsink` inserts a software `videoconvert ! videoscale` in front of
|
|
any encoder it does not recognise, and `mpph264enc` converts on the SoC's 2D accelerator
|
|
instead, so a CPU pass over every frame is added to work the hardware was going to do anyway,
|
|
on cores the robot's control loop shares.
|
|
- **`gstreamer-rockchip`** — `mpph264enc`'s pad template omitted `constrained-baseline` from its
|
|
profile list, which is the one profile WebRTC asks for. Without this, `webrtcsink` cannot offer
|
|
H.264 on an RK3566 at all, and the only trace is a `videorate` complaining about NV12.
|
|
|
|
[`patches/README.md`](patches/README.md) has the reasoning for each, the trade-off it accepts, and
|
|
the route upstream that would delete it.
|
|
|
|
Together the ref and the patch list are both the licence answer and the reason a media bug found on
|
|
a robot can be traced to a specific build.
|
|
|
|
This repository's own build scripts are Apache-2.0, matching the daemon.
|