`make_converter_for_video_caps` builds the chain webrtcsink inserts ahead of an
encoder it selected, special-casing hardware it knows — NVMM, D3D11, CUDA, GL,
VA, and on main also v4l2h264enc — and falling back to software
`videoconvert ! videoscale` for anything else. Rockchip's MPP encoder takes
NV12, I420, YUY2 and more directly and converts on the SoC's 2D accelerator, so
the fallback adds a full CPU pass over every frame to do work the hardware was
going to do anyway, on the four A55s robotd's 50 Hz loop shares.
The reason this matters more than CPU: the robot currently avoids the whole
question by pre-encoding and handing webrtcsink finished H.264. That works, and
it means webrtcsink cannot reach the encoder — so congestion control cannot
adapt the bitrate to the link, and a peer's PLI cannot produce a keyframe, which
leaves a viewer that lost one broken until the next periodic GOP. Letting
webrtcsink own the encoder fixes both. This patch is what makes that affordable.
**This repository is no longer patch-free, and says so.** MPL-2.0 asks that
modifications be identifiable, so the README states it, patches/README.md gives
each patch's reasoning, and build.sh records every applied patch in the release
MANIFEST beside the upstream ref. `git apply --check` runs first, so a patch that
stops applying fails the build naming itself rather than yielding a plugin
quietly missing the change it was carried for.
The trade-off is written down rather than glossed: without videoscale the bin
cannot resize, so the negotiated resolution must be one the source produces.
True on this robot, which pins its caps upstream of the tee — and the reason
upstream may want RGA-backed scaling instead of nothing before taking it. The
v4l2h264enc arm on main is the same shape for another hardware encoder, so the
precedent exists, and if it lands this file is deleted at the next bump.
Verified to apply cleanly against a real 0.15.3 checkout.
Assisted-by: Claude:claude-opus-5[1m] shellcheck
It contains every encoder — strings on its .so lists mpph264enc, mpph265enc,
mppjpegenc and mppvp8enc. The earlier claim here was wrong, and it was the
headline reason given for building our own, so it needs correcting rather than
quietly dropping.
What actually happened is one cause behind four symptoms, and this build
surfaced the fourth: an MPP plugin registers its decoders unconditionally but
*probes MPP* before registering its encoders. With /dev/mpp_service at 0600
root:root the probe fails silently. So mpi_enc_test wrote an empty file and
exited 0; Radxa's deb looked decode-only; a third-party deb installed and still
showed no mpph264enc; and this repository's own CI build lists only mppjpegdec
and mppvideodec, because a container has no such node either. That last one is
expected, not a failure, and the README now says so where somebody reading a
green build log will find it.
The reasons to build that survive: no libx11-6, since rkximage and kmssrc are
disabled; a pin we control, recorded in every release; and this repository has
to exist for gst-plugins-rs regardless, which no archive packages at all — so
the MPP plugin riding along costs almost nothing.
Assisted-by: Claude:claude-opus-5[1m]
Second failure, and a self-inflicted one: `gst-plugin-rsrtp` is not a package
in gst-plugins-rs at 0.15.3 — or at 0.14.5. The crate is `gst-plugin-rtp`,
whose lib is named `gstrsrtp`, so the plugin filename and the package name
differ.
I took `gst-plugin-rsrtp` from reachy-mini-desktop-app README, which documents
a cargo cinstall line that cannot have worked at either tag. Read the crate
Cargo.toml, not a README — the source was one API call away.
The rockchip plugin and gst-plugin-webrtc both built fine, so the pin move in
the previous commit was right; this was the next thing along.
Also validates every crate name with `cargo pkgid` before compiling any of
them. cargo cinstall only checks a package name when it reaches it, so a typo
in the second crate surfaces after the first has spent three minutes building.
Now it costs a second.
Assisted-by: Claude:claude-opus-5[1m] shellcheck
The first build failed, precisely and usefully. Branch HEAD does not compile
against the MPP Radxa ships: 666203a8 ("mppjpegdec: Get the required size from
mpp_sys_cfg", 2026-03-06) started using MppSysCfg and mpp_sys_cfg_*, which do
not exist in librockchip-mpp 1.5.0-1.
gstmppjpegdec.c:219:3: error: unknown type name 'MppSysCfg'
Confined to the JPEG decoder — gstmppenc.c, gstmppjpegenc.c and gstmppvideodec.c
all compiled — so the encoders we actually need were never the problem.
Pinned to 1ceaf42c, its parent. That loses exactly three commits and none of
them matter here: a mppvideodec input-packet-leak fix (a decoder, not the encode
path), an rkximage aspect-ratio fix (a sink this build disables), and the
breaking change.
The alternative was building a newer librockchip-mpp as well and shipping it to
the robot. Rejected: 1.5.0-1 is the version hardware H.264 was proven on —
720p through mpi_enc_test, a bitstream that decodes clean — and replacing it
would trade that proof for a decoder fix we do not use.
Assisted-by: Claude:claude-opus-5[1m]
Two plugins the micro duck robot needs and no archive provides:
`libgstrockchipmpp.so` for hardware H.264 through Rockchip MPP, and
`libgstrswebrtc.so`/`libgstrsrtp.so` for webrtcsink. Debian ships no Rockchip
encoder in any suite, Radxa's prebuilt plugin is decode-only (measured on a
Zero 3W: mppvideodec and mppjpegdec, nothing else — 1.14.4 predates the
encoders), and gstreamer1.0-plugins-rs exists in no Debian suite at all.
`webrtcbin` is deliberately absent: Debian has it.
Native arm64 in a debian:trixie container, which is the point. The daemon
cross-compiles with cargo-zigbuild and its one C dependency is already the
documented cost of that; GStreamer would be a far larger second one, and both
routes — x86 multiarch or a sysroot with a meson cross file — link against an
approximation of the target rather than the target. Here nothing is
approximated.
Public because arm64 runners are free on public repositories, and more
importantly because a release asset is fetched by the updater's preinstall
hook, which runs with a cleared environment and no token. That is the same
arrangement the daemon already relies on for ONNX Runtime.
Pins are commits or tags, never branches, in one file both the build and the
release manifest read — a plugin whose version nobody can name is what makes a
media bug unreproducible, which is exactly what the third-party debs we
rejected could not answer. gst-plugins-rs is pinned at 0.15.3 rather than the
0.14.5 the reachy_mini SDK documents: 0.14.5 is the floor that matters, since
the tags below it miss a webrtcsink deadlock fix between remote description and
ICE handling, and both series declare a GStreamer v1_22 floor against a robot
running 1.26.2.
Two guards written from reading the trees rather than hoping.
`gst/rockchipmpp/meson.build` ends in `if not mpp_dep.found() → subdir_done()`,
so a missing librockchip-mpp-dev makes meson skip the plugin and *succeed*;
pkg-config is checked first and the .so checked for after. And the Radxa debs
are installed as one closure, because dpkg -i resolves nothing for direct
downloads — learning that one package at a time cost three rounds on a board.
Building also drops rkximage and kmssrc, the X11 and KMS sinks in the same
tree, which a headless robot has no use for and which are why the prebuilt deb
depends on libx11-6.
Assisted-by: Claude:claude-opus-5[1m] shellcheck
