microduck/microduck
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Theremin: an explicit mode, and believe the statuses a real hand arrives with

Browse Source 
Two findings from the first run on a duck, one of them the whole cause.

**Status bytes.** ST documents 5 and 9 as "range valid", and that is what the
first version accepted. On this sensor at 15 Hz a hand past ~30 cm comes back as
4 or 13 — consistency failed, sigma too high — carrying a distance that is
perfectly good for a pitch. So the instrument died at exactly 30 cm and chopped
at the boundary where the status flickered. `robotctl monitor` showed the same
thing and nobody could read it: a grid of `x`, "could not measure", on a sensor
that was measuring fine. The believed set is now config, defaults to
[4,5,6,9,10,12,13], and there is a test that says why not to narrow it.

**The clever layer had to go, and antoine called it.** The arming background —
capture what is in front of the duck, play only what is nearer — worked in tests
and could not mean the same thing twice on real frames: which zones carry a
usable status varies frame to frame, so the same gesture armed one moment and was
refused the next. Cleverness on top of a noisy input multiplies the noise. It is
an explicit mode now: while it is up, the nearest return in the band is the hand.
Gone with it: the plane fit, the drift, the refusals, the reprojection and the
floor filter (both were another way for a note to vanish for a reason the player
cannot see).

What replaced it is smaller and about the *sensor* rather than the room: a 250 ms
hold so a flickering zone cannot chop a note, and a status histogram carried in
robot.state and rendered by `robotctl theremin` — "14 usable · 255:38 4*:9 5*:5"
— which is the line that would have found the first bug in a minute.

Also: the mouth no longer waits on `driving`. A duck playing a theremin while
sitting is how anyone tries this first, and gating there made the visible half of
the whole gesture silently absent.

Two test-harness bugs fixed while chasing intermittent failures, both mine: the
fake theremin's reader thread cleared the frame slot a test had just pushed into
(construction and the reader are separate now), and a hold test read the wall
clock instead of driving one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
apirrone 2026-08-24 15:44:29 +02:00
parent d195808ac8
commit 4fd95fb87d
8 changed files with 665 additions and 1244 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
deploy/robotd.toml
View File

@ -280,22 +280,28 @@ mode = "walk"
# tofd's depth stream.
# socket = "/run/tofd/tof.sock"
# The playable band, metres. Closer than near_m the field of view is narrower than a hand
# (it is only 0.828 x distance across) and the sensor's own crosstalk starts inventing short
# returns; past far_m the returns are the noisy ones, and they would be the low notes.
# Shorten the band for a duck on a desk, where the background is 20 cm away.
# near_m = 0.15
# far_m = 0.60
# The playable band, metres. Below near_m the sensor's own cover-glass crosstalk invents
# short returns; past far_m the returns are the noisy ones, and they would be the low notes.
# near_m = 0.10
# far_m = 0.70
# How much nearer than the background a return has to be to count as a hand. The theremin's
# zero is whatever was in front of the duck when it armed — that is what lets it work facing
# a wall — and this margin is what keeps the *trunk's own sway* from crossing it. Raise it on
# a duck whose stand policy sways more than most; lower it for a lighter touch.
# margin_m = 0.08
# Fewest zones that make a hand. Two, because at the far end of the band a hand is a handful
# of zones and only some of them will carry a usable status.
# min_zones = 2
# Fewest zones that make a hand. Two is noise.
# min_zones = 3
# **The field that decides how far the instrument reaches.** ST documents 5 and 9 as "range
# valid", and a build believing only those stops seeing a hand at about 30 cm: past that a
# moving hand comes back as 4 or 13 — consistency failed, sigma too high — carrying a
# distance that is perfectly good for a pitch. 6, 10 and 12 are the other usable-in-practice
# codes (12 is "blurred by a sharp edge", which is what the edge of a hand is). 255 is the
# sensor saying it looked and found nothing, and is never in this list.
#
# Add codes if the reach is short; remove them if it plays phantom notes at empty air.
# `robotctl theremin`'s last column is the per-code count for the live frame, with a * on the
# ones this list believes — that is how you tune it.
# statuses = [4, 5, 6, 9, 10, 12, 13]
# Seconds over which the zero drifts toward a rearranged room. Long on purpose: a hand held
# still for a minute must not become the new background.
# background_tau_s = 120.0
# How long a note is held through a sensor dropout, milliseconds. This is what keeps a zone
# flickering between usable and not from chopping a note into gravel. Too long and a hand you
# withdrew keeps sounding.
# hold_ms = 250

40
docs/robot/cheatsheet.md
View File

@ -232,21 +232,35 @@ robotctl theremin
The head's depth sensor becomes an instrument: a hand in front of the beak is the pitch —
closer is higher — and the mouth opens with the note, wide at the top of the range. Runs
until Ctrl-C and puts the instrument down on the way out. `--off` puts down one that a
client left up.
until Ctrl-C and puts the instrument down on the way out. `--off` puts down one a client left
up.
The first half-second is **arming**, and it is the thing worth understanding: whatever is in
front of the duck right then becomes the silent zero, and only returns nearer than that
play. That is what lets it work with the duck facing a wall 30 cm away — the wall *is* the
zero. It also means arming with your hand already in front of the beak is refused, and says
so, because that hand would have become the zero. `robotctl theremin` prints what the zero
turned out to be (`plane at 0.42 m`, `open space`, `cluttered`), which is the line that
explains a theremin behaving differently in two corners of one room.
An explicit mode with nothing clever inside it: while it is up, the nearest return inside the
playable band is the hand. Point the duck at open space and it is silent; point it at a wall
40 cm away and it plays a steady note. It plays sitting, standing or walking — the mouth is
not part of any policy.
Walking or going over puts the instrument down rather than quietly re-zeroing it: a
background is a picture of where the duck was standing. Needs `tofd` delivering frames and
`[audio]` on — it refuses with a reason otherwise, rather than accepting into silence. The
playable band and the sway margin are `[theremin]` in `robotd.toml`.
The readout's last column is **what the sensor said about that frame**, and it is the answer
to every "why did it stop playing":
```
0.34 m 438.1 Hz 60% ██████ 14 usable · 255:38 4*:9 5*:5 1:12
```
How many zones carry a status the robot believes, then the count per ST status code with a
`*` on the believed ones. A `~` before the note means it is a *held* note bridging a sensor
dropout rather than something measured right now.
That column exists because of the bug it would have found in a minute: ST documents 5 and 9
as "range valid", and a build believing only those **stops seeing a hand at about 30 cm** —
past that a moving hand comes back as 4 or 13 (*consistency failed*, sigma too high) carrying
a distance that is perfectly good for a pitch. If the reach is short, add codes to
`[theremin] statuses` in `robotd.toml`; if it plays phantom notes at empty air, remove some.
`hold_ms` is the anti-chop: it rides over a flickering zone.
Note that `robotctl monitor`'s ToF grid is stricter than the theremin — it marks anything
outside 5/9 as `x`, *could not measure*. A grid full of `x` does not mean the sensor is
broken; it means it is being pessimistic about numbers it does have.
### The ToF sensor (`tofd`)

75
duck-ipc-proto/src/lib.rs
View File

@ -332,13 +332,15 @@ pub mod method {
/// instrument, and the distance of a hand in front of the beak is the pitch — and the
/// mouth opening, which rises with it, so the note is visible as well as audible.
///
/// Discrete; send as a request. The answer only says whether the robot *took* the
/// instrument — has a voice, has depth frames, is not face-down. Arming itself takes
/// about half a second of frames, because the theremin's zero is whatever is in front
/// of the duck at that moment (a wall, a table, an empty room) and one frame is not a
/// background. What that came out as, and whether it was refused because a hand was
/// already in the way, arrives in [`ROBOT_STATE`]'s `theremin` block — which is also
/// where the live pitch is, so a client can show the note being played.
/// Discrete; send as a request, and idempotent both ways. The answer says whether the
/// robot took the instrument — it has a voice, and depth frames are arriving. From then
/// on the nearest return inside the playable band is the hand: an explicit mode with
/// nothing clever inside it, because the clever version could not be relied on to mean
/// the same thing twice on real frames.
///
/// [`ROBOT_STATE`]'s `theremin` block carries the live pitch, the mouth, and a line of
/// what the sensor said about the frame — which is the field diagnostic for this whole
/// feature.
pub const ROBOT_THEREMIN: &str = "robot.theremin";
/// Sit down gracefully, then power the machine off. The prototype's Select long-press.
pub const ROBOT_SHUTDOWN: &str = "robot.shutdown";
@ -1199,11 +1201,10 @@ pub struct ThereminParams {
/// Answer to [`Call::RobotTheremin`].
///
/// `accepted` is about the *instrument*, not about the arming: a robot with a voice, depth
/// frames and its feet under it accepts, and then spends half a second working out what its
/// background is. Watch [`RobotState::theremin`] for how that turned out — including the one
/// refusal a client will actually hit, which is arming with a hand already in front of the
/// beak.
/// Refused only for what the robot can know at the door: no voice, no depth frames, the
/// feature switched off. Once accepted it plays immediately — there is no arming step — and
/// [`RobotState::theremin`] carries what it is doing, including what the sensor is saying
/// about each frame.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default)]
pub struct ThereminResult {
@ -1230,19 +1231,8 @@ impl Default for ThereminResult {
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct ThereminState {
/// The background has been captured and the instrument is playable. False while the
/// arming window is still filling, or after it was refused.
pub armed: bool,
/// Why arming was refused — a hand in the way, or a sensor seeing nothing. Set instead
/// of `armed`, and cleared by asking again.
#[serde(skip_serializing_if = "Option::is_none")]
pub refused: Option<String>,
/// What the background turned out to be, e.g. `plane at 0.42 m (0.9 cm rms)`. The one
/// line that explains a theremin behaving oddly in a particular spot.
#[serde(skip_serializing_if = "Option::is_none")]
pub background: Option<String>,
/// Distance to the hand being played, metres. `None` when no hand is in the band —
/// which is silence, not an error.
/// Distance to the hand being played, metres. `None` when nothing is in the playable
/// band — which is silence, not an error.
#[serde(skip_serializing_if = "Option::is_none")]
pub hand_range_m: Option<f64>,
/// The note being sounded, hertz. `None` when silent.
@ -1250,6 +1240,24 @@ pub struct ThereminState {
pub note_hz: Option<f64>,
/// How far open the mouth is being driven, 0..1 — the same number the pitch came from.
pub mouth: f64,
/// How many zones the hand covers. Zero while silent. The number that says whether a
/// dropout was the hand leaving or the sensor blinking.
pub zones: u32,
/// This note is the held memory of a frame just gone, bridging a sensor dropout, rather
/// than something measured now. Reported so a readout can show that rather than implying
/// it still sees a hand.
#[serde(default, skip_serializing_if = "not")]
pub held: bool,
/// What the sensor said about this frame, as a line: how many zones carry a status the
/// robot believes, then the count per status code with a `*` on the believed ones — e.g.
/// `12 usable · 255:40 4*:12 5*:8 1:4`.
///
/// Diagnostic, and the one worth carrying on the wire: a theremin that stops working past
/// 30 cm and a frame where status 4 covers half the grid are the same fact, but only the
/// second says what to change. The first version of this feature accepted only ST's two
/// "valid" codes and died at exactly that distance, invisibly, for want of this line.
#[serde(skip_serializing_if = "Option::is_none")]
pub sensor: Option<String>,
}
/// The one-shot skills, plus the sit↔stand toggle. See [`method::ROBOT_DO`].
@ -3481,22 +3489,19 @@ mod tests {
assert!(!down.contains("theremin"), "{down}");
state.theremin = Some(ThereminState {
armed: true,
refused: None,
background: Some("plane at 0.42 m (0.9 cm rms)".into()),
hand_range_m: Some(0.31),
note_hz: Some(412.0),
mouth: 0.64,
zones: 9,
held: false,
sensor: Some("12 usable · 255:40 4*:12 5*:8 1:4".into()),
});
let up = serde_json::to_string(&state).unwrap();
assert!(up.contains(r#""theremin":{"armed":true"#), "{up}");
assert!(up.contains(r#""theremin":{"hand_range_m":0.31"#), "{up}");
assert!(up.contains(r#""note_hz":412.0"#), "{up}");
// A silent armed theremin omits the note rather than sending a zero, which would
// read as "playing 0 Hz".
state.theremin = Some(ThereminState {
armed: true,
..Default::default()
});
// A silent theremin omits the note rather than sending a zero, which would read as
// "playing 0 Hz".
state.theremin = Some(ThereminState::default());
let silent = serde_json::to_string(&state).unwrap();
assert!(!silent.contains("note_hz"), "{silent}");
assert!(!silent.contains("hand_range_m"), "{silent}");

938
kinematics/src/hand.rs
View File

File diff suppressed because it is too large Load Diff

58
robotctl/src/main.rs
View File

@ -139,11 +139,13 @@ enum Namespace {
/// beak is the pitch — closer is higher, and the mouth opens with the note.
///
/// Runs until Ctrl-C, printing what the instrument is doing, and puts it down on the way
/// out. What the first half-second is doing is *arming*: whatever is in front of the duck
/// right then — a wall, a table, an empty room — becomes the silent zero, and only things
/// nearer than that play. So a duck facing a wall is fine; a duck arming with your hand
/// already in front of it refuses and says so, because that hand would have become the
/// zero.
/// out. An explicit mode with nothing clever inside it: while it is up, the nearest thing
/// in the playable band is the hand. Point the duck at open space and it is silent; point
/// it at a wall 40 cm away and it plays a steady note.
///
/// The readout's last column is what the *sensor* said about the frame — how many zones
/// carry a status the robot believes, then the count per status code. That line is the
/// answer to every "why did it stop playing".
Theremin {
/// Put the instrument down instead of picking it up. For a theremin left up by a
/// client that went away.
@ -402,7 +404,6 @@ fn run_quack(socket: &Path) -> Result<(), Failure> {
Ok(())
}
/// Set by the `SIGINT` handler so the theremin is put down on the way out rather than left
/// sounding. A bare `AtomicBool` store is the only thing a signal handler may safely do.
static INTERRUPTED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
@ -413,19 +414,19 @@ extern "C" fn note_interrupt(_signal: libc::c_int) {
/// `robotctl theremin` — pick the ToF theremin up, watch it, put it down.
///
/// A live view rather than a one-shot, because every question anyone has about this feature
/// is a question about what it is doing *now*: has it armed, what did it decide its
/// background was, is it seeing my hand, what note is that. All four are in
/// `robot.state`'s theremin block, so this is a subscription with a one-line renderer over
/// it.
/// A live view rather than a one-shot, because every question anyone has about this feature is
/// a question about what it is doing *now*: is it seeing my hand, what note is that, and —
/// the one that matters when it is not working — what is the sensor actually reporting. All
/// three are in `robot.state`'s theremin block, so this is a subscription with a one-line
/// renderer over it.
fn run_theremin(socket: &Path, off: bool) -> Result<(), Failure> {
let mut client = Client::connect_to("robotd", socket)?;
client.hello()?;
let ask = |client: &mut Client, active: bool| -> Result<proto::ThereminResult, Failure> {
let result = result_of(client.call(&proto::Call::RobotTheremin(proto::ThereminParams {
active,
}))?)?;
let result = result_of(client.call(&proto::Call::RobotTheremin(
proto::ThereminParams { active },
))?)?;
decode(&result)
};
@ -457,7 +458,7 @@ fn run_theremin(socket: &Path, off: bool) -> Result<(), Failure> {
);
}
println!("arming — hold still, whatever is in front of the beak becomes the zero");
println!("playing — a hand in front of the beak, closer is higher · Ctrl-C to stop");
// A second connection for the state stream: the first one is kept to put the instrument
// down with, and a subscription is a stream of notifications rather than a call/response,
// so sharing one would mean untangling the two.
@ -468,7 +469,6 @@ fn run_theremin(socket: &Path, off: bool) -> Result<(), Failure> {
&proto::Call::RobotSubscribe(proto::SubscribeParams { hz: Some(15) }),
))?;
let mut armed = false;
let mut line = String::new();
while !INTERRUPTED.load(std::sync::atomic::Ordering::Relaxed) {
line.clear();
@ -490,32 +490,22 @@ fn run_theremin(socket: &Path, off: bool) -> Result<(), Failure> {
continue;
};
// The block is absent while the instrument is down, which after a successful pick-up
// means the robot put it down itself — it started walking, or went over.
// means the robot put it down itself.
let Some(theremin) = state.theremin else {
println!("\rthe robot put the theremin down ");
println!("\rthe robot put the theremin down ");
return Ok(());
};
if let Some(refused) = theremin.refused {
println!("\r ");
return Err(Failure::new(exit::REFUSED, refused));
}
if theremin.armed && !armed {
armed = true;
let background = theremin.background.as_deref().unwrap_or("unknown");
println!("armed against {background} — play it, Ctrl-C to stop");
}
if !armed {
continue;
}
// One rewritten line: this is a live readout, and a scrolling one would be unreadable
// at 15 Hz.
let sensor = theremin.sensor.as_deref().unwrap_or("");
match (theremin.hand_range_m, theremin.note_hz) {
(Some(range), Some(hz)) => print!(
"\r {range:.2} m {hz:6.1} Hz mouth {:>3.0}% {:<20}",
"\r {range:.2} m {:1} {hz:6.1} Hz {:>3.0}% {:<10} {sensor:<34}",
if theremin.held { "~" } else { " " },
theremin.mouth * 100.0,
bar(theremin.mouth),
),
_ => print!("\r {:<58}", "— no hand —"),
_ => print!("\r {:<32}{sensor:<34}", "— no hand —"),
}
let _ = std::io::stdout().flush();
}
@ -528,10 +518,10 @@ fn run_theremin(socket: &Path, off: bool) -> Result<(), Failure> {
Ok(())
}
/// A twenty-cell meter for the mouth opening — the one part of the readout you can watch
/// A ten-cell meter for the mouth opening — the one part of the readout you can watch
/// without reading it.
fn bar(fraction: f64) -> String {
let filled = (fraction.clamp(0.0, 1.0) * 20.0).round() as usize;
let filled = (fraction.clamp(0.0, 1.0) * 10.0).round() as usize;
"█".repeat(filled)
}

39
robotd/src/main.rs
View File

@ -1205,13 +1205,8 @@ async fn control_loop<T: RobotIo>(
// frames — a second of silence that reads as a broken feature. Off entirely when the
// params say so, or when audio is off, since a theremin with no voice is a mouth
// opening for no reason.
let mut theremin = (params.theremin.enabled && params.audio.enabled).then(|| {
theremin::Theremin::spawn(
params.theremin.socket.clone(),
params.theremin.hand(),
kinematics::tof::Reprojector::alpha(),
)
});
let mut theremin = (params.theremin.enabled && params.audio.enabled)
.then(|| theremin::Theremin::spawn(params.theremin.socket.clone(), params.theremin.hand()));
// The note the theremin is holding, kept across ticks so a hand leaving the frame fades
// the note at its own pitch instead of gliding to the bottom of the range on its way out.
let mut theremin_hz = 0.0f64;
@ -1827,25 +1822,10 @@ async fn control_loop<T: RobotIo>(
if let Some(active) = intents.take_theremin_request() {
instrument.set_active(active);
}
// A background is a picture of what is in front of the duck, so a duck that has
// started moving is holding an instrument tuned to somewhere it no longer is.
// Dropped rather than silently re-armed: a zero that changes under the player is
// worse than an instrument that has to be picked up again.
if instrument.active() {
if safety.fallen() {
instrument.invalidate("the robot is down");
} else if moving {
instrument.invalidate("walking");
}
}
let note = instrument.tick(
command.head,
&kinematics::tof::Posture {
gravity: sensors.as_ref().map_or([0.0, 0.0, -1.0], |s| s.imu.gravity),
trunk_height_m: Some(odometry.position()[2]),
},
period.as_secs_f64(),
);
// Nothing takes the instrument away but asking. Walking used to drop it, because
// a captured background is a picture of one spot — with no background there is
// nothing to invalidate, and a duck that plays while it walks is a feature.
let note = instrument.tick(tick_start);
match note {
Some(note) => {
let mut block = note.state;
@ -1872,7 +1852,12 @@ async fn control_loop<T: RobotIo>(
voice.theremin_set(theremin_hz, level, note.mouth);
}
}
if driving {
// Whatever the policy is doing, unlike the mouth *intent* below. The
// mouth is not part of any policy, and a duck playing a theremin while
// sitting — which is how anyone will first try this — has to be able to
// open its beak: gating on `driving` made the visible half of the whole
// gesture silently absent on a sitting robot.
if snapshot.enabled && bringup == Bringup::Ready {
targets[duck_control::model::MOUTH_INDEX] =
duck_control::model::mouth_target(note.mouth);
}

39
robotd/src/params.rs
View File

@ -33,12 +33,12 @@ pub struct Params {
/// `[theremin]` — the ToF theremin: what counts as a hand, and where the depth frames come
/// from.
///
/// Every field here is a *number the field will want to argue with*, which is why they are
/// config and not constants: how near you have to get before it plays, how far away the note
/// bottoms out, and how much nearer than the background a hand has to be before it counts.
/// The defaults are `kinematics::hand::Config`'s, and the reason to move them is a room, not
/// a robot — a duck on a desk has a background 20 cm away and wants a shorter band than one
/// on a floor.
/// The interesting field is `statuses`, and it is the reason this section exists at all. ST
/// documents 5 and 9 as "range valid", and a build that believes only those stops seeing a
/// hand at about 30 cm on this sensor — past that a moving hand comes back as 4 or 13,
/// *consistency failed*, carrying a distance that is fine for a pitch. That took a bench
/// session to find, so the set is configurable: a duck whose theremin has a short reach wants
/// more codes in, and one that plays phantom notes at nothing wants fewer.
#[derive(Debug, Clone, Deserialize)]
#[serde(deny_unknown_fields, default)]
pub struct ThereminParams {
@ -52,13 +52,14 @@ pub struct ThereminParams {
pub near_m: f64,
/// Farthest playable range, metres.
pub far_m: f64,
/// How much nearer than the background a return must be to count as a hand, metres.
/// Raise it on a duck whose stand policy sways more than most.
pub margin_m: f64,
/// Fewest zones that make a hand.
pub min_zones: usize,
/// Seconds over which the background drifts toward a rearranged room.
pub background_tau_s: f64,
/// ST status bytes whose distance is believed. See the section docs — this is the one
/// that decides how far the instrument reaches.
pub statuses: Vec<u8>,
/// How long a note is held through a sensor dropout, milliseconds. This is what keeps a
/// flickering zone from chopping a note into gravel.
pub hold_ms: u64,
}
impl Default for ThereminParams {
@ -69,34 +70,26 @@ impl Default for ThereminParams {
socket: PathBuf::from(duck_ipc_proto::socket::TOF),
near_m: hand.near_m,
far_m: hand.far_m,
margin_m: hand.margin_m,
min_zones: hand.min_zones,
background_tau_s: hand.background_tau_s,
statuses: hand.statuses,
hold_ms: hand.hold.as_millis() as u64,
}
}
}
impl ThereminParams {
/// The hand-detection config these params describe.
///
/// The fields not exposed above keep their library defaults on purpose: `max_fill` and
/// `wall_fill` are geometry (a wall fills the field of view, a hand does not), not taste,
/// and an operator who moved them would be describing a different sensor.
pub fn hand(&self) -> kinematics::hand::Config {
kinematics::hand::Config {
near_m: self.near_m,
far_m: self.far_m,
margin_m: self.margin_m,
min_zones: self.min_zones,
background_tau_s: self.background_tau_s,
..kinematics::hand::Config::default()
statuses: self.statuses.clone(),
hold: std::time::Duration::from_millis(self.hold_ms),
}
}
}
/// `[audio]` — the voice and the microphone. All optional equipment: a robot without a
/// codec (or a bank) walks identically and stays quiet, so nothing here reaches a health
/// verdict.
#[derive(Debug, Clone, Deserialize)]
#[serde(deny_unknown_fields, default)]
pub struct AudioParams {

682
robotd/src/theremin.rs
View File

@ -10,22 +10,24 @@
//!
//! **One gesture, three outputs.** Closeness (0 at the far end of the playable band, 1 at
//! the near end, from [`kinematics::hand`]) drives the pitch, the level, *and* how far the
//! mouth opens. Not three tunings of the same thing but literally one number, because a
//! duck whose mouth opens on a different curve from its pitch reads as a mouth animation
//! playing over a sound rather than as an animal making one.
//! mouth opens. Not three tunings of the same thing but literally one number, because a duck
//! whose mouth opens on a different curve from its pitch reads as a mouth animation playing
//! over a sound rather than as an animal making one.
//!
//! **Where the hand-versus-wall problem is solved: not here.** [`kinematics::hand`] owns
//! that, and its answer is a background captured when the theremin arms — see its module
//! docs. What this module owns is the *sequencing* that answer needs: an arming window that
//! spans several frames, a refusal that has to survive until a client reads it, and the fact
//! that a walking duck invalidates its own background and so is not allowed to hold the
//! instrument.
//! **An explicit mode, and nothing clever inside it.** The first version armed: it captured
//! what was in front of the duck as a background so it could tell a hand from a wall without
//! being told. On a bench that worked; on a duck the same gesture armed one moment and was
//! refused the next, because *which zones carry a usable status varies frame to frame* and a
//! background is only as stable as the frames it was averaged from. So the mode is now
//! something you turn on, and while it is on the nearest return in the band is the hand —
//! see [`kinematics::hand`] for what is left and why. The state machine here is two states,
//! and the only judgement it makes is about the *sensor*: is a frame recent enough to play.
//!
//! **Rate mismatch is a fade, not a gate.** A depth frame that stops arriving — `tofd`
//! restarted, the sensor dropped off the bus — must not leave a note sounding forever, and
//! must not chop one off either. A frame older than [`FRAME_STALE`] takes the level to zero
//! and leaves everything else alone, so the instrument goes quiet and comes back when the
//! frames do.
//! frames do. Short dropouts never reach here at all: `hand::Tracker` bridges those.
use std::io::{BufRead, BufReader, Write};
use std::os::unix::net::UnixStream;
@ -35,56 +37,38 @@ use std::time::{Duration, Instant};
use arc_swap::ArcSwapOption;
use duck_ipc_proto as proto;
use kinematics::hand::{self, Arming, Background, Hand, Refusal};
use kinematics::tof::{COLS, Posture, ROWS, Reprojector};
use kinematics::hand::{self, Tracker};
const N_ZONES: usize = ROWS * COLS;
/// How long a depth frame stays playable. Three frames at 15 Hz: long enough to ride out a
/// dropped one, short enough that a dead sensor falls silent rather than holding a note.
const FRAME_STALE: Duration = Duration::from_millis(200);
/// How long a depth frame stays playable.
///
/// Longer than the tracker's own hold, and doing a different job: the tracker bridges a
/// sensor that fumbled a zone, this notices a sensor that has stopped talking at all.
const FRAME_STALE: Duration = Duration::from_millis(500);
/// How fast the reader thread retries a depth socket that is not there. `tofd` may be
/// restarting, or may not be running at all on a duck without the sensor — either way this
/// is a background thread and its failure is a log line, not an error anybody waits on.
/// restarting, or may not be running at all on a duck without the sensor — either way this is
/// a background thread and its failure is a log line, not an error anybody waits on.
const RECONNECT: Duration = Duration::from_secs(2);
/// Request id for the subscription. Any number; `tofd` echoes it.
const SUBSCRIBE_ID: u64 = 1;
/// One depth frame as the loop consumes it: slant ranges in metres, `None` where the sensor
/// had nothing usable to say.
/// One depth frame, as the wire sent it.
///
/// Interpreted in the reader thread rather than the loop — the status-byte rules are the
/// `tof` crate's (`Frame::zone`), transcribed here because `robotd` must not link that crate
/// for its vendored C driver, and doing it once per frame off the loop is free.
/// Kept raw — distances and status bytes, uninterpreted — because which statuses count is
/// [`hand::Config`]'s decision and it is the decision this feature turned out to hinge on.
/// Interpreting here would have buried it.
struct Frame {
ranges_m: [Option<f64>; N_ZONES],
distance_mm: Vec<i16>,
status: Vec<u8>,
at: Instant,
}
/// What the instrument is doing.
enum State {
/// Put down. Nothing is read, nothing sounds.
Down,
/// Collecting frames for a background. Half a second, at 15 Hz.
Arming(Arming),
/// Playable. The background is the zero every frame is measured against. Boxed because
/// it is by far the largest thing a `State` can be — 64 ranges and a shape — and an
/// instrument that is down should not carry the size of one that is up.
Armed(Box<Background>),
/// Arming came back with a reason, which is kept until the instrument is picked up
/// again — a refusal that vanished before the client's next state frame would present
/// as a theremin that silently does nothing.
Refused(Refusal),
}
/// What one tick of the theremin produced.
///
/// Deliberately *not* a frequency: mapping closeness to a note needs this duck's register,
/// which lives with its voice (`crate::sound::Sound::theremin_hz_at`) and not with its
/// depth sensor. This module says how close the hand is; the voice says what that sounds
/// like.
/// which lives with its voice (`crate::sound::Sound::theremin_hz_at`) and not with its depth
/// sensor. This module says how close the hand is; the voice says what that sounds like.
#[derive(Debug, Clone, PartialEq)]
pub struct Note {
/// Mouth opening to drive, 0..1. Always present while the instrument is up, because a
@ -93,52 +77,55 @@ pub struct Note {
/// The state block for `robot.state`. `note_hz` is left for the caller to fill, for the
/// reason above.
pub state: proto::ThereminState,
/// Where the hand is in the playable band, 0 far to 1 near. `None` is silence — no hand
/// in the band, still arming, refused, or a sensor that stopped delivering.
/// Where the hand is in the playable band, 0 far to 1 near. `None` is silence.
pub closeness: Option<f64>,
}
pub struct Theremin {
/// Newest frame from the reader thread. `ArcSwapOption` for the reason every other
/// intent slot is one: the loop does an atomic load and can never be held up by the
/// thread writing the other side.
/// Newest frame from the reader thread. `ArcSwapOption` for the reason every other intent
/// slot is one: the loop does an atomic load and can never be held up by the thread
/// writing the other side.
latest: Arc<ArcSwapOption<Frame>>,
reprojector: Reprojector,
config: hand::Config,
state: State,
/// The frame the arming window last took, so a stalled sensor cannot fill a window with
/// sixty copies of one frame and call it a background.
last_armed_at: Option<Instant>,
tracker: Tracker,
/// Up or down. The whole state machine — see the module docs for what used to be here.
active: bool,
}
impl Theremin {
/// Start the depth reader and hold an instrument that is down.
///
/// The reader runs whether or not a theremin is ever asked for: it is one parked read on
/// a socket, and starting it lazily would mean the first arming window waited for a
/// connection as well as for frames — a second of nothing, which reads as a broken
/// feature.
pub fn spawn(socket: PathBuf, config: hand::Config, reprojector: Reprojector) -> Self {
let latest = Arc::new(ArcSwapOption::empty());
let slot = latest.clone();
/// The reader runs whether or not a theremin is ever asked for: it is one parked read on a
/// socket, and connecting lazily would make picking the instrument up wait for a
/// connection as well as for frames.
pub fn spawn(socket: PathBuf, config: hand::Config) -> Self {
let theremin = Self::new(config);
let slot = theremin.latest.clone();
let spawned = std::thread::Builder::new()
.name("tof-reader".into())
.spawn(move || read_frames(&socket, &slot));
if spawned.is_err() {
tracing::warn!("cannot spawn the depth reader; no theremin");
}
theremin
}
/// An instrument with no depth reader behind it — frames are whatever is put in the slot.
///
/// Exists for the tests, and it is not a convenience: a reader pointed at a socket nobody
/// answers on clears the slot the moment its connect fails, which under a parallel suite
/// wiped the frame a test had just pushed. A construction that starts no thread is the
/// difference between a deterministic test and an intermittent one.
fn new(config: hand::Config) -> Self {
Self {
latest,
reprojector,
config,
state: State::Down,
last_armed_at: None,
latest: Arc::new(ArcSwapOption::empty()),
tracker: Tracker::new(config),
active: false,
}
}
/// True while the instrument is up — arming, armed, or refused.
pub fn active(&self) -> bool {
!matches!(self.state, State::Down)
#[cfg(test)]
fn active(&self) -> bool {
self.active
}
/// Whether depth frames are arriving at all. What a refusal at the door is made of:
@ -151,38 +138,20 @@ impl Theremin {
.is_some_and(|frame| frame.at.elapsed() < FRAME_STALE)
}
/// Pick the instrument up, or put it down. Idempotent: picking up an instrument already
/// in hand does *not* restart the arming window, because that would drop the background
/// a player is in the middle of using.
/// Pick the instrument up, or put it down.
pub fn set_active(&mut self, active: bool) {
match (active, &self.state) {
(true, State::Down | State::Refused(_)) => {
self.state = State::Arming(Arming::new());
self.last_armed_at = None;
}
(true, _) => {}
(false, _) => self.state = State::Down,
if active != self.active {
tracing::warn!(active, "theremin");
}
self.active = active;
// Held notes do not survive being put down: picking the instrument back up must not
// open with a note from the last time.
self.tracker.reset();
}
/// Put the instrument down because the robot is doing something incompatible with
/// holding it — walking, or on its side.
///
/// A background is a picture of what is in front of the duck, so a duck that moves has a
/// background of somewhere it no longer is. Re-arming automatically when it stops would
/// be worse than dropping it: the player would get an instrument whose zero silently
/// changed under them.
pub fn invalidate(&mut self, why: &str) {
if self.active() {
tracing::warn!(why, "theremin: put down");
self.state = State::Down;
}
}
/// One tick. Never blocks, never allocates a frame, and returns `None` when there is
/// nothing for the loop to do.
pub fn tick(&mut self, head_joints: [f64; 4], posture: &Posture, dt_s: f64) -> Option<Note> {
if matches!(self.state, State::Down) {
/// One tick. Never blocks, and returns `None` when there is nothing for the loop to do.
pub fn tick(&mut self, now: Instant) -> Option<Note> {
if !self.active {
return None;
}
let frame = self.latest.load_full();
@ -190,108 +159,61 @@ impl Theremin {
.as_ref()
.filter(|frame| frame.at.elapsed() < FRAME_STALE);
// A stale sensor is silence with everything else held: the note fades, the
// background is kept, and the instrument plays again the moment frames return.
// A silent sensor is silence with the instrument still in hand: it plays again the
// moment frames return, rather than needing to be picked up afresh.
let Some(frame) = fresh else {
self.tracker.reset();
return Some(Note {
mouth: 0.0,
state: self.state_block(None, None),
state: proto::ThereminState {
sensor: Some("no depth frames".to_owned()),
..Default::default()
},
closeness: None,
});
};
// The same frame twice is not two observations. Without this an arming window could
// fill from one frozen frame, and the background would be a picture of a stall.
let repeated = self.last_armed_at == Some(frame.at);
match &mut self.state {
State::Down => None,
State::Arming(arming) => {
if !repeated {
arming.observe(&frame.ranges_m);
self.last_armed_at = Some(frame.at);
}
if arming.ready(&self.config) {
let outcome = arming.finish(&self.config, self.reprojector.beams());
self.state = match outcome {
Ok(background) => {
tracing::warn!(
background = %describe(&background.shape),
"theremin: armed"
);
State::Armed(Box::new(background))
}
Err(refusal) => {
tracing::warn!(reason = refusal.as_str(), "theremin: refused to arm");
State::Refused(refusal)
}
};
}
// Silent while arming — and the mouth closed, which is what makes the
// half-second read as the duck getting ready rather than as a lag.
Some(Note {
mouth: 0.0,
state: self.state_block(None, None),
closeness: None,
})
}
State::Refused(_) => Some(Note {
mouth: 0.0,
state: self.state_block(None, None),
closeness: None,
}),
State::Armed(background) => {
let zones = self
.reprojector
.project(&frame.ranges_m, head_joints, posture);
let hand = background.detect(&frame.ranges_m, &zones, &self.config);
if !repeated {
background.relax(&frame.ranges_m, hand.as_ref(), &self.config, dt_s);
self.last_armed_at = Some(frame.at);
}
let closeness = hand.as_ref().map(|h| h.closeness);
Some(Note {
// A silent theremin closes the beak; a played one opens it as far as the
// note is high, which is the same number.
mouth: closeness.unwrap_or(0.0),
state: self.state_block(hand.as_ref(), closeness),
closeness,
})
}
}
}
/// The block clients watch. Assembled here rather than at the call site because half of
/// it is this module's private state.
fn state_block(&self, hand: Option<&Hand>, closeness: Option<f64>) -> proto::ThereminState {
proto::ThereminState {
armed: matches!(self.state, State::Armed(_)),
refused: match &self.state {
State::Refused(refusal) => Some(refusal.as_str().to_owned()),
_ => None,
},
background: match &self.state {
State::Armed(background) => Some(describe(&background.shape)),
_ => None,
},
hand_range_m: hand.map(|h| h.range_m),
// Filled in by the caller, which is the only one that knows this duck's
// register — see `Sound::theremin_hz_at`.
note_hz: None,
let hand = self.tracker.track(&frame.distance_mm, &frame.status, now);
let closeness = hand.map(|h| h.closeness);
Some(Note {
// A silent theremin closes the beak; a played one opens it as far as the note is
// high, which is the same number.
mouth: closeness.unwrap_or(0.0),
}
state: proto::ThereminState {
hand_range_m: hand.map(|h| h.range_m),
zones: hand.map_or(0, |h| h.zones) as u32,
held: hand.is_some_and(|h| h.held),
// The one diagnostic that would have found the first version's bug in a
// minute instead of a session: what the sensor is actually saying about each
// zone, rather than what this build makes of it.
sensor: Some(describe(&frame.status, self.tracker.config())),
note_hz: None,
mouth: closeness.unwrap_or(0.0),
},
closeness,
})
}
}
/// One line saying what the theremin is measuring against — the thing that explains a
/// theremin behaving differently in two corners of the same room.
fn describe(shape: &hand::Shape) -> String {
match shape {
hand::Shape::Open { zones } => format!("open space ({zones} zones)"),
hand::Shape::Plane { distance_m, rms_m } => {
format!("plane at {distance_m:.2} m ({:.1} cm rms)", rms_m * 100.0)
}
hand::Shape::Cluttered { rms_m } => format!("cluttered ({:.1} cm rms)", rms_m * 100.0),
}
/// The frame's status bytes as a line: how many zones each code covers, and how many of those
/// this build believes.
fn describe(status: &[u8], config: &hand::Config) -> String {
let histogram = hand::status_histogram(status);
let believed: usize = histogram
.iter()
.filter(|(code, _)| config.believes(*code))
.map(|(_, count)| count)
.sum();
let codes: Vec<String> = histogram
.iter()
.map(|(code, count)| {
// A marker on the codes this build acts on, so "the sensor is talking but we are
// ignoring it" is visible at a glance rather than needing the config to hand.
let mark = if config.believes(*code) { "*" } else { "" };
format!("{code}{mark}:{count}")
})
.collect();
format!("{believed} usable · {}", codes.join(" "))
}
/// Park on `tofd`'s depth stream, forever, leaving the newest frame in `slot`.
@ -348,138 +270,84 @@ fn stream_frames(socket: &Path, slot: &ArcSwapOption<Frame>) -> Result<(), Strin
continue;
};
slot.store(Some(Arc::new(Frame {
ranges_m: interpret(&wire),
distance_mm: wire.distance_mm,
status: wire.status,
at: Instant::now(),
})));
}
}
/// Turn a wire frame into ranges in metres.
///
/// The status byte is the whole point: "nothing is in range" and "the measurement failed"
/// look identical in a distance-only view and mean opposite things to a background model —
/// but *both* are `None` here, because neither is a range, and the difference between them
/// belongs to a consumer that maps rather than to one that plays a note. The codes are the
/// `tof` crate's `Frame::zone`, which `robotd` cannot call without linking that crate's
/// vendored C.
fn interpret(wire: &proto::TofFrame) -> [Option<f64>; N_ZONES] {
/// Status codes ST documents as a usable range: valid, and valid with a large pulse.
const VALID: [u8; 2] = [5, 9];
let mut out = [None; N_ZONES];
for (zone, slot) in out.iter_mut().enumerate() {
let Some(&status) = wire.status.get(zone) else {
continue;
};
let Some(&mm) = wire.distance_mm.get(zone) else {
continue;
};
// A negative range under a valid status comes back on a failed convergence; it is
// not a measurement whatever the status says.
if VALID.contains(&status) && mm > 0 {
*slot = Some(f64::from(mm) / 1000.0);
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
fn theremin() -> Theremin {
// A socket path nothing listens on: the reader thread retries in the background and
// the state machine under test is driven by frames pushed in directly.
Theremin::spawn(
PathBuf::from("/nonexistent/tofd.sock"),
hand::Config::default(),
Reprojector::alpha(),
)
// No reader thread: frames are pushed in directly. See `Theremin::new`.
Theremin::new(hand::Config::default())
}
/// A wall at `distance`, as the sensor would report it: every beam reaches it at
/// `d / (beam · axis)`.
fn wall(distance: f64) -> [Option<f64>; N_ZONES] {
let mut out = [None; N_ZONES];
for (slot, beam) in out.iter_mut().zip(Reprojector::alpha().beams()) {
*slot = Some(distance / beam[0]);
/// A frame with `zones` zones at `distance_m` carrying `code`, the rest reporting nothing.
fn frame(distance_m: f64, code: u8, zones: usize) -> (Vec<i16>, Vec<u8>) {
let mut distance = vec![0i16; 64];
let mut status = vec![255u8; 64];
for zone in 0..zones.min(64) {
distance[zone] = (distance_m * 1000.0) as i16;
status[zone] = code;
}
out
(distance, status)
}
fn push(t: &Theremin, ranges_m: [Option<f64>; N_ZONES]) {
fn push(t: &Theremin, distance_m: f64, code: u8, zones: usize) {
let (distance_mm, status) = frame(distance_m, code, zones);
t.latest.store(Some(Arc::new(Frame {
ranges_m,
distance_mm,
status,
at: Instant::now(),
})));
}
/// Drive `frames` ticks, each with a distinct arrival time so the repeat guard does not
/// eat them, and hand back the last note.
fn run(t: &mut Theremin, ranges_m: [Option<f64>; N_ZONES], frames: usize) -> Option<Note> {
let mut last = None;
for _ in 0..frames {
push(t, ranges_m);
last = t.tick([0.0; 4], &Posture::default(), 1.0 / 15.0);
}
last
}
/// A theremin that was never picked up produces nothing at all — not silence, nothing:
/// the loop must not be commanding a mouth for a feature nobody asked for.
#[test]
fn a_theremin_that_is_down_says_nothing() {
let mut t = theremin();
assert!(!t.active());
push(&t, wall(0.5));
assert!(t.tick([0.0; 4], &Posture::default(), 0.02).is_none());
push(&t, 0.3, 5, 8);
assert!(t.tick(Instant::now()).is_none());
}
/// Arming spans frames and is silent while it does, then the background is named.
/// Picked up, it plays at once — no arming, no window, no waiting. That immediacy is the
/// point of the rewrite.
#[test]
fn arming_takes_frames_and_then_names_its_background() {
fn it_plays_on_the_first_frame_after_being_picked_up() {
let mut t = theremin();
t.set_active(true);
assert!(t.active());
let far = wall(1.5);
let note = run(&mut t, far, 1).expect("up");
assert!(!note.state.armed, "one frame is not a background");
assert_eq!(note.mouth, 0.0, "the beak stays shut while arming");
let note = run(&mut t, far, hand::Config::default().min_arming_frames).expect("up");
assert!(note.state.armed, "{:?}", note.state);
let background = note
.state
.background
.expect("armed states name their background");
assert!(background.starts_with("plane at 1.5"), "{background}");
}
/// The scenario the feature has to survive: armed in front of a wall the duck walked up
/// to, then a hand between duck and wall. The wall is silent, the hand plays, and the
/// mouth opens with the pitch.
#[test]
fn a_hand_in_front_of_the_wall_plays_and_the_wall_does_not() {
let mut t = theremin();
t.set_active(true);
let wall_m = 0.45;
let note = run(&mut t, wall(wall_m), 8).expect("up");
assert!(note.state.armed, "{:?}", note.state);
assert_eq!(note.mouth, 0.0, "the wall it armed against must be silent");
assert_eq!(note.state.hand_range_m, None);
// A hand in the middle of the frame, 15 cm in front of that wall.
let mut hand_frame = wall(wall_m);
for row in 2..6 {
for col in 2..6 {
hand_frame[row * COLS + col] = Some(wall_m - 0.15);
}
}
let note = run(&mut t, hand_frame, 1).expect("up");
push(&t, 0.30, 5, 8);
let note = t.tick(Instant::now()).expect("up");
let range = note.state.hand_range_m.expect("a hand");
assert!((range - 0.30).abs() < 0.03, "{range}");
assert!((range - 0.30).abs() < 0.01, "{range}");
assert!(note.mouth > 0.0, "the beak opens on a note");
assert_eq!(note.mouth, note.state.mouth);
assert_eq!(note.state.zones, 8);
}
/// The regression that sent this back to the drawing board: a hand at 40 cm arrives with
/// a consistency-failure status, and the theremin must play it.
#[test]
fn a_hand_past_thirty_centimetres_plays() {
let mut t = theremin();
t.set_active(true);
for code in [4u8, 13] {
for distance in [0.35, 0.45, 0.60] {
push(&t, distance, code, 6);
let note = t.tick(Instant::now()).expect("up");
let range = note
.state
.hand_range_m
.unwrap_or_else(|| panic!("status {code} at {distance} m must play"));
assert!((range - distance).abs() < 0.01);
}
}
}
/// Closer is higher *and* wider: the mouth opening and the pitch are one number, so the
@ -488,20 +356,13 @@ mod tests {
fn the_mouth_opens_with_the_note() {
let mut t = theremin();
t.set_active(true);
run(&mut t, wall(1.5), 8);
let at = |distance: f64, t: &mut Theremin| {
let mut frame = wall(1.5);
for row in 3..5 {
for col in 3..5 {
frame[row * COLS + col] = Some(distance);
}
}
run(t, frame, 1).expect("up").mouth
let at = |t: &mut Theremin, distance: f64| {
push(t, distance, 5, 8);
t.tick(Instant::now()).expect("up").mouth
};
let far = at(0.55, &mut t);
let mid = at(0.35, &mut t);
let near = at(0.18, &mut t);
let far = at(&mut t, 0.65);
let mid = at(&mut t, 0.40);
let near = at(&mut t, 0.12);
assert!(far < mid && mid < near, "{far} {mid} {near}");
assert!(
near > 0.8,
@ -509,155 +370,120 @@ mod tests {
);
}
/// Arming with a hand already in front of the beak is refused, the refusal is readable
/// by a client, and it survives until the instrument is picked up again — a refusal that
/// vanished would present as a theremin that silently does nothing.
/// The readout has to say what the sensor said, not what this build made of it — the
/// diagnostic that would have found the status bug immediately.
#[test]
fn a_refusal_is_reported_and_kept() {
fn the_state_reports_what_the_sensor_actually_said() {
let mut t = theremin();
t.set_active(true);
let mut in_the_way = wall(1.5);
for row in 3..6 {
for col in 3..6 {
in_the_way[row * COLS + col] = Some(0.22);
}
}
let note = run(&mut t, in_the_way, 8).expect("up");
assert!(!note.state.armed);
assert_eq!(
note.state.refused.as_deref(),
Some(Refusal::SomethingInTheWay.as_str())
// Half the frame consistency-failed, the rest saw nothing.
push(&t, 0.40, 4, 32);
let note = t.tick(Instant::now()).expect("up");
let sensor = note
.state
.sensor
.expect("the sensor line is always present");
assert!(sensor.starts_with("32 usable"), "{sensor}");
assert!(
sensor.contains("4*:32"),
"believed codes are marked: {sensor}"
);
// Still refused several frames later, and still silent.
let note = run(&mut t, wall(1.5), 5).expect("up");
assert!(note.state.refused.is_some(), "{:?}", note.state);
assert_eq!(note.mouth, 0.0);
assert!(sensor.contains("255:32"), "{sensor}");
// Asking again re-arms — and now that nothing is in the way, it takes.
t.set_active(true);
let note = run(&mut t, wall(1.5), 8).expect("up");
assert!(note.state.armed, "{:?}", note.state);
// A frame this build ignores entirely still reports itself, and says zero usable —
// which is the difference between "no sensor" and "a sensor we are not listening to".
// On a fresh instrument, so the hold is not bridging the previous frame's hand.
let mut fresh = theremin();
fresh.set_active(true);
push(&fresh, 0.40, 1, 64);
let note = fresh.tick(Instant::now()).expect("up");
let sensor = note.state.sensor.expect("present");
assert!(sensor.starts_with("0 usable"), "{sensor}");
assert!(sensor.contains("1:64"), "{sensor}");
assert_eq!(note.state.hand_range_m, None);
// And on the instrument that *was* playing, that same frame is a bridged dropout
// rather than a silence — reported as held, so the readout can say so.
push(&t, 0.40, 1, 64);
let note = t.tick(Instant::now()).expect("up");
assert!(note.state.held, "{:?}", note.state);
assert_eq!(note.state.hand_range_m, Some(0.40));
}
/// A sensor that stops delivering falls silent with everything else held, and plays
/// again when it comes back — rather than holding the last note forever, or dropping the
/// background and needing a re-arm.
/// A sensor that stops delivering falls silent with the instrument still in hand, and
/// plays again when it comes back.
#[test]
fn a_stale_frame_is_silence_and_not_a_lost_background() {
fn a_stale_frame_is_silence_and_the_instrument_stays_up() {
let mut t = theremin();
t.set_active(true);
let mut hand_frame = wall(1.0);
for row in 3..6 {
for col in 3..6 {
hand_frame[row * COLS + col] = Some(0.25);
}
}
run(&mut t, wall(1.0), 8);
assert!(run(&mut t, hand_frame, 1).expect("up").mouth > 0.0);
push(&t, 0.25, 5, 8);
assert!(t.tick(Instant::now()).expect("up").mouth > 0.0);
// Age the frame past the staleness window.
let (distance_mm, status) = frame(0.25, 5, 8);
t.latest.store(Some(Arc::new(Frame {
ranges_m: hand_frame,
distance_mm,
status,
at: Instant::now() - FRAME_STALE * 2,
})));
let note = t.tick([0.0; 4], &Posture::default(), 0.02).expect("up");
let note = t.tick(Instant::now()).expect("still up");
assert_eq!(note.mouth, 0.0, "a dead sensor closes the beak");
assert_eq!(note.closeness, None, "and plays nothing");
// Frames return; the background was kept, so the hand plays again straight away.
let note = run(&mut t, hand_frame, 1).expect("up");
assert!(note.mouth > 0.0, "the background survived the gap");
assert!(note.state.armed);
}
/// A frozen sensor must not be able to fill an arming window with copies of one frame:
/// that background would be a picture of a stall.
#[test]
fn a_repeated_frame_does_not_fill_the_arming_window() {
let mut t = theremin();
t.set_active(true);
push(&t, wall(1.0));
for _ in 0..50 {
let note = t.tick([0.0; 4], &Posture::default(), 0.02).expect("up");
assert!(
!note.state.armed,
"one frame, however often read, is one frame"
);
}
}
/// Walking puts the instrument down rather than quietly changing its zero: a background
/// is a picture of where the duck was standing.
#[test]
fn moving_puts_the_instrument_down() {
let mut t = theremin();
t.set_active(true);
run(&mut t, wall(0.5), 8);
assert!(t.active());
t.invalidate("walking");
assert!(!t.active());
assert!(t.tick([0.0; 4], &Posture::default(), 0.02).is_none());
}
/// Picking up an instrument already in hand must not restart its arming window — that
/// would drop the background out from under a player mid-note.
#[test]
fn picking_up_twice_does_not_re_arm() {
let mut t = theremin();
t.set_active(true);
run(&mut t, wall(0.5), 8);
assert_eq!(note.closeness, None);
assert_eq!(note.state.sensor.as_deref(), Some("no depth frames"));
assert!(
t.tick([0.0; 4], &Posture::default(), 0.02)
.expect("up")
.state
.armed
t.active(),
"the instrument is not taken away by a quiet sensor"
);
t.set_active(true);
let note = run(&mut t, wall(0.5), 1).expect("up");
assert!(note.state.armed, "still armed, not arming again");
push(&t, 0.25, 5, 8);
assert!(t.tick(Instant::now()).expect("up").mouth > 0.0);
}
/// The wire's status byte decides what a distance means. A frame of failed measurements
/// is not a frame of near returns, however small the numbers in it are.
/// A dropped frame does not chop the note — the chop was the loudest thing wrong with the
/// first version on a real duck.
#[test]
fn only_a_valid_status_is_a_range() {
let wire = proto::TofFrame {
seq: 1,
at_us: 0,
rows: ROWS as u8,
cols: COLS as u8,
distance_mm: vec![250; N_ZONES],
status: {
let mut status = vec![4u8; N_ZONES];
status[0] = 5;
status[1] = 9;
status[2] = 255;
status
},
};
let ranges = interpret(&wire);
assert_eq!(ranges[0], Some(0.25), "status 5 is a range");
assert_eq!(ranges[1], Some(0.25), "status 9 is a range");
assert_eq!(ranges[2], None, "status 255 measured nothing");
assert_eq!(ranges[3], None, "status 4 failed");
fn a_flickering_sensor_does_not_chop_the_note() {
let mut t = theremin();
t.set_active(true);
let mut played = 0;
// An explicit clock, not `Instant::now()`: the hold is a duration, and a test that
// read the wall clock would depend on whether its thread was descheduled — which it
// duly was, under a parallel suite. The frames are pushed fresh each step so only the
// hold is under test.
let start = Instant::now();
// Alternate usable and unusable frames, as the sensor does at the edge of its range.
for step in 0..20 {
if step % 2 == 0 {
push(&t, 0.35, 5, 6);
} else {
push(&t, 0.35, 1, 6);
}
let note = t
.tick(start + Duration::from_millis(66 * step))
.expect("up");
if note.closeness.is_some() {
played += 1;
}
}
assert_eq!(played, 20, "every frame must sound, half of them held");
}
// A valid status over a negative distance is a failed convergence, not a range.
let negative = proto::TofFrame {
distance_mm: vec![-3; N_ZONES],
status: vec![5; N_ZONES],
..wire
};
assert_eq!(interpret(&negative)[0], None);
/// Putting the instrument down forgets the held note, so picking it back up does not open
/// with a note from before.
#[test]
fn putting_it_down_forgets_the_held_note() {
let mut t = theremin();
t.set_active(true);
push(&t, 0.25, 5, 8);
assert!(t.tick(Instant::now()).expect("up").closeness.is_some());
// A short frame from a peer of another release must not panic or read past its end.
let ragged = proto::TofFrame {
distance_mm: vec![300; 3],
status: vec![5; 2],
..Default::default()
};
let ranges = interpret(&ragged);
assert_eq!(ranges[0], Some(0.3));
assert_eq!(ranges[2], None, "no status for this zone");
assert_eq!(ranges[N_ZONES - 1], None, "past the end");
t.set_active(false);
assert!(t.tick(Instant::now()).is_none());
// Up again, with a frame carrying nothing: silence, not the old note.
t.set_active(true);
push(&t, 0.25, 1, 8);
let note = t.tick(Instant::now()).expect("up");
assert_eq!(note.closeness, None);
assert_eq!(note.mouth, 0.0);
}
}