radar_system/README.md

# Radar System

A real-time radar acquisition and imaging system for vector-network-analyzer (VNA)
and ADC front-ends. It drives the radar through a frequency sweep, calibrates and
preprocesses the measured S21 traces, runs a selectable imaging/detection
processor, and presents the live result in a desktop GUI and an in-process web
interface. The same application runs interactively on a workstation or unattended
as a headless Raspberry Pi appliance.

## Overview

The system uses **Python acquisition producers**, **C++ preprocessing and processing**,
and a **Python application** for supervision, visualization and control.

The pipeline runs as three independent processes connected by lock-free POSIX
shared-memory ring buffers. Each stage advances on its own, and a slow consumer
never blocks a producer — the rings are latest-wins, so a reader that falls behind
skips overwritten frames and continues from the retained tail:

1. **Acquisition** — Python producers configure the radar, acquire frequency sweeps,
   and publish raw S21 traces. Both single and multi-device LibreVNA use
   `matrix_raw_producer`; RadioPhotonic radar (RPH) uses `rph_radar_raw_producer` with its C++
   collector (E-502) or the RFG board's UDP stream.
2. **Preprocessing** — `data_preprocessor` applies through calibration,
   reference subtraction, optional frequency-domain notch filtering, and an
   optional object-based MIMO correction of S21.
3. **Processing** — `data_processor` runs the selected processing mode and emits
   results (pass-through traces, B-scan images, GPR images with detected objects,
   or 3D SAR views). Detected objects are additionally broadcast to subscribers
   over a TCP locator server.

The **Python application** (`python_app`) starts and monitors the producer and C++ processes,
reads the result rings, renders the live view with pyqtgraph, and exposes acquisition
controls, processing settings and a copy of the live plot through an embedded web UI.

```
 radar hardware
       │
       ▼
 Acquisition ──[raw ring]──▶ Preprocessing ──[preprocessed ring]──▶ Processing ──[result ring]──▶ GUI / Web
                                                                          │
                                                                          └──▶ TCP locator (detected objects)
```

## Supported hardware

- **librevna** — single LibreVNA.
- **librevna_multi** — synchronized multi-device LibreVNA.
- **rph_radar** — [RadioPhotonic radar (RPH) collector](data_acq_and_processing/rph_radar_collector/README.md),
  with manual laser setpoints or LD1 current variation via **Apply Radar**.
  `radar.rph_radar.source` selects the ADC:
  - `e502` (default) — the application owns the collector and exchanges complete
    sweeps over a PTY. Switching to a different antenna pair starts a fresh ADC session
    after settling, discarding queued samples and an initial partial sweep.
  - `rfg_udp` — the RFG board sends averaged sweep points to `udp_bind`
    (`run_config_rph_radar_rfg.example.json`); main = ch0 − ch1, reference = ch2. It
    needs `laser_control.mode: variation` (even when the laser is started elsewhere):
    a sweep must have exactly the task's step count, `ceil((max − min) / step)` in DAC
    codes, with no lost points; others are dropped (the board loses a few percent).
    Capture windows come from kernel receive timestamps. Only one antenna pair (both
    switches with one position): without a source-side marker a sweep after a switch
    cannot be proven fresh.
- **mock / simulator** — driver-less mode for development without hardware.

## Processing modes

- **pass_through** — magnitude and phase of S21 for each combo.
- **bscan** — depth-versus-sweep B-scan heatmap.
- **gpr** — coherent back-projection on the ground plane, as in `Horns_motion_4x4_ground.py`,
  with object detection. Radar height and look angle define the slice in radar X/Z axes;
  the wave speed is 3×10⁸ m/s (air).
- **sar3d** — 3D SAR of the reference notebook `Multi_antenna_demonstrate_depthnorm.ipynb`
  on an NVIDIA GPU through CUDA: front (X-Y), side (Z-Y), top (X-Z)
  and oblique maximum projections of the imaged
  volume. Its settings carry the notebook's parameter names; the scene window is
  set by the selected antennas with a 2 m X margin and height −0.5…2 m above the ground; its range extent follows
  `MIN_DEPTH`/`MAX_DEPTH`, and the voxels are cubes with edge `SAR_VOXEL_M`
  (default 2 cm). The radar motion is corrected from each trace's capture
  timestamps, so the sweep timing needs no parameters.

Both coherent GPR and SAR apply `PAIR_DEPTH_EQUALIZE` (on by default) before the
per-pair median normalization. The notebook's 1.5 m sliding median estimates each
pair's depth-dependent level; positive gains bring the pairs to their geometric
mean while preserving phase and the common depth envelope. **Subtract mean of previous
collections** subtracts up to **Mean count** preceding spectra for each pair before
motion correction and the IFFT; zero count disables it. SAR uses a causal aperture:
`SAR_FRAME_RADIUS` previous frames plus the current frame, with the current scene origin.
The processor retains enough history for the aperture and each frame's preceding
mean; transport ring capacity does not shorten this background window.
Scene width `SAR_X_SIZE_M` and length `SAR_Z_SIZE_M` use 0 for the automatic notebook
bounds; positive values resize the axes about their automatic centres. `SAR_Y_RANGE_M`
sets the height limits. The depth gate remains independent of these scene bounds.

SAR **Zoom (%)** shows the full scene at 100%; increasing it shrinks all four
views uniformly towards the brightest response. **SAR_CLOUD_THRESHOLD** hides
values below the selected fraction of the normalization maximum. Both controls
redraw the existing result without recalculating it. Tx/Rx markers appear on the
front, side and top projections in ground coordinates. A hollow square marks the
peak; its size follows the contiguous half-maximum width along the volume axes
(FWHM, at least one voxel). This measures the imaged response rather than the
reflector's physical dimensions.

The desktop uses a light theme with dark plot canvases. **Mode** stays below the
**Actions** buttons while the settings sections scroll; the settings panel and
runtime log can be collapsed with their arrow buttons. SAR3D status, peak
coordinates and level appear above the projections.

For B-scans on every radar model, **B-scan pair** selects a pair from the displayed
history independently of the run combinations. **Next Combination** advances output
first, then input, and wraps to the first recorded pair; it is enabled with at least
two pairs. Desktop and browser selectors redraw the cached B-scan immediately.

Tx/Rx antenna positions are edited in tables with switch position and X/Y/Z in
metres, on both the desktop and web interface. Coordinate changes apply to the
running processor and recompute its retained result. Profiles
preserve all three coordinates. A new table row becomes active after X is entered;
desktop, web and profile geometry use one Python validator; C++ validates its own
input. GPR/SAR views wait for a result marked with the current settings revision,
so an older in-flight result cannot overwrite new geometry.

## Repository layout

| Path | Contents |
| --- | --- |
| `data_acq_and_processing/` | C++ pipeline: `preprocessing/`, `processing/` (including `locator/`), `rph_radar_collector/`, shared `common_cpp/` (config, IPC rings, H7992 driver), and vendored `third_party/` (Eigen, nlohmann/json). |
| `python_app/` | Python application: `gui/`, `orchestration/` (process supervisor, shared-memory readers, config), `hardware_full/` (device drivers and services), `webui/`, `storage/`, `models/`, `workflows/`, `scripts/` (producers and tools), and `tests/`. |
| `run_config_examples/` | Ready-made configuration profiles, one per hardware setup. |
| `deploy/` | systemd daemon installer, and the USB update kit for the Pi and Jetson (`pack-usb.sh`, `update.sh`). |
| `build/bin/` | Compiled C++ binaries. |
| `device_firmware/` | Device firmware and protocol documentation. |

## Requirements

- Linux (POSIX shared memory; GPIO on Raspberry Pi). The deployment target is a
  **Raspberry Pi 5**.
- A C++20 compiler and `make`; Eigen and nlohmann/json are vendored.
- `libusb-1.0` runtime and `libatomic.so.1` (`libatomic1` on Debian/Ubuntu).
  Python ring readers/writers use lock-free atomic loads, stores and cursor CAS
  through libatomic, including memory barriers for ARM. Slot sizes must be multiples
  of 8 bytes, and the complete ring mapping must not exceed 1 TiB.
- **Python 3.10 or newer**, in a virtual environment.
- For `sar3d`: an NVIDIA GPU with its driver (`libcuda`). The
  kernels are compiled at run time by NVRTC, so there is no nvcc step; `make`
  needs only `nvrtc.h`, `cuda.h` and `libnvrtc`, taken from the pip
  `nvidia-cuda-*` packages in `.venv` (`requirements.txt`, x86_64) or from
  a system toolkit in `/usr/local/cuda` (Jetson).
  Without these dependencies or a usable GPU, SAR3D shows a warning above the
  plots and does not calculate an image. Other processing modes remain available.

## Build and run

From the repository root:

```bash
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
make                     # data_preprocessor and data_processor in build/bin/
make rph_radar_collector  # RadioPhotonic radar (RPH) with radar.rph_radar.source: e502
```

`make` builds the sar3d GPU code when it finds CUDA (see Requirements) and
prints a note when it does not. After installing the CUDA requirements into an
existing venv, run `make` again: it notices the change and rebuilds that part.
CUDA 13 is installed from `requirements.txt` on x86_64. For GPUs older than
sm_75, install NVRTC 12 outside `.venv` and build with
`make BUILD_DIR=/tmp/build-nvrtc12 CUDA_ROOTS="<dir>/nvidia/cuda_nvrtc <dir>/nvidia/cuda_runtime"`.
A separate build directory keeps the two CUDA builds independent.

Run the desktop GUI through the launcher for build and single-instance handling:

```bash
./start.sh --profile run_config.json          # explicit local mock profile
# or choose a hardware profile from run_config_examples/
```

`start.sh` also supports `--headless` (offscreen, auto-start), `--auto-start`,
`--producer-only`, `--skip-build`, `--build-only`, and `--clean-shm`; see
`./start.sh --help`. Pass `--profile PATH` so launcher provisioning and the GUI
use the same profile. The producer is selected from `radar.model`; there is no
per-device flag. Provision dependencies
before unattended use: headless startup does not install missing packages.

For development, `.venv/bin/python -m python_app.gui.main` starts the GUI directly,
without the launcher's build step or single-instance lock.

## Jetson (aarch64) bring-up

`requirements.txt` installs PyQt6 from wheels only, so pip picks the newest release
the system glibc can run: 6.7 on Jetson's glibc 2.35, current releases on Debian 13. CUDA comes from JetPack in `/usr/local/cuda`; pip CUDA packages
are installed only on x86_64. Use the same build commands and launcher as above.

The USB updater can create a fresh venv with an already installed interpreter,
for example `bash radar_update/update.sh --python 3.13`. The system Python does
not need to be replaced.

## Running without hardware (emulation)

The default `run_config.json` already sets `radar.driver_mode: "mock"`, so the
producer generates synthetic S-parameters and no radar is needed. Mock acquisition
supports the LibreVNA models; RadioPhotonic radar (RPH) needs native acquisition.
Both LibreVNA modes share one simulator: a point reflector moves back and forth
along the antenna array and between depths 3.25–4.25 m over an eight-second cycle.
Each S21 uses the actual transmitter–target–receiver distance from the configured
XYZ coordinates; coincident antennas produce identical signals. All combinations
share the same target position and measured capture window, with collections paced
at 50 ms intervals. The root mock profile explicitly selects a 0–8 m B-scan
range. Both shipped mock profiles use a 0.445 m vertical antenna baseline
(Rx1/Rx3 and, for the four-output matrix, Tx2/Tx3), avoiding coincident pairs.
The general defaults are 0.824–1.0 m; stored profiles retain their range. The
simulator models a stationary radar, so radar speed should be zero.
The scene uses the antenna positions from its startup profile; live geometry edits
change the reconstruction of captured sweeps, whose samples remain unchanged.
The preprocessor still requires S21 calibration/reference sets. Start the GUI:

```bash
./start.sh --profile run_config.json
```

Open **Preprocessing…**, use **Create Neutral S21 Sets**, then **Start**. This makes
identity calibration + zero reference, preserving the simulated S21 samples.

Finer voxels increase SAR3D work as 1/`SAR_VOXEL_M`³. Increase the voxel size when
a faster preview matters more than spatial resolution. Processing failures appear
above the plots and a failed single capture
finishes with its explanation while retaining the acquired data.

## Web interface

The embedded server listens at `http://<host>:8080`; set
`RADAR_SYSTEM_WEBUI_PORT` to use another port. It is available in desktop and
headless mode. Browser actions are:

- **Start**, **Single Capture**, **Stop**, **Capture Temporary Reference** and **Remove Last Measurement**;
- load a profile from `run_configs/*.json` using **Load**;
- **Save Last 1000** and **Start + Record to Disk** with a path, optional name and recording count;
- edit processor settings and antenna tables with **Apply**, or clear history with **Clear Runtime History**;
- select a **B-scan pair** or advance with **Next Combination**.

Copy profiles from `run_config_examples/` into `run_configs/` to offer them in the
browser. JSON export and calibration dialogs are desktop actions. PNG capture
runs only while a browser is connected and the plot has changed, including zoom
and resize, at most five times per second. Each client retains the latest frame,
status and settings independently; reconnecting refreshes settings, and **Apply**
sends only edited fields. Field labels, units and checkboxes match the desktop form.

## Data and preprocessing

### Snapshots and recording

**Save Last 1000** and **Save S21 JSON** keep up to 1000 available collections from
each stage independently, including raw or preprocessed sweeps without a result.
The limit is fixed; legacy profile `save_count` values are ignored. Saving
keeps an active disk recording running.
Without raw/preprocessed taps, result snapshots remain available, but S21 JSON
requires retained traces.
**Start + Record to Disk** has a separate count
for future results. It keeps collections received after arming until that count,
Stop, or exhaustion of the bounded writer queue. If storage falls behind,
recording stops accepting data, drains the queue and final chunk, and reports the limit.
Stop and closing the window also flush accepted data without blocking the GUI.
Pausing acquisition for a capture or history action ends an active recording:
the log reports the accepted count, requested count and stop reason before draining
the writer. Latest-wins rings cannot recover sweeps overwritten before the GUI
receives them. NumPy datasets preserve S21, capture
windows, every supported result payload, and processing time. Failed exports remove
their staging files, leaving existing datasets intact. Companion profiles retain
the radar, antenna layout and preprocessing used for acquisition, including when
another profile is loaded after Stop. Dataset name prefixes also describe the acquisition;
a manual save is cancelled if acquisition is restarted during its final history drain.

Each NumPy dataset includes `processing_settings.json`, mapping result revisions to
the live settings used for processing. Results retain their revision, capture window,
and actual `speed_m_s` (including speed supplied by the locator). Settings
changed while acquisition is stopped are recorded for the reprocessed results as well.
**Remove Last Measurement** uses capture order and removes matching collections
from every stage; the processor suppresses that sweep if it is still pending.
**Clear Runtime History** clears all retained
stages and rejects late results from before the cutoff. These actions leave saved
files intact. A sweep is identified by `(collection_id, capture_start_ns)`; all
stages preserve the producer's capture window. `monotonic_ns` is diagnostic only.

**Save S21 JSON** can add exports to an existing NumPy dataset. Existing JSON files
are never overwritten; missing raw/preprocessed counterparts produce empty arrays,
and stages pair only by the exact acquisition key. Its `vna_config_profile.json` companion
leaves the NumPy dataset's `config_profile.json` intact. A streaming write failure
keeps all collections already published; only the incomplete collection is discarded.
Legacy snapshots without capture windows retain repeated IDs through one-to-one
time/index matching; processing completion timestamps are not compared with
acquisition timestamps.

### Calibration and reference sets

Calibration and reference sets keep their trace arrays and metadata together in one
atomically replaced NPZ archive, with a readable JSON sidecar. Both files and their
directory entries are synced before successful completion. Existing file permissions
are retained; new files and directories respect the process umask. Earlier NPZ/JSON pairs
remain supported. Invalid trace shapes, duplicate combinations and non-finite
calibration/reference values are rejected before a saved set is replaced.
Median captures also require identical frequency axes and channel coverage.
Open **Preprocessing…** to create or select the ordinary S21 calibration and
empty-scene reference. The processing order is through calibration → calibrated
reference subtraction → notch → object correction → the selected processor.

### Object calibration

**Create…** in the **Object calibration** row opens a separate dialog. Use
**Capture Data** to take fresh preprocessed sweeps, then enter the measured
reflector centre X/Y/Z in metres and use **Calculate and Save**. The optional
**Calculation Settings** section expands inside the scrollable dialog body;
the calculation, cancellation and close buttons remain visible below it.
Coordinates follow the notebook: X across the array, Y above the ground, Z along
the ground.
Height above ground, radar pitch and sphere radius follow `MIMO_calibration.ipynb`;
several reflector positions may contribute to one calibration. The only diagnostic
plot shows the phase of each antenna pair's contribution at the target point,
before and after correction, as in cell 5 of `MIMO_calibration.ipynb`: Tx rows,
Rx columns, phase relative to the complex sum of each map, with cell values and
standard deviation in degrees on a cyclic ±180° scale.
With partial antenna geometry, only the specified Tx × Rx pairs contribute to
the calculation and phase maps. Acquisition still retains every physical stage;
pairs outside that geometry receive unit correction and appear in `unchanged_combos`.
The selected reference is already subtracted, and the previous object correction
is disabled while these sweeps are acquired. A capture owns an exclusive pipeline
pause until it finishes or is cancelled, including during automatic-restart backoff.
An explicit **Stop** cancels automatic resumption.

Object corrections use the same NPZ collection format as other preprocessing sets:
S21 holds each pair's multiplier on the captured frequency grid. Metadata retain
the capture radar key, antenna geometry, through-set name and content hash, target
positions, estimation settings and phase diagnostics. Saving uses the capture's
configuration even if the current profile changes during calculation. The next
**Start** checks the selected through set's S21 and frequency arrays, geometry and
coverage of the requested combinations before applying the correction after notch
filtering and before any processor. **Off** disables it. Changing antenna geometry
disables an incompatible correction and reloads acquisition without restarting
the application. Changing relative permittivity only updates processing settings.

Legacy `cal_*.npy` / `cal_meta.json` folders are not application calibration sets.
Capture a set through **Create…** to use object correction; the legacy files on
disk are left intact. Ordinary through/reference NPZ sets need no conversion.

Older NPZ sets and snapshots remain readable: extra S11 arrays are ignored;
new trace files contain only frequency and S21.

## Runtime interfaces

Tap rings are optional: an empty `rings.raw_tap.name` or
`rings.preprocessed_tap.name` disables that copy and its GUI history. Core processing
and single capture still work without taps; object calibration needs the preprocessed
tap. All producers share raw publication and hardware retry handling.

Raw and preprocessed IPC use **RAW3** and **PRP3** respectively: each trace contains
float32 frequencies followed by complex64 S21 (12 bytes per point). The complete
frame requires collection and per-trace capture windows; its size is `44 + combos * (28 + 12 * points)` bytes.
Older RAW2/PPR2 bundles are incompatible and are rebuilt from NPZ sets at Start;
update Python and C++ together. Results use RSL3; older RSL2 frames are rejected.

RF switches use H7992 with 1–4 positions and two GPIO control lines. Profiles use
`driver_mode`, GPIO settings and the position count. An old `driver: "h7992"` is accepted;
other explicit driver values or enabled inversion are rejected before GPIO opens.
Two positions use the existing H7992 mapping `(A,B) = (0,0), (0,1)`; physical
wiring and channel numbers follow the profile.

Single-device LibreVNA acquires the configured `run.combos` in order. Multi-device
acquisition keeps the full matrix: SAR3D derives intra-sweep stage timing from
the complete list of outputs sharing a capture window.

## Configuration

A profile selects the radar model and sweep, switches and acquisition combos,
preprocessing sets, antenna geometry, ring sizes and GUI settings. The GUI loads
`RADAR_SYSTEM_PROFILE` (set by `start.sh --profile`), otherwise the last profile in
`python_app/runtime/gui_session_state.json`, otherwise root `run_config.json`.
If a selected external profile cannot be loaded, it reports the error and falls
back to the root profile. **Load Config** and **Save Config** manage desktop profiles.

Hardware examples are in `run_config_examples/`. The root `run_config.json` is
intentionally mock; select a hardware profile explicitly on a real device. A
change of model, serials, sweep parameters or multi-device switch positions changes
the `radar_key` used to find calibration/reference sets; select or capture matching
sets before **Start**. Changing the multi-device reference-clock source also resets
display history. For RadioPhotonic radar (RPH), identity follows the phase anchors, output
band and acquisition/laser settings; executable/TTY paths and LibreVNA-only sweep
fields do not affect it. RPH processing frequency controls use its output band.
Phase, band and timeout errors are rejected before a running pipeline is stopped.

Profiles with model `kamil_adc` are rejected with an explicit error: use
`rph_radar` and the `radar.rph_radar` section. An inactive `radar.kamil_adc`
section left in older saved profiles is ignored; there are no aliases or automatic
data migrations. Old RPH calibration/reference sets use a different
measurement key and are not selected by the renamed mode.

### RPH phase calibration

Use `run_config_examples/run_config_rph_radar.example.json` (E-502) or
`run_config_rph_radar_rfg.example.json` (RFG board) for **RadioPhotonic radar (RPH)**.
Apply the desired laser settings with **Apply Radar**, then stop acquisition before
running the calibration tool:

```bash
.venv/bin/python -m python_app.scripts.rph_radar_calibrate --config run_config.json --sweeps 200
.venv/bin/python -m python_app.scripts.rph_radar_calibrate --config run_config.json --sweeps 200 --apply
```

Pass the path to your active RPH profile. The first command reports measured phase
anchors; `--apply` captures again and writes only `phase0_rad` and `phase1_rad` in
`radar.rph_radar.phase_calibration`. Frequency anchors must already describe the
known sweep endpoints. The tool uses current acquisition settings, sends no laser
commands, preserves every other JSON byte, and refuses to overwrite a profile
edited during capture. Band and point count stay unchanged. `--warmup` controls
how many initial sweeps to discard (default 10).

## Deployment

On a Raspberry Pi, `deploy/install-daemon.sh` installs a systemd service that runs
the application through `start.sh --headless --skip-build` and restarts it on
failure. The launcher holds a single-instance lock for the radar, rings and locator
port; an interactive launch first asks systemd to stop an active daemon.

**Updating a device from a USB flash drive.** On the development machine
`deploy/pack-usb.sh /path/to/drive` writes the host part of the **last commit**
and its update script into `radar_update/` on the flash drive. Uncommitted changes,
venv, build outputs, recorded data and local configs are excluded.
On the Pi or Jetson, close the program and run, from the drive's folder:

```bash
bash radar_update/update.sh                  # or: --python 3.13 for a fresh venv
```

Run the updater as the normal device user. It installs the newest archive into
`~/Documents/radar_system` (or `RADAR_DIR`), renaming the current version to the
next free `radar_system<N>`. It preserves:

- `run_config.json` and changed/added `run_config_examples/*.json` by independent
  copy; a conflicting release profile is placed beside the device profile as `.new`;
- `python_app/runtime/` by copy, excluding logs and `*.pids`;
- `python_app/data*` and extra non-hidden top-level entries absent from the release
  by hard links, excluding `build`, `graphify-out`, `device_firmware`, `docs` and `RELEASE`;
- the venv by independent copy, or a fresh one when `--python` is supplied.

Saved data is written through replacement or new files, preserving the backup's
hard links. `RELEASE` gets a new inode. The updater builds the C++ pipeline and ADC
collector, then restarts `radar.service` only if it was running before the update.
Keep the device's real hardware profile: the release's root profile is mock.

For rollback, stop `radar.service` and close the GUI, rename the new installation
aside to preserve measurements taken since the update, then rename the chosen
backup to `radar_system`. Restart the service if it is normally used.

## Tests

```bash
QT_QPA_PLATFORM=offscreen .venv/bin/python -m unittest discover -s python_app/tests
make check-native
make check-sar3d PYTHON=.venv/bin/python
make check-gpr
ASAN_OPTIONS=detect_leaks=0 make check-native NATIVE_SANITIZERS=address,undefined
```

The tests cover failed writes and settings commits, recordings during manual Save,
repeated Start/Stop/Close, web-action cancellation, malformed IPC frames, hardware
resource cleanup with mock failures, and an isolated deployment update. Native tests
also exercise mutated frames, ring concurrency, locator input and ADC whole-sweep PTY overflow recovery.
Firmware host tests check cart timer rollover and SBUS packing; they do not validate
physical timing. Real devices, GPIO and the CUDA backend need their own hardware run.

`check-sar3d` runs the real CUDA processor (`data_acq_and_processing/tests/test_backprojection.py`).
It compares it with `sar3d_reference.npz`: compact float32/complex64 inputs, the three
maximum projections of the notebook's volume, its joint peak/level and response
widths for six compensation/normalization, equalization, background, scene and
timing cases. The expected values come from unchanged functions in cells 1 and 14
of the original notebook, with each frame's previous-collection mean subtracted
from its input spectra by NumPy. The same test checks the published
status for invalid input, a result too large for the ring slot, an empty and a very weak
signal. Without a usable GPU it checks only the CUDA warning and skips the rest; the
notebook itself is not needed to run it. To regenerate the reference, pass the
original notebook (it is kept outside the repository):

```bash
.venv/bin/python data_acq_and_processing/tests/generate_backprojection_reference.py \
    --notebook /path/to/Multi_antenna_demonstrate_depthnorm.ipynb \
    --gpr-script /path/to/Horns_motion_4x4_ground.py
```

The generator also writes `gpr_reference.npz` from the unchanged Horns coherent BP
functions, with the notebook equalizer inserted before scalar normalization.
`make check-gpr` compares the CPU processor with those images, including motion and
previous-collection subtraction. Neither source file is needed to run the tests.

The archives record source SHA256 hashes and NumPy/SciPy versions. Another build
can be checked with `make check-sar3d BUILD_DIR=/path/to/build` or by running
`test_backprojection.py Sar3dTest` with
`BACKPROJECTION_FIXTURE=/path/to/build/native_tests/backprojection_fixture`.
For older GPUs, pass the same `BUILD_DIR` and NVRTC 12 `CUDA_ROOTS` used to build.

For development, install `pyright` into the venv, then run
`.venv/bin/pyright python_app` to check all Python modules and tests.

## Application architecture

Invalid device/switch settings and undersized raw/preprocessed slots fail before
acquisition starts. Oversized processed results report the required slot size in
the log; switching to a mode that fits resumes processing without a restart.

`AppWindow` assembles controllers and shuts them down. `WindowView` owns layout
and typed sections; controllers receive concrete dependencies, without GUI mixins
or duplicate window state. `ProfileController` owns startup selection, session
state and profile persistence. `RadarSettingsController` applies device limits,
while `LiveSettingsController` publishes revisions and updates plots.

GUI scalar settings use dataclass metadata in `models/gui_profile_schema.py`:
profile validation, desktop values, live processing fields, processing log summaries
and web forms share those bindings. Hardware fields and geometry have their own
typed mappings and validators. Section builders return typed widget groups; their
controllers connect signals directly, with no string table of action callbacks.
`gui/controllers/config_controller.py` reads and applies scalar metadata without owning
the window or acquisition processes. Web batches roll back together when validation
or live-file publication fails; desktop settings also revert to the last publication.
Profile loading blocks editor signals until all settings are populated.

`HistoryController` owns retained acquisitions; `PlotController` selects the view
and supplies independent B-scan, GPR, trace and SAR renderers with typed settings.
Renderers own their plots and display caches, without access to the acquisition
processes. Stopped B-scan replay uses the last run's ring, matches exact acquisition
keys and the requested settings revision, and cancels when that run changes. With
no retained preprocessed sweeps (including disabled taps), the processor recomputes
its own retained history. Capture previews preserve the selected processing view.
`PipelineController` owns processes, ring readers, polling/recovery, Single Capture
and replay. Its pause prevents recovery from starting a producer over a hardware
capture; Stop or Close cancels any pending resumption, including during a drain.
Validation rejects Start before stopping existing processes. Pauses resume only
continuous acquisition, retry failed resumption with the usual backoff, and release
ownership before presenting capture errors. GPIO errors are logged without dialogs.
Rendering failures leave acquisition and the locator running and log each cause once;
repeated ring-read failures retain their separate recovery and stop policy.
`PreprocessController` owns calibration/reference sessions and asset selection.
One `SequentialCaptureSession` handles both current settings and a list of sweep
variants; a single variant adds no extra acquisition or reconfiguration. Multi-device
collection always captures one complete synchronized cycle. The native
`DataPreprocessor` owns all three bundles in `preprocessing/bundles`: through,
through-calibrated reference and object correction, applied in that order with notch
filtering before object correction.
Object capture checks both acquisition processes and releases its pause only after
completion or cancellation; losing the producer reports a failure immediately.
`SnapshotController` coordinates manual exports, paused Clear/Remove operations
and Start + Record to Disk.
History deletion is retained before publishing it to the processor, so a failed
settings write can be retried without restoring deleted sweeps. `RecordingController`
owns a bounded writer session; its disk thread receives data and storage only.
A processor restart during Start + Record to Disk does not finish the newly armed session.
Temporary references and neutral sets clear retained processing history without
restarting the locator while the pipeline is paused.

`WebBridge` queues browser actions onto Qt and exposes immutable status/frame
snapshots. Clear and Remove use separate action endpoints; live settings only edit
settings. Trace display options (including its Y range) do not publish processing
revisions. A timeout cancels queued actions; an action already running reports that
state without inviting a duplicate request. The GPIO controller invokes the same
reference-capture action as desktop and web, and ignores queued presses after Close.
`LogController` batches worker logs and returns web-action errors to the browser
without opening a desktop modal. Stop/Close also cancels delayed launcher startup.

GUI profiles, run configs, session state and live processing settings use shared
atomic writers with unique temporary files. Profiles and run/session settings sync
their file and directory; frequent live settings avoid extra disk syncs. A failed
write before replacement leaves the previous file intact. The launcher and supervisor
share acquisition producer selection.