# Radar System

A real-time radar acquisition and imaging system for vector-network-analyzer (VNA)
and ADC front-ends. It drives the radar through a frequency sweep, calibrates and
preprocesses the measured S-parameters, runs a selectable imaging/detection
processor, and presents the live result in a desktop GUI and an in-process web
interface. The same application runs interactively on a workstation or unattended
as a headless Raspberry Pi appliance.

## Overview

The system is split into a **C++ real-time pipeline** and a **Python application**
that supervises it and handles visualization and control.

The pipeline runs as three independent processes connected by lock-free POSIX
shared-memory ring buffers. Each stage advances on its own, and a slow consumer
never blocks a producer — the rings are latest-wins, so a reader that falls behind
simply skips to the freshest frame:

1. **Acquisition** — `sweep_orchestrator` (or a hardware-specific producer)
   configures the radar, performs the frequency sweep, and publishes raw S11/S21
   traces.
2. **Preprocessing** — `data_preprocessor` applies through/OSL calibration,
   reference subtraction, and optional frequency-domain notch filtering.
3. **Processing** — `data_processor` runs the selected processing mode and emits
   results (pass-through traces, B-scan images, GPR images with detected objects,
   or 3D SAR views). Detected objects are additionally broadcast to subscribers
   over a TCP locator server.

The **Python application** (`python_app`) starts and monitors the C++ binaries,
reads the result rings, renders the live view with pyqtgraph, and exposes the same
controls plus a pixel-identical live plot through an embedded web UI.

```
 radar hardware
       │
       ▼
 Acquisition ──[raw ring]──▶ Preprocessing ──[preprocessed ring]──▶ Processing ──[result ring]──▶ GUI / Web
                                                                          │
                                                                          └──▶ TCP locator (detected objects)
```

## Supported hardware

- **librevna** — single LibreVNA.
- **librevna_multi** — synchronized multi-device LibreVNA.
- **compact_m_k209** — Compact-M K209 / S2VNA (including Raspberry Pi remote mode).
- **sn9000** — SNVNA SN9000 over VISA (pure-Python `@py` backend).
- **kamil_adc** — external Kamil ADC acquisition process, with optional laser control.
- **mock / simulator** — driver-less mode for development without hardware.

## Processing modes

- **pass_through** — magnitude and phase of the selected channel for each combo.
- **bscan** — depth-versus-sweep B-scan heatmap.
- **gpr** — coherent back-projection GPR imaging with object detection.
- **legacy_gpr** — ellipse-intersection MIMO localization (point and extended/region modes).
- **sar3d** — 3D SAR of the reference notebook `Multi_antenna_3D_clean_v1.ipynb`
  on an NVIDIA GPU: front (X-Y), side (Z-Y), top (X-Z) and 3D views of the imaged
  volume. Its settings carry the notebook's parameter names; the scene window is
  fixed (X −2…2 m, height −0.5…2 m above the ground), its range extent follows
  `MIN_DEPTH`/`MAX_DEPTH`, and the voxels are cubes with edge `SAR_VOXEL_M`
  (default 2 cm). The radar motion is corrected from each trace's capture
  timestamps, so the sweep timing needs no parameters.

## Repository layout

| Path | Contents |
| --- | --- |
| `data_acq_and_processing/` | C++ pipeline: `sweep_orchestrator/`, `preprocessing/`, `processing/` (including `locator/`), shared `common_cpp/` (config, IPC rings), and vendored `third_party/` (Eigen, nlohmann/json). |
| `python_app/` | Python application: `gui/`, `orchestration/` (process supervisor, shared-memory readers, config), `hardware_full/` (device drivers and services), `webui/`, `storage/`, `models/`, `workflows/`, `scripts/` (producers and tools), and `tests/`. |
| `run_config_examples/` | Ready-made configuration profiles, one per hardware setup. |
| `deploy/` | systemd daemon installer for Raspberry Pi appliances. |
| `build/bin/` | Compiled C++ binaries. |
| `device_firmware/`, `docs/` | Device firmware and vendor manuals. |

## Requirements

- Linux (POSIX shared memory; GPIO on Raspberry Pi). The deployment target is a
  **Raspberry Pi 5**; hardware PWM gating for the generator sweep is Pi 5
  specific (GPIO12/13/18/19, `dtoverlay=pwm-2chan`).
- A C++20 compiler and `make` — Eigen and nlohmann/json are vendored, so no extra C++ dependencies are needed.
- **Python 3.10 or newer**, in a virtual environment.
- For the `sar3d` processor: an NVIDIA GPU with its driver (`libcuda`). The
  kernels are compiled at run time by NVRTC, so there is no nvcc step; `make`
  needs only `nvrtc.h`, `cuda.h` and `libnvrtc`, taken from the pip
  `nvidia-cuda-*` packages in `.venv` (`requirements.txt`, x86_64; CUDA 13 covers
  Turing and newer GPUs) or from a system toolkit in `/usr/local/cuda` (Jetson).
  Without them everything else still builds, and sar3d reports that CUDA is
  missing — a Raspberry Pi needs none of it.

## Build and run

From the repository root:

```bash
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
make                     # build the C++ binaries into build/bin/
```

`make` builds the sar3d GPU code when it finds CUDA (see Requirements) and
prints a note when it does not. After installing the CUDA requirements into an
existing venv, run `make` again: it notices the change and rebuilds that part.

Run the desktop GUI:

```bash
.venv/bin/python -m python_app.gui.main
# or, with the build step and single-instance handling:
./start.sh
```

`start.sh` supports `--headless` (offscreen, auto-start — for unattended Pi use),
`--profile PATH`, `--producer-only`, `--skip-build`, and `--clean-shm`; see
`./start.sh --help`. The acquisition device (LibreVNA, Kamil ADC, …) is detected
automatically from the active config's `radar.model` — there is no per-device flag.

## Jetson (aarch64) bring-up

Ubuntu 22.04 on Jetson has glibc 2.35, and PyQt6 6.8+ ships only
`manylinux_2_39` aarch64 wheels — `requirements.txt` therefore pins PyQt6 to
`<6.8` on aarch64 (6.7.1 is the last release with a compatible wheel). With
that pin the stock `./start.sh` provisioning works; the extra steps are only
for the CUDA-backed `sar3d` processor:

```bash
# 1. copy the replay dataset (for dataset-replay profiles) and point
#    radar.replay.dataset_dir in the profile at the local copy — the path in
#    run_config_examples/*.json is machine-specific.

# 2. normal launch; make takes the JetPack CUDA from /usr/local/cuda
#    (requirements.txt installs no pip nvidia-* packages on aarch64) and the
#    driver's /usr/lib/aarch64-linux-gnu/nvidia/libcuda.so.1:
./start.sh --profile run_config_examples/run_config_replay_vna.example.json
```

## Running without hardware (emulation)

Two ways to run the full pipeline with no radar attached:

**Mock producer (synthetic data).** The default `run_config.json` already sets
`radar.driver_mode: "mock"`, so the producer generates synthetic S-parameters.
The preprocessor still requires S21 calibration/reference sets; create them once:

```bash
.venv/bin/python -m python_app.scripts.manual_smoke_run --duration 1.0  # creates smoke_cal/smoke_ref
./start.sh   # then in the GUI: Preprocessing Panel -> pick the sets -> Start
```

(Alternatively use the GUI's "Create neutral sets" button, which makes
identity calibration + zero reference so preprocessing is a pass-through.)

**Dataset replay (recorded scene, meaningful sar3d picture).** When
`radar.replay.dataset_dir` is set in the profile, the supervisor launches
`python_app.scripts.replay_raw_producer` instead of the normal producer: it
replays recorded snapshots into the raw ring at the recorded cadence, looping
forever with a monotonic timeline. Pair it with NEUTRAL preprocess sets so the
already-preprocessed recording passes through the preprocessor bit-exactly.
A ready profile for the VNA_AlGo tank005 dataset is provided:

```bash
# one-time: neutral sets for this profile (or use the GUI button)
.venv/bin/python -m python_app.scripts.create_neutral_preprocess_sets \
    --config run_config_examples/run_config_replay_vna.example.json
# interactive GUI (mode sar3d is pre-selected by the profile's gui section):
./start.sh --profile run_config_examples/run_config_replay_vna.example.json
```

The sar3d processor keeps about 75 MB of buffers on the GPU at the default
2 cm voxel and depth 2…8 m (plus the CUDA context). The volume grows as
1/`SAR_VOXEL_M`³: 1 cm takes about 600 MB and ran about 6 times slower in
testing. If the GPU has no room, it logs the CUDA error once and publishes empty
results until memory frees.

The web interface is served in-process at `http://<host>:8080`. It mirrors the live
plot and offers Start / Stop / Capture controls and the active processor settings.

## Configuration

The active profile is `run_config.json`. It selects the radar model and sweep, the
switch matrix and acquisition combos, the preprocessing calibration/reference
assets, the GPR geometry, and the shared-memory ring sizes. Profiles for each
hardware setup are provided in `run_config_examples/` — copy one to
`run_config.json`, or pass it with `--profile`, to use it.

## Deployment

On a Raspberry Pi, `deploy/install-daemon.sh` installs a systemd service that runs
the application headless and restarts it on failure. A single-instance lock ensures
the interactive GUI and the daemon never run at the same time, since they share the
radar, the shared-memory rings, and the locator port.

## Tests

```bash
.venv/bin/python -m unittest discover -s python_app/tests
```