Radar System

A real-time radar acquisition and imaging system for vector-network-analyzer (VNA) and ADC front-ends. It drives the radar through a frequency sweep, calibrates and preprocesses the measured S-parameters, runs a selectable imaging/detection processor, and presents the live result in a desktop GUI and an in-process web interface. The same application runs interactively on a workstation or unattended as a headless Raspberry Pi appliance.

Overview

The system is split into a C++ real-time pipeline and a Python application that supervises it and handles visualization and control.

The pipeline runs as three independent processes connected by lock-free POSIX shared-memory ring buffers. Each stage advances on its own, and a slow consumer never blocks a producer — the rings are latest-wins, so a reader that falls behind simply skips to the freshest frame:

Acquisition — sweep_orchestrator (or a hardware-specific producer) configures the radar, performs the frequency sweep, and publishes raw S11/S21 traces.
Preprocessing — data_preprocessor applies through/OSL calibration, reference subtraction, and optional frequency-domain notch filtering.
Processing — data_processor runs the selected processing mode and emits results (pass-through traces, B-scan images, or GPR images with detected objects). Detected objects are additionally broadcast to subscribers over a TCP locator server.

The Python application (python_app) starts and monitors the C++ binaries, reads the result rings, renders the live view with pyqtgraph, and exposes the same controls plus a pixel-identical live plot through an embedded web UI.

 radar hardware
       │
       ▼
 Acquisition ──[raw ring]──▶ Preprocessing ──[preprocessed ring]──▶ Processing ──[result ring]──▶ GUI / Web
                                                                          │
                                                                          └──▶ TCP locator (detected objects)

Supported hardware

librevna — single LibreVNA.
librevna_multi — synchronized multi-device LibreVNA.
compact_m_k209 — Compact-M K209 / S2VNA (including Raspberry Pi remote mode).
sn9000 — SNVNA SN9000 over VISA (pure-Python @py backend).
kamil_adc — external Kamil ADC acquisition process, with optional laser control.
mock / simulator — driver-less mode for development without hardware.

Processing modes

pass_through — magnitude and phase of the selected channel for each combo.
bscan — depth-versus-sweep B-scan heatmap.
gpr — coherent back-projection GPR imaging with object detection.
legacy_gpr — ellipse-intersection MIMO localization (point and extended/region modes).

Repository layout

Path	Contents
`data_acq_and_processing/`	C++ pipeline: `sweep_orchestrator/`, `preprocessing/`, `processing/` (including `locator/`), shared `common_cpp/` (config, IPC rings), and vendored `third_party/` (Eigen, nlohmann/json).
`python_app/`	Python application: `gui/`, `orchestration/` (process supervisor, shared-memory readers, config), `hardware_full/` (device drivers and services), `webui/`, `storage/`, `models/`, `workflows/`, `scripts/` (producers and tools), and `tests/`.
`run_config_examples/`	Ready-made configuration profiles, one per hardware setup.
`deploy/`	systemd daemon installer for Raspberry Pi appliances.
`build/bin/`	Compiled C++ binaries.
`device_firmware/`, `docs/`	Device firmware and vendor manuals.

Requirements

Linux (POSIX shared memory; GPIO on Raspberry Pi). The deployment target is a Raspberry Pi 5; hardware PWM gating for the generator sweep is Pi 5 specific (GPIO12/13/18/19, dtoverlay=pwm-2chan).
A C++20 compiler and make — Eigen and nlohmann/json are vendored, so no extra C++ dependencies are needed.
Python 3.10 or newer, in a virtual environment.
For the sar3d processor (3D SAR): an NVIDIA GPU with the driver installed (libcuda) and the external gpr_cpp library from the VNA_AlGo project. It must be built before make (see below); its location is configurable via the GPR_CPP_DIR make variable (default /home/awe/Documents/VNA_AlGo/gpr_cpp).

Build and run

From the repository root:

python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
# 3D SAR library (external, CMake + NVRTC; CUDA headers come from pip nvidia-* packages):
bash /home/awe/Documents/VNA_AlGo/gpr_cpp/build_cmake.sh
make                     # build the C++ binaries into build/bin/

To verify the sar3d processor against the VNA_AlGo reference on the recorded dataset, build and run the on-demand parity tool:

make sar3d_parity
./build/bin/sar3d_parity <preprocessed_dataset_dir> <center_snapshot_name>
# compare the printed peak/argmax with gpr_cpp's ./build/standalone_bp on the same arguments

Run the desktop GUI:

.venv/bin/python -m python_app.gui.main
# or, with the build step and single-instance handling:
./start.sh

start.sh supports --headless (offscreen, auto-start — for unattended Pi use), --profile PATH, --producer-only, --skip-build, and --clean-shm; see ./start.sh --help. The acquisition device (LibreVNA, Kamil ADC, …) is detected automatically from the active config's radar.model — there is no per-device flag.

Running without hardware (emulation)

Two ways to run the full pipeline with no radar attached:

Mock producer (synthetic data). The default run_config.json already sets radar.driver_mode: "mock", so the producer generates synthetic S-parameters. The preprocessor still requires S21 calibration/reference sets; create them once:

.venv/bin/python -m python_app.scripts.manual_smoke_run --duration 1.0  # creates smoke_cal/smoke_ref
./start.sh   # then in the GUI: Preprocessing Panel -> pick the sets -> Start

(Alternatively use the GUI's "Create neutral sets" button, which makes identity calibration + zero reference so preprocessing is a pass-through.)

Dataset replay (recorded scene, meaningful sar3d picture). When radar.replay.dataset_dir is set in the profile, the supervisor launches python_app.scripts.replay_raw_producer instead of the normal producer: it replays recorded snapshots into the raw ring at the recorded cadence, looping forever with a monotonic timeline. Pair it with NEUTRAL preprocess sets so the already-preprocessed recording passes through the preprocessor bit-exactly. A ready profile for the VNA_AlGo tank005 dataset is provided:

# one-time: neutral sets for this profile (or use the GUI button)
.venv/bin/python -m python_app.scripts.create_neutral_preprocess_sets \
    --config run_config_examples/run_config_replay_vna.example.json
# interactive GUI (mode sar3d is pre-selected by the profile's gui section):
./start.sh --profile run_config_examples/run_config_replay_vna.example.json
# or a fully headless end-to-end check:
.venv/bin/python -m python_app.scripts.replay_smoke_run --duration 30

Note: the sar3d processor allocates GPU memory; if the desktop (e.g. a browser) has filled the GPU, sar3d logs out of memory and publishes empty results until memory frees. Reduce sar3d_grid_nx/ny/nz in the processing panel to shrink the footprint.

The web interface is served in-process at http://<host>:8080. It mirrors the live plot and offers Start / Stop / Capture controls and the active processor settings.

Configuration

The active profile is run_config.json. It selects the radar model and sweep, the switch matrix and acquisition combos, the preprocessing calibration/reference assets, the GPR geometry, and the shared-memory ring sizes. Profiles for each hardware setup are provided in run_config_examples/ — copy one to run_config.json, or pass it with --profile, to use it.

Deployment

On a Raspberry Pi, deploy/install-daemon.sh installs a systemd service that runs the application headless and restarts it on failure. A single-instance lock ensures the interactive GUI and the daemon never run at the same time, since they share the radar, the shared-memory rings, and the locator port.

Tests

.venv/bin/python -m unittest discover -s python_app/tests

8.3 KiB Raw Blame History