Radar System

A real-time radar acquisition and imaging system for vector-network-analyzer (VNA) and ADC front-ends. It drives the radar through a frequency sweep, calibrates and preprocesses the measured S-parameters, runs a selectable imaging/detection processor, and presents the live result in a desktop GUI and an in-process web interface. The same application runs interactively on a workstation or unattended as a headless Raspberry Pi appliance.

Overview

The system is split into a C++ real-time pipeline and a Python application that supervises it and handles visualization and control.

The pipeline runs as three independent processes connected by lock-free POSIX shared-memory ring buffers. Each stage advances on its own, and a slow consumer never blocks a producer — the rings are latest-wins, so a reader that falls behind simply skips to the freshest frame:

1. Acquisition — sweep_orchestrator (or a hardware-specific producer) configures the radar, performs the frequency sweep, and publishes raw S11/S21 traces.
2. Preprocessing — data_preprocessor applies through/OSL calibration, reference subtraction, and optional frequency-domain notch filtering.
3. Processing — data_processor runs the selected processing mode and emits results (pass-through traces, B-scan images, or GPR images with detected objects). Detected objects are additionally broadcast to subscribers over a TCP locator server.

The Python application (python_app) starts and monitors the C++ binaries, reads the result rings, renders the live view with pyqtgraph, and exposes the same controls plus a pixel-identical live plot through an embedded web UI.

 radar hardware
       │
       ▼
 Acquisition ──[raw ring]──▶ Preprocessing ──[preprocessed ring]──▶ Processing ──[result ring]──▶ GUI / Web
                                                                          │
                                                                          └──▶ TCP locator (detected objects)

Supported hardware

• librevna — single LibreVNA.
• librevna_multi — synchronized multi-device LibreVNA.
• compact_m_k209 — Compact-M K209 / S2VNA (including Raspberry Pi remote mode).
• sn9000 — SNVNA SN9000 over VISA (pure-Python @py backend).
• kamil_adc — external Kamil ADC acquisition process, with optional laser control.
• mock / simulator — driver-less mode for development without hardware.

Processing modes

• pass_through — magnitude and phase of the selected channel for each combo.
• bscan — depth-versus-sweep B-scan heatmap.
• gpr — coherent back-projection GPR imaging with object detection.
• legacy_gpr — ellipse-intersection MIMO localization (point and extended/region modes).

Repository layout

Path	Contents
data_acq_and_processing/	C++ pipeline: sweep_orchestrator/, preprocessing/, processing/ (including locator/), shared common_cpp/ (config, IPC rings), the vendored gpr_cpp/ C++/CUDA 3D SAR library, and vendored third_party/ (Eigen, nlohmann/json).
python_app/	Python application: gui/, orchestration/ (process supervisor, shared-memory readers, config), hardware_full/ (device drivers and services), webui/, storage/, models/, workflows/, scripts/ (producers and tools), and tests/.
run_config_examples/	Ready-made configuration profiles, one per hardware setup.
deploy/	systemd daemon installer for Raspberry Pi appliances.
build/bin/	Compiled C++ binaries.
device_firmware/, docs/	Device firmware and vendor manuals.

Requirements

• Linux (POSIX shared memory; GPIO on Raspberry Pi). The deployment target is a Raspberry Pi 5; hardware PWM gating for the generator sweep is Pi 5 specific (GPIO12/13/18/19, dtoverlay=pwm-2chan).
• A C++20 compiler and make — Eigen and nlohmann/json are vendored, so no extra C++ dependencies are needed.
• Python 3.10 or newer, in a virtual environment.
• For the sar3d processor (3D SAR): an NVIDIA GPU with the driver installed (libcuda) and cmake (the vendored gpr_cpp library at data_acq_and_processing/gpr_cpp/ builds via its own CMake, invoked automatically by make). CUDA headers/libnvrtc are taken from the pip nvidia-* packages in this repo's venv (installed by requirements.txt on x86_64), otherwise from the system toolkit at /usr/local/cuda (Jetson/JetPack, apt cuda-toolkit).

Build and run

From the repository root:

python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
make                     # build the C++ binaries into build/bin/

make also builds the vendored 3D SAR library (data_acq_and_processing/gpr_cpp/, CMake + NVRTC — needs cmake; CUDA headers come from the pip nvidia-* packages in .venv, or /usr/local/cuda as a fallback).

To verify the sar3d processor against the gpr_cpp reference on the recorded dataset, build and run the on-demand parity tool:

make sar3d_parity
./build/bin/sar3d_parity <preprocessed_dataset_dir> <center_snapshot_name>
# compare the printed peak/argmax with
# ./data_acq_and_processing/gpr_cpp/build/standalone_bp on the same arguments

Run the desktop GUI:

.venv/bin/python -m python_app.gui.main
# or, with the build step and single-instance handling:
./start.sh

start.sh supports --headless (offscreen, auto-start — for unattended Pi use), --profile PATH, --producer-only, --skip-build, and --clean-shm; see ./start.sh --help. The acquisition device (LibreVNA, Kamil ADC, …) is detected automatically from the active config's radar.model — there is no per-device flag.

Jetson (aarch64) bring-up

Ubuntu 22.04 on Jetson has glibc 2.35, and PyQt6 6.8+ ships only manylinux_2_39 aarch64 wheels — requirements.txt therefore pins PyQt6 to <6.8 on aarch64 (6.7.1 is the last release with a compatible wheel). With that pin the stock ./start.sh provisioning works; the extra steps are only for the CUDA-backed sar3d processor:

# 1. copy the replay dataset (for dataset-replay profiles) and point
#    radar.replay.dataset_dir in the profile at the local copy — the path in
#    run_config_examples/*.json is machine-specific.

# 2. normal launch; the vendored gpr_cpp builds against the JetPack system CUDA
#    (requirements.txt installs no pip nvidia-* packages on aarch64, and
#    build_cmake.sh falls back to /usr/local/cuda automatically; override with
#    CUDA_INC=... CUDA_LIB=... if the toolkit lives elsewhere):
./start.sh --profile run_config_examples/run_config_replay_vna.example.json

If linking fails with cannot find -lcuda, the driver stub lives at /usr/local/cuda/lib64/stubs/libcuda.so; on JetPack the real driver is /usr/lib/aarch64-linux-gnu/nvidia/libcuda.so.1 (present after boot).

Running without hardware (emulation)

Two ways to run the full pipeline with no radar attached:

Mock producer (synthetic data). The default run_config.json already sets radar.driver_mode: "mock", so the producer generates synthetic S-parameters. The preprocessor still requires S21 calibration/reference sets; create them once:

.venv/bin/python -m python_app.scripts.manual_smoke_run --duration 1.0  # creates smoke_cal/smoke_ref
./start.sh   # then in the GUI: Preprocessing Panel -> pick the sets -> Start

(Alternatively use the GUI's "Create neutral sets" button, which makes identity calibration + zero reference so preprocessing is a pass-through.)

Dataset replay (recorded scene, meaningful sar3d picture). When radar.replay.dataset_dir is set in the profile, the supervisor launches python_app.scripts.replay_raw_producer instead of the normal producer: it replays recorded snapshots into the raw ring at the recorded cadence, looping forever with a monotonic timeline. Pair it with NEUTRAL preprocess sets so the already-preprocessed recording passes through the preprocessor bit-exactly. A ready profile for the VNA_AlGo tank005 dataset is provided:

# one-time: neutral sets for this profile (or use the GUI button)
.venv/bin/python -m python_app.scripts.create_neutral_preprocess_sets \
    --config run_config_examples/run_config_replay_vna.example.json
# interactive GUI (mode sar3d is pre-selected by the profile's gui section):
./start.sh --profile run_config_examples/run_config_replay_vna.example.json
# or a fully headless end-to-end check:
.venv/bin/python -m python_app.scripts.replay_smoke_run --duration 30

Note: the sar3d processor allocates GPU memory; if the desktop (e.g. a browser) has filled the GPU, sar3d logs out of memory and publishes empty results until memory frees. Reduce sar3d_grid_nx/ny/nz in the processing panel to shrink the footprint.

The web interface is served in-process at http://<host>:8080. It mirrors the live plot and offers Start / Stop / Capture controls and the active processor settings.

Configuration

The active profile is run_config.json. It selects the radar model and sweep, the switch matrix and acquisition combos, the preprocessing calibration/reference assets, the GPR geometry, and the shared-memory ring sizes. Profiles for each hardware setup are provided in run_config_examples/ — copy one to run_config.json, or pass it with --profile, to use it.

Deployment

On a Raspberry Pi, deploy/install-daemon.sh installs a systemd service that runs the application headless and restarts it on failure. A single-instance lock ensures the interactive GUI and the daemon never run at the same time, since they share the radar, the shared-memory rings, and the locator port.

Tests

.venv/bin/python -m unittest discover -s python_app/tests