Conversion of code to a program-based system where the current ethernet stack acts as one "program"

[iamdarshg]

Overview

Convert this repository from a single “software-defined radar stack” into a profile-driven multi-target “program” where you can swap the hardware behavior at build time (e.g., Radar vs QAM Tx), and flashing FPGA + RP2040 is one seamless command.

Target architecture

• Profile = a build-time selection (no runtime swapping required): radar, qam_tx, qam_rx (later), cw_tone, loopback_test, etc.
• Each profile selects:
• FPGA “application” core (DSP chain) + a shared “platform” wrapper (clocks, ADC/DAC I/O, FIFOs/DMA, control bus)
• RP2040 firmware role (board control + control-plane gateway + optional streaming assist)
• Host pipeline (Python + optional GNU Radio/ExtIO integration)
• Key rule: keep a stable “platform contract” and only swap “application cores” + “profile config”.

Repo layout (proposed)

This repo already has distinct areas for HDL + software (e.g., verilog/, sdr_fpga_task/, wideband-sdr-software/, plus gr-wideband_sdr/ and extio_plugin/).
The Python host-side stack appears to live in wideband-sdr-software/ (controllers + DSP utilities + setup.py).

• /profiles/
• radar.yml
• qam_tx.yml
• loopback.yml
• /fpga/
• /platform/ (fixed): clocks/resets, ADC/DAC interface, sample CDC, FIFO/DMA, register bus, timing/trigger primitives
• /apps/ (swappable): radar/, qam_tx/, test/
• /build/ (generated): bitstreams per profile
• /mcu/ (RP2040)
• /platform/ (fixed): USB, transport, flash/update helpers, config storage, board drivers (SPI/I2C/GPIO), timebase helpers
• /apps/ (swappable): radar_ctrl/, qam_tx_ctrl/, factory_test/
• /build/ (generated): UF2 per profile
• /host/
• /codesdr/ python package (CLI + profile system + common control-plane + streaming)
• /plugins/ optional adapters for GNU Radio (gr-wideband_sdr) and ExtIO (extio_plugin)
• /common/
• protocol/ (control-plane messages + versioning)
• regmap/ (single source of truth; autogenerate C headers + Python bindings)
• dsp/ shared math, framing formats, IQ packing, etc.
• /tools/
• codesdr CLI entrypoint (build/flash/run)
• flash_fpga.*, flash_rp2040.*, bundle_profile.*
• /ci/ + /tests/

Integrated build/flash (one command)

• Add a top-level “profile bundling” artifact: build/<profile>/bundle/ containing:
• fpga.bit (or .bin)
• rp2040.uf2
• manifest.json (git sha, build timestamp, profile id, expected regmap version, expected FPGA app id)
• codesdr flash --profile <name> does:

1. Build FPGA bitstream for <name>
2. Build RP2040 UF2 for <name>
3. Program FPGA
4. Put RP2040 into bootloader + flash UF2
5. Post-flash verify: read RP2040 firmware version + query FPGA app id/regmap hash, fail fast if mismatch

• Make “host run” symmetrical: codesdr run --profile <name> ... loads the correct host pipeline + validates connected hardware reports the same profile id.

TODO checklist

• Define “Profile” spec (YAML/JSON): profile_id, fpga_app, mcu_app, sample rates, center freq constraints, stream format, required peripherals, build flags.

• Create a manifest.json schema + versioning policy (include regmap_version, fpga_app_id, mcu_fw_version, git sha).

• Decide the single “truth” for regmaps and protocol (recommend: generate both C + Python from one description).

• Create common/protocol/ message format (CBOR/protobuf/custom TLV): GET_INFO, SET_PARAM, ARM, TRIGGER, START_STREAM, STOP_STREAM, READ_REG, WRITE_REG, HEALTH, ERROR.

• Define a strict compatibility contract: host refuses to run if regmap_version mismatch or fpga_app_id unexpected for selected profile.

• FPGA: split into platform/ vs apps/:

• Implement platform wrapper: clock/reset tree, ADC/DAC sample I/O, standard AXI-stream-like internal bus (or your existing FIFO interface), debug counters, timestamp/trigger distribution.

• Implement control plane inside FPGA: memory-mapped registers (wishbone/AXI-lite/custom) + readback ID registers (FPGA_APP_ID, REGMAP_HASH, BUILD_ID).

• Implement a shared “stream formatter” block: IQ packing, framing, metadata (timestamp, overflow flags, channel id).

• Radar app:

• Preserve existing radar chain but refactor into an app core with clean ports: iq_in, iq_out (optional), ctrl, trigger, timestamp.

• Move radar-only registers into an app-specific register region.

• QAM Tx app (first new non-radar target):

• DSP chain definition: symbol source (PRBS/file/host), mapper (e.g., 16-QAM), pulse shaping (RRC), interpolation, NCO/upconversion, DAC formatting.

• Add app registers: constellation order, symbol rate, roll-off, NCO freq, amplitude scaling, test pattern selection.

• Provide at least one “known-good” loopback mode for validation (e.g., internal DDS/QAM generator to TX path).

• Add simulation hooks: golden vectors for mapper/RRC/NCO; basic testbench (cocotb or your current sim flow).

• RP2040: restructure into platform + apps:

• Platform: USB transport (CDC/WinUSB), packet parser, command dispatcher, error handling, logging, firmware version endpoint.

• Platform: board drivers (GPIO/SPI/I2C) for RF front-end control (LNA/attenuator/antenna switch/PLL/PA enable as applicable).

• Platform: FPGA control bridge (SPI/QSPI/PIO parallel/etc): read_reg/write_reg, reset FPGA, read FPGA IDs, stream control if RP2040 participates.

• App: radar_ctrl:

• Expose radar-specific high-level commands (arm/trigger settings/PRI config) that translate to FPGA reg writes.

• App: qam_tx_ctrl:

• Expose QAM Tx configuration commands + safe TX enable sequencing (PA enable last; disable first).

• Add nonvolatile config storage policy (optional): calibration blobs per profile, board serial, last-known-good profile.

• Add a “factory test” app/profile: blink + basic SPI/I2C enumeration + FPGA reg read/write sanity.

• Host software: make a real “program” entrypoint:

• Create /host/codesdr/ python package with a CLI:

• codesdr list-profiles

• codesdr build --profile <p> [--fpga-only|--mcu-only]

• codesdr flash --profile <p> [--verify]

• codesdr run --profile <p> [args...]

• Implement profile loader that selects:

• Control-plane mapping (which commands exist)

• Streaming format decoder/encoder (IQ framing)

• Optional DSP pipeline stage(s) per profile

• Implement “device discovery” and “sanity handshake”:

• Read RP2040 fw version, board id

• Read FPGA app id/regmap hash

• Confirm matches selected profile’s manifest

• Integrate existing Python components (from wideband-sdr-software/) behind stable interfaces:

• FrontEndController (LNA/antenna/calibration hooks)

• StreamSource (USB/Ethernet/file)

• DSPChain (DDC/DC offset/etc)

• Add adapters so GNU Radio (gr-wideband_sdr/) and ExtIO (extio_plugin/) can consume the same stream + control-plane without duplicating logic.

• Seamless “two-target flashing”:

• Define how the FPGA is programmed (cable/programmer/tool choice) and wrap it in codesdr flash with clear error reporting.

• Implement RP2040 flash automation (auto-reset into BOOTSEL via DTR/RTS or a dedicated GPIO if wired; fallback instructions if manual).

• Add post-flash verification steps (must fail if wrong image on either target).

• Add “bundle build” output: one folder/zip per profile containing bitstream+UF2+manifest.

• CI + tests:

• Add unit tests for: profile parsing, manifest validation, protocol encoding/decoding, regmap generation.

• Add “hardware-in-the-loop” hooks (optional): a CI job that can be run locally to validate flashing + handshake on a connected device.

• Add regression tests for QAM Tx numerics (EVM/SNR sanity with a reference capture, even if manual initially).

• Documentation:

• docs/ARCHITECTURE.md: platform vs app boundary (FPGA/RP2040/host responsibilities).

• docs/PROFILES.md: how to add a new hardware behavior end-to-end.

• docs/FLASHING.md: one-command flow + troubleshooting.

• docs/REGMAP.md: how regmaps are generated + versioning rules.

• Acceptance criteria:

• codesdr flash --profile radar and codesdr flash --profile qam_tx both work end-to-end (build + program FPGA + program RP2040 + verify).

• After flashing, codesdr run --profile <name> rejects mismatched hardware images (wrong app id or regmap hash).

• QAM Tx profile generates a measurable, stable constellation (at least via a reference receiver/capture path) and has a deterministic test pattern mode.

• Radar profile continues to function with no worse performance than current baseline (documented with a before/after test).

[Amitk003]

assign it to me

[iamdarshg]

I've assigned you for now, create a branch and tell me if you need elaboration on the hardware integration, I will let the issue remain open for other collaborators to help since this is quite a large pivot