Implement row-averaged FFT feature for same-row SNR cleanup
Add row-averaged FFT feature with configurable window size (row_avg_n parameter) for improved signal-to-noise ratio on noisy scans. Includes: - Gaussian-weighted same-row neighbor averaging (never crosses rows) - Masked/renormalized convolution handling for edge cases and masked samples - Cache format v2 with row_avg_n tracking to prevent silent cache mismatches - GUI dialog option for row-average window configuration - Comprehensive tests validating kernel properties, background subtraction invariance, and cache dispatch Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
+39
-8
@@ -220,7 +220,7 @@ actions.
|
||||
| Offset | Size | Type | Field | Description |
|
||||
|--------|------|------|-------|-------------|
|
||||
| 0 | 4 | `char[4]` | `cach_magic` | `CACH` (ASCII). Missing/wrong magic → treat file as having no cache. |
|
||||
| 4 | 1 | `u8` | `cach_version` | Cache format version. Currently `1`. Readers must treat the file as uncached if this is not a version they understand (unlike v5's `PREC` section, which read but never validated its version byte). |
|
||||
| 4 | 1 | `u8` | `cach_version` | Cache format version. Currently `2`; readers also accept `1` (a `1` tail predates row-averaged FFT caching — see the `SFFT` block below and [CACH tail version history](#cach-tail-version-history)). Any other value → treat the file as uncached (unlike v5's `PREC` section, which read but never validated its version byte). |
|
||||
| 5 | 1 | `u8` | `block_flags` | Bit 0 = DC block (`SDCB`) follows. Bit 1 = FFT block (`SFFT`) follows, immediately after the DC block if both are present. Bits 2–7 reserved, must be zero on write. |
|
||||
|
||||
### DC block `SDCB` (present iff `block_flags & 0x01`)
|
||||
@@ -249,13 +249,20 @@ value, same as v5's `PREC` section.
|
||||
|
||||
### FFT block `SFFT` (present iff `block_flags & 0x02`)
|
||||
|
||||
7-byte block header, format `">4sBH"`:
|
||||
Block header layout depends on `cach_version`:
|
||||
|
||||
- **`cach_version` 1**: 7 bytes, format `">4sBH"` — magic, flags, n_stored.
|
||||
- **`cach_version` 2**: 8 bytes, format `">4sBHB"` — magic, flags, n_stored,
|
||||
`row_avg_n`. Always written by current code; a `cach_version` 1 tail (no
|
||||
trailing byte) is still read, with `row_avg_n` taken as `0` for every
|
||||
entry it stores.
|
||||
|
||||
| Offset (rel) | Size | Type | Field | Description |
|
||||
|--------------|------|------|-------|-------------|
|
||||
| 0 | 4 | `char[4]` | `magic` | `SFFT` |
|
||||
| 4 | 1 | `u8` | `flags` | Bit 0 = `bg_sub_applied` — background waveform was subtracted from CH1 before the FFT when these images were computed. Bits 1–7 reserved. |
|
||||
| 4 | 1 | `u8` | `flags` | Bit 0 = `bg_sub_applied` — background waveform was subtracted from CH1 before the FFT when these images were computed. Bit 1 = `row_averaged` — `peak_freq_mhz` came from same-row, distance-weighted averaged CH1 waveforms rather than raw per-pixel ones; `row_avg_n` (below) is the neighbor half-width used. Bits 2–7 reserved. |
|
||||
| 5 | 2 | `u16` | `n_stored` | Number of angle entries that follow |
|
||||
| 7 | 1 | `u8` | `row_avg_n` | *`cach_version` 2 only.* Same-row neighbor half-width, in pixels, that `peak_freq_mhz` was averaged over before its FFT; `0` = raw (unaveraged). Meaningful only when `flags` bit 1 is set — a `cach_version` 1 tail has no such byte and is always `row_avg_n = 0`. |
|
||||
|
||||
followed by `n_stored` entries, each:
|
||||
|
||||
@@ -264,9 +271,10 @@ u16 angle_idx — index into the angle table (0-ba
|
||||
f32[n_rows[angle_idx] × n_frames[angle_idx]] peak_freq_mhz — CH1 FFT peak frequency, MHz, row-major
|
||||
```
|
||||
|
||||
**`peak_freq_mhz`** is computed without any DC-threshold masking (i.e. the
|
||||
FFT is run on every pixel unconditionally, same as v5's `PREC` convention).
|
||||
Readers apply the DC4 threshold at display time:
|
||||
**`peak_freq_mhz`** for a raw store (`row_avg_n == 0`) is computed without
|
||||
any DC-threshold masking (i.e. the FFT is run on every pixel
|
||||
unconditionally, same as v5's `PREC` convention). Readers apply the DC4
|
||||
threshold at display time:
|
||||
|
||||
```
|
||||
pixel is valid ⟺ dc4_mv[r][f] ≥ threshold_mv
|
||||
@@ -276,10 +284,22 @@ display_value = peak_freq_mhz[r][f] if valid, else 0
|
||||
using the DC4 image from the DC block if that angle is also cached there,
|
||||
else computed on demand.
|
||||
|
||||
For a row-averaged store (`row_avg_n > 0`), the DC4 threshold is applied
|
||||
*during* the store — a pixel below threshold is left at `0` and never
|
||||
contributes to any neighbor's average — since neighbor validity can't be
|
||||
deferred to display time the way plain masking can. The threshold value
|
||||
itself is not recorded, only that averaging happened and at what window
|
||||
size. Readers still apply their own live DC4 threshold at display time
|
||||
exactly as for a raw store, using whatever mask they currently have.
|
||||
|
||||
Readers must fall back to real-time FFT computation (ignoring stored
|
||||
`peak_freq_mhz`) under the same conditions as v5's PREC fast path: time-domain
|
||||
gating is active, zero-padding (`n_fft ≠ samples_per_frame`) is requested, or
|
||||
the reader's background-subtraction setting doesn't match `flags.bg_sub_applied`.
|
||||
gating is active, zero-padding (`n_fft ≠ samples_per_frame`) is requested,
|
||||
the reader's background-subtraction setting doesn't match
|
||||
`flags.bg_sub_applied`, or the reader's requested `row_avg_n` doesn't match
|
||||
the stored value exactly — a raw request must never be served a
|
||||
row-averaged store, or vice versa, and a request at one window size must
|
||||
never be served a store at another.
|
||||
|
||||
### In-place write ordering
|
||||
|
||||
@@ -294,6 +314,17 @@ interrupted write leaves harmless trailing bytes rather than a corrupt file,
|
||||
and the next successful write overwrites them via the same deterministic
|
||||
`cache_offset`.
|
||||
|
||||
### CACH tail version history
|
||||
|
||||
Distinct from the outer `.sras` file `version` byte (top of this document),
|
||||
which has stayed `7` since the Cache Tail was introduced — this is the inner
|
||||
`cach_version` byte inside the `CACH` header itself.
|
||||
|
||||
| cach_version | Change |
|
||||
|--------------|--------|
|
||||
| 1 | Initial Cache Tail: `SDCB` (DC) and `SFFT` (FFT, 7-byte header) blocks. |
|
||||
| 2 | `SFFT` header grows one byte, `row_avg_n` — the same-row neighbor half-width the stored `peak_freq_mhz` was averaged over before its FFT, `0` = raw. Readers still accept a `cach_version` 1 tail, treated as `row_avg_n = 0` for every angle it stores, so files cached before this change keep working without a recompute. |
|
||||
|
||||
---
|
||||
|
||||
## Acquisition Settings (fixed by sc3_aui_app.py)
|
||||
|
||||
Reference in New Issue
Block a user