tka/sras-viewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement FFT pad-factor caching and the stored-cache display fast path

Browse Source 
tests/test_stored_cache.py exercised two features that were never built,
so six of its tests had been failing on main. Both are now implemented.

Pad-factor caching. A padded view could not use a stored FFT cache at all:
the store was pad 1 by definition and cached_rf_image rejected any n_fft
outright, so a user working at a pad factor got nothing from batch-computing
a file. CACH tail v3 records the pad the images were resolved at, cache_file
computes at a requested pad, and the batch actions pass the viewer's own pad
down — while still refusing a store resolved at a different pad, since a
padded FFT interpolates between the natural bins and so resolves genuinely
different peak frequencies. v1/v2 tails read as pad 1 and keep working.

Stored-cache dispatch. _refresh_display only ever consulted this window's
in-session dicts, so after a batch every angle change still queued a worker
and a progress popup for an image already on disk — the exact cost the batch
was run to avoid. It now checks the file's own DC/FFT blocks first, asking
with allow_dc_recompute=False so the GUI thread never touches I/O. When the
stored cache genuinely cannot serve the view, the scan info panel says why
rather than leaving the silent recompute a mystery.

Two bugs surfaced on the way:

- The angle spinbox was wired on editingFinished, which QAbstractSpinBox
  emits only on Return or focus-out — never on a step. Clicking its arrows,
  the ordinary way to walk a scan, moved the number and left the image
  behind. Now valueChanged with keyboard tracking off, which fires on a step
  and once on commit, but not per keystroke mid-typing. Every existing GUI
  test called _on_view_changed() by hand and so could not have caught this;
  two new tests pin it and fail against the old wiring.

- A plain "fft" batch over a file previously cached with fft_rowavg carried
  the old row_avg_n forward, labelling raw images as row-averaged. It now
  writes row_avg_n=0 explicitly.

Verified: 111 passed (was 103 passed / 6 failed), and
tools/check_equivalence.py is byte-identical to the pre-change baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Thomas Ales
2026-08-09 13:50:37 -05:00
parent f40c965b74
commit 8348ad313c
7 changed files with 358 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sras_format.py
+56 -20
View File
@@ -60,11 +60,15 @@ PREC_FLAG_BG_SUB = 0x01
CACH_MAGIC = b"CACH"
CACH_HDR_FMT = ">4sBB" # magic, cach_version, block_flags
CACH_HDR_SIZE = struct.calcsize(CACH_HDR_FMT)
CACH_VERSION = 2 # written on every fresh write
CACH_VERSIONS_READABLE = (1, 2) # accepted on read — see
# _read_sfft_block: a v1 tail
# predates row-averaged FFT
# caching and reads as row_avg_n=0
CACH_VERSION = 3 # written on every fresh write
CACH_VERSIONS_READABLE = (1, 2, 3) # accepted on read — see
# _read_sfft_block. Each bump only
# appended a field, and every older
# tail has a well-defined reading:
# v1 predates row-averaged FFT
# caching (row_avg_n=0) and v1/v2
# predate padded caching, so both
# are natural-resolution (pad 1).
CACH_FLAG_DC = 0x01
CACH_FLAG_FFT = 0x02
@@ -74,9 +78,12 @@ SDCB_HDR_SIZE = struct.calcsize(SDCB_HDR_FMT)
SFFT_MAGIC = b"SFFT"
SFFT_HDR_FMT_V1 = ">4sBH" # magic, flags, n_stored (cach_version 1)
SFFT_HDR_FMT = ">4sBHB" # + row_avg_n (cach_version 2)
SFFT_HDR_FMT_V2 = ">4sBHB" # + row_avg_n (cach_version 2)
SFFT_HDR_FMT = ">4sBHBH" # + pad_factor (cach_version 3)
SFFT_HDR_SIZE_V1 = struct.calcsize(SFFT_HDR_FMT_V1)
SFFT_HDR_SIZE_V2 = struct.calcsize(SFFT_HDR_FMT_V2)
SFFT_HDR_SIZE = struct.calcsize(SFFT_HDR_FMT)
MAX_PAD_FACTOR = 0xFFFF # the H field above
SFFT_FLAG_BG_SUB = 0x01
SFFT_FLAG_ROW_AVG = 0x02 # peak_freq_mhz came from same-row,
# distance-weighted averaged CH1
@@ -180,7 +187,10 @@ class SrasFile:
``precomputed_dc3_mv`` / ``precomputed_dc4_mv`` / ``precomputed_freq_mhz``,
always as ragged per-angle lists (``list[np.ndarray | None]``, one entry
per angle, ``None`` where that angle was never stored) regardless of
source version.
source version. The scalars ``precomputed_bg_sub`` /
``precomputed_row_avg_n`` / ``precomputed_pad_factor`` record the
settings the stored FFT images were computed under, so a reader can tell
whether they answer the question it is actually asking.
"""
def __init__(self, path: str):
@@ -238,6 +248,11 @@ class SrasFile:
self.precomputed_dc3_mv: list[np.ndarray | None] = [None] * n_angles
self.precomputed_bg_sub: bool = False
self.precomputed_row_avg_n: int = 0
# Zero-padding factor the stored peak_freq_mhz images were resolved
# at: 1 = natural resolution (n_fft == samples_per_frame). A padded
# FFT resolves peaks a padded view would, and only such a view can
# be served from it — see sras_compute.cached_rf_image.
self.precomputed_pad_factor: int = 1
def cached_dc_mv(self, angle_idx: int, ch_idx: int) -> np.ndarray | None:
"""A stored DC image (already in mV) for (angle, channel), or None."""
@@ -529,24 +544,30 @@ class SrasFile:
store[angle_idx] = _read_f32_image(f, shape)
return flags
def _read_sfft_block(self, f, cach_version: int) -> tuple[int, int] | None:
def _read_sfft_block(self, f, cach_version: int) -> tuple[int, int, int] | None:
"""Read the SFFT block header — its layout depends on cach_version,
since v2 appended a trailing row_avg_n byte — then n_stored per-angle
peak_freq_mhz entries (unchanged across versions).
since each bump appended a trailing field (v2 row_avg_n, v3
pad_factor) — then n_stored per-angle peak_freq_mhz entries
(unchanged across versions).
Returns (flags, row_avg_n), or None if the block is malformed.
row_avg_n is always 0 for a v1 tail, which predates row-averaged FFT
caching entirely.
Returns (flags, row_avg_n, pad_factor), or None if the block is
malformed. The absent fields of an older tail take the value that
describes what such a tail can only have been: row_avg_n=0 for v1,
which predates row-averaged FFT caching, and pad_factor=1 for v1/v2,
which predate padded caching and so are natural-resolution.
"""
hdr_fmt = SFFT_HDR_FMT_V1 if cach_version == 1 else SFFT_HDR_FMT
hdr_fmt = {1: SFFT_HDR_FMT_V1, 2: SFFT_HDR_FMT_V2}.get(
cach_version, SFFT_HDR_FMT)
raw = f.read(struct.calcsize(hdr_fmt))
if len(raw) < struct.calcsize(hdr_fmt):
return None
row_avg_n, pad_factor = 0, 1
if cach_version == 1:
magic, flags, n_stored = struct.unpack(hdr_fmt, raw)
row_avg_n = 0
else:
elif cach_version == 2:
magic, flags, n_stored, row_avg_n = struct.unpack(hdr_fmt, raw)
else:
magic, flags, n_stored, row_avg_n, pad_factor = struct.unpack(hdr_fmt, raw)
if magic != SFFT_MAGIC:
return None
for _ in range(n_stored):
@@ -555,7 +576,7 @@ class SrasFile:
break
self.precomputed_freq_mhz[angle_idx] = _read_f32_image(
f, self.image_shape(angle_idx))
return flags, row_avg_n
return flags, row_avg_n, pad_factor
def _parse_cach_section(self, offset: int):
"""Parse the v7 CACH tail that holds precomputed DC/FFT images."""
@@ -578,16 +599,18 @@ class SrasFile:
result = self._read_sfft_block(f, cach_version)
if result is None:
return
flags, row_avg_n = result
flags, row_avg_n, pad_factor = result
self.precomputed_bg_sub = bool(flags & SFFT_FLAG_BG_SUB)
self.precomputed_row_avg_n = row_avg_n if (flags & SFFT_FLAG_ROW_AVG) else 0
self.precomputed_pad_factor = max(1, pad_factor)
def write_v7_cache(self, *,
new_dc3_mv: list[np.ndarray | None] | None = None,
new_dc4_mv: list[np.ndarray | None] | None = None,
new_freq_mhz: list[np.ndarray | None] | None = None,
new_bg_sub: bool | None = None,
new_row_avg_n: int | None = None):
new_row_avg_n: int | None = None,
new_pad_factor: int | None = None):
"""Store computed DC and/or FFT images into this file's CACH tail,
in place, converting a v6 source to v7 (or updating an existing v7
file). Only the block(s) passed in are recomputed; whichever block
@@ -601,6 +624,12 @@ class SrasFile:
It describes the whole stored FFT block, not per-angle, mirroring
how bg-sub has never been tracked per-angle either.
*new_pad_factor* is the zero-padding factor the passed *new_freq_mhz*
was resolved at (1 = natural resolution), carried forward the same
way. Like row_avg_n it is provenance, not a hint: a view at a
different pad resolves different peaks, so recording it is what lets
a reader refuse the cache instead of showing the wrong numbers.
The waveform data itself is never touched: the cache tail always
starts at ``_cache_tail_offset()``, a fixed offset derived from the
header and geometry table alone.
@@ -615,8 +644,13 @@ class SrasFile:
final_bg_sub = new_bg_sub if new_bg_sub is not None else self.precomputed_bg_sub
final_row_avg_n = (new_row_avg_n if new_row_avg_n is not None
else self.precomputed_row_avg_n)
final_pad_factor = (new_pad_factor if new_pad_factor is not None
else self.precomputed_pad_factor)
if not (0 <= final_row_avg_n <= 255):
raise ValueError(f"row_avg_n must fit in a byte (0-255), got {final_row_avg_n}")
if not (1 <= final_pad_factor <= MAX_PAD_FACTOR):
raise ValueError(
f"pad_factor must be 1-{MAX_PAD_FACTOR}, got {final_pad_factor}")
dc_entries = [a for a in range(self.n_angles) if final_dc3[a] is not None]
fft_entries = [a for a in range(self.n_angles) if final_freq[a] is not None]
@@ -638,7 +672,8 @@ class SrasFile:
fft_flags = SFFT_FLAG_BG_SUB if final_bg_sub else 0
fft_flags |= SFFT_FLAG_ROW_AVG if final_row_avg_n else 0
payload += struct.pack(SFFT_HDR_FMT, SFFT_MAGIC, fft_flags,
len(fft_entries), final_row_avg_n)
len(fft_entries), final_row_avg_n,
final_pad_factor)
for a in fft_entries:
payload += struct.pack(">H", a)
payload += final_freq[a].astype(">f4").tobytes()
@@ -667,6 +702,7 @@ class SrasFile:
self.precomputed_freq_mhz = final_freq
self.precomputed_bg_sub = final_bg_sub
self.precomputed_row_avg_n = final_row_avg_n
self.precomputed_pad_factor = final_pad_factor
# ------------------------------------------------------------------
# Axes helpers