tka/sras-viewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement FFT pad-factor caching and the stored-cache display fast path

Browse Source 
tests/test_stored_cache.py exercised two features that were never built,
so six of its tests had been failing on main. Both are now implemented.

Pad-factor caching. A padded view could not use a stored FFT cache at all:
the store was pad 1 by definition and cached_rf_image rejected any n_fft
outright, so a user working at a pad factor got nothing from batch-computing
a file. CACH tail v3 records the pad the images were resolved at, cache_file
computes at a requested pad, and the batch actions pass the viewer's own pad
down — while still refusing a store resolved at a different pad, since a
padded FFT interpolates between the natural bins and so resolves genuinely
different peak frequencies. v1/v2 tails read as pad 1 and keep working.

Stored-cache dispatch. _refresh_display only ever consulted this window's
in-session dicts, so after a batch every angle change still queued a worker
and a progress popup for an image already on disk — the exact cost the batch
was run to avoid. It now checks the file's own DC/FFT blocks first, asking
with allow_dc_recompute=False so the GUI thread never touches I/O. When the
stored cache genuinely cannot serve the view, the scan info panel says why
rather than leaving the silent recompute a mystery.

Two bugs surfaced on the way:

- The angle spinbox was wired on editingFinished, which QAbstractSpinBox
  emits only on Return or focus-out — never on a step. Clicking its arrows,
  the ordinary way to walk a scan, moved the number and left the image
  behind. Now valueChanged with keyboard tracking off, which fires on a step
  and once on commit, but not per keystroke mid-typing. Every existing GUI
  test called _on_view_changed() by hand and so could not have caught this;
  two new tests pin it and fail against the old wiring.

- A plain "fft" batch over a file previously cached with fft_rowavg carried
  the old row_avg_n forward, labelling raw images as row-averaged. It now
  writes row_avg_n=0 explicitly.

Verified: 111 passed (was 103 passed / 6 failed), and
tools/check_equivalence.py is byte-identical to the pre-change baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Thomas Ales
2026-08-09 13:50:37 -05:00
parent f40c965b74
commit 8348ad313c
7 changed files with 358 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sras_compute.py
+62 -22
View File
@@ -18,7 +18,8 @@ import numpy as np
import scipy.fft as scipy_fft
import scipy.ndimage as scipy_ndimage
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, SrasFile, adc_to_mv
from sras_format import (CH1_IDX, CH3_IDX, CH4_IDX, MAX_PAD_FACTOR, SrasFile,
adc_to_mv)
# ---------------------------------------------------------------------------
# FFT backend
@@ -482,6 +483,23 @@ def _row_average_waveforms(masked_waves: np.ndarray, valid: np.ndarray,
return num / den_safe[:, None]
def pad_factor_for(sras: SrasFile, n_fft: int | None) -> int:
"""The integer zero-padding factor an *n_fft* request represents, or 0
if it represents none — i.e. it is not a whole multiple of this file's
samples_per_frame, so no stored image (which only ever records an
integer factor) can answer it.
0 rather than None so callers can compare it straight against
``sras.precomputed_pad_factor``, which is never 0.
"""
if n_fft is None:
return 1
spf = sras.samples_per_frame
if spf <= 0 or n_fft % spf:
return 0
return max(1, n_fft // spf)
def cached_rf_image(sras: SrasFile, angle_idx: int,
dc_threshold_mv: float | None,
apply_bg_sub: bool = True,
@@ -495,13 +513,21 @@ def cached_rf_image(sras: SrasFile, angle_idx: int,
real FFT).
A stored image stands in only when *all* hold: this angle actually has
a stored image (v5 PREC or v7 CACH); no custom zero-padding is
requested (n_fft is None) — the store is always natural-resolution;
the stored bg-sub flag matches what the caller wants; and
sras.precomputed_row_avg_n == row_avg_n exactly (0 == "raw") — this
last check is what stops a raw request from ever being silently served
a row-averaged image, or vice versa, or a request at one window size
being served a cache stored at a different one.
a stored image (v5 PREC or v7 CACH); the requested zero-padding matches
what the store was computed at (see below); the stored bg-sub flag
matches what the caller wants; and sras.precomputed_row_avg_n ==
row_avg_n exactly (0 == "raw") — this last check is what stops a raw
request from ever being silently served a row-averaged image, or vice
versa, or a request at one window size being served a cache stored at a
different one.
Padding is checked the same way, and for the same reason: a padded FFT
interpolates between the natural bins, so it resolves genuinely
different peak frequencies. *n_fft* of None means natural resolution,
i.e. pad 1; anything else must be an exact whole multiple of
samples_per_frame equal to sras.precomputed_pad_factor. A ragged n_fft
that is not such a multiple can never match a stored image, since the
store only ever records an integer pad factor.
If *allow_dc_recompute* is False and no DC4 image is already cached or
supplied via *dc4_mv*, applying the mask would mean reading a whole
@@ -511,7 +537,7 @@ def cached_rf_image(sras: SrasFile, angle_idx: int,
"""
cached_freq = sras.precomputed_freq_mhz[angle_idx]
if (cached_freq is None
or n_fft is not None
or pad_factor_for(sras, n_fft) != sras.precomputed_pad_factor
or sras.precomputed_bg_sub != (apply_bg_sub and sras.background is not None)
or sras.precomputed_row_avg_n != row_avg_n):
return None
@@ -723,6 +749,7 @@ def compute_rf_image(sras: SrasFile, angle_idx: int,
def cache_file(path: str, mode: str, apply_bg_sub: bool,
fft_backend: str = "scipy", max_workers: int = 0,
pad_factor: int = 1,
dc_threshold_mv: float | None = None,
row_avg_n: int = 0) -> str:
"""Compute and store DC or FFT images for every angle of one file,
@@ -732,21 +759,27 @@ def cache_file(path: str, mode: str, apply_bg_sub: bool,
FFT backend and worker cap are passed explicitly because module globals
do not survive a spawn.
mode is "dc", "fft" (raw per-pixel FFT, natural-resolution, unmasked —
masking applied at display time), or "fft_rowavg" (same-row,
distance-weighted CH1 averaging before the FFT — see compute_rf_image's
row_avg_n). fft_rowavg needs *dc_threshold_mv* up front, unlike plain
"fft": neighbor validity is baked into the stored numbers, so it can't
be deferred to display time the way plain masking can.
mode is "dc", "fft" (raw per-pixel FFT, unmasked — masking applied at
display time), or "fft_rowavg" (same-row, distance-weighted CH1
averaging before the FFT — see compute_rf_image's row_avg_n).
fft_rowavg needs *dc_threshold_mv* up front, unlike plain "fft":
neighbor validity is baked into the stored numbers, so it can't be
deferred to display time the way plain masking can.
The stored FFT cache is always natural-resolution (pad 1): the v7 SFFT
block records no pad factor, and padded views compute live fast enough
(see _peak_bins_zoom) that caching them is not worth a format change.
*pad_factor* is the zero-padding factor to resolve the peaks at: 1 (the
default) is natural resolution, n_fft == samples_per_frame. It is
recorded in the SFFT block so a reader knows which views the stored
numbers answer for — and it has to be the pad the viewer is *actually*
using, since a pad-1 cache is dead weight to a padded view and vice
versa (cached_rf_image refuses the mismatch rather than showing peaks
resolved at the wrong resolution).
"""
global _MAX_WORKERS
try:
if mode not in ("dc", "fft", "fft_rowavg"):
return f"unknown cache mode {mode!r} (expected 'dc', 'fft', or 'fft_rowavg')"
if not (1 <= pad_factor <= MAX_PAD_FACTOR):
return f"pad_factor must be 1-{MAX_PAD_FACTOR}, got {pad_factor}"
set_fft_backend(fft_backend)
if max_workers:
_MAX_WORKERS = max_workers
@@ -757,6 +790,7 @@ def cache_file(path: str, mode: str, apply_bg_sub: bool,
"can be batch-cached")
n = sras.n_angles
n_fft = sras.samples_per_frame * pad_factor if pad_factor > 1 else None
n_workers, angle_budget = plan_angle_level(sras)
if mode == "dc":
dc3 = _parallel_map(
@@ -778,9 +812,13 @@ def cache_file(path: str, mode: str, apply_bg_sub: bool,
# internally over blocks, so angles run one at a time with the
# full budget.
freq = [compute_rf_image(sras, a, dc_threshold_mv=None,
apply_bg_sub=effective_bg)
apply_bg_sub=effective_bg, n_fft=n_fft)
for a in range(n)]
sras.write_v7_cache(new_freq_mhz=freq, new_bg_sub=effective_bg)
# new_row_avg_n=0 explicitly: these images are raw per-pixel FFTs,
# and carrying forward a row_avg_n left by an earlier fft_rowavg
# write would label them as something they are not.
sras.write_v7_cache(new_freq_mhz=freq, new_bg_sub=effective_bg,
new_row_avg_n=0, new_pad_factor=pad_factor)
else: # "fft_rowavg"
if row_avg_n <= 0:
return "row_avg_n must be a positive neighbor half-width for fft_rowavg mode"
@@ -789,10 +827,12 @@ def cache_file(path: str, mode: str, apply_bg_sub: bool,
"(neighbor validity depends on it)")
effective_bg = apply_bg_sub and sras.background is not None
freq = [compute_rf_image(sras, a, dc_threshold_mv=dc_threshold_mv,
apply_bg_sub=effective_bg, row_avg_n=row_avg_n)
apply_bg_sub=effective_bg, n_fft=n_fft,
row_avg_n=row_avg_n)
for a in range(n)]
sras.write_v7_cache(new_freq_mhz=freq, new_bg_sub=effective_bg,
new_row_avg_n=row_avg_n)
new_row_avg_n=row_avg_n,
new_pad_factor=pad_factor)
return ""
except Exception as exc:
return str(exc)