tka/sras-viewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge the Documents/sras-viewer working copy into this repo

Browse Source 
The two clones had diverged from origin/main (191d1b8) and both had
grown uncommitted work. This merges the other clone's three commits and
reconciles four conflicting files.

Both clones independently grew a feature called "batch export", but they
are different sweeps and both are kept:
- BatchExportWorker (this clone) — every angle x channel of the one open
  file, fixed colorbar per channel, under a new Export menu.
- BatchExportImagesWorker (other clone) — the current view across many
  selected files, process-pooled, under Convert.

Conflict resolutions of note:
- sras_average.py: the other clone's memory-bounded rewrite had silently
  reverted this clone's float32 -> int16 accumulator fix, which exists to
  keep averaged output from landing 1 ADC count off the original tool.
  Kept the rewrite, re-applied the fix, and corrected the two docstrings
  that still claimed float32.
- tests/test_compute.py: kept the other clone's meta["waveforms"] key
  (meta["data"] no longer exists for this fixture and would KeyError)
  and its laser_freq_hz assertions, plus this clone's int16 expectation
  and its dropped subprocess returncode assertions.
- ComputeWorker: row_avg_n and min_freq_mhz were added independently on
  either side; both are now threaded through.
- compute_rf_image's `exact` parameter is gone, removed by the other
  clone's PyFFTW peak-search rewrite. It had no remaining callers.

Verified: 149 passed with a complete dependency set. The failures seen
with this clone's own .venv are a missing scikit-image, which predates
this merge and reproduces identically on the pre-merge commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Thomas Ales [M S E]
2026-08-12 10:36:25 -05:00
parent efecc154fe a8b962e317
commit 2937c017a7
19 changed files with 2130 additions and 617 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sras_format.py
+78 -24
View File
@@ -60,15 +60,18 @@ PREC_FLAG_BG_SUB = 0x01
CACH_MAGIC = b"CACH"
CACH_HDR_FMT = ">4sBB" # magic, cach_version, block_flags
CACH_HDR_SIZE = struct.calcsize(CACH_HDR_FMT)
CACH_VERSION = 3 # written on every fresh write
CACH_VERSIONS_READABLE = (1, 2, 3) # accepted on read — see
CACH_VERSION = 4 # written on every fresh write
CACH_VERSIONS_READABLE = (1, 2, 3, 4) # accepted on read — see
# _read_sfft_block. Each bump only
# appended a field, and every older
# tail has a well-defined reading:
# v1 predates row-averaged FFT
# caching (row_avg_n=0) and v1/v2
# caching (row_avg_n=0), v1/v2
# predate padded caching, so both
# are natural-resolution (pad 1).
# are natural-resolution (pad 1),
# and v1-v3 predate the min peak
# frequency floor (min_freq 0 =
# no floor).
CACH_FLAG_DC = 0x01
CACH_FLAG_FFT = 0x02
@@ -79,11 +82,20 @@ SDCB_HDR_SIZE = struct.calcsize(SDCB_HDR_FMT)
SFFT_MAGIC = b"SFFT"
SFFT_HDR_FMT_V1 = ">4sBH" # magic, flags, n_stored (cach_version 1)
SFFT_HDR_FMT_V2 = ">4sBHB" # + row_avg_n (cach_version 2)
SFFT_HDR_FMT = ">4sBHBH" # + pad_factor (cach_version 3)
SFFT_HDR_FMT_V3 = ">4sBHBH" # + pad_factor (cach_version 3)
SFFT_HDR_FMT = ">4sBHBHI" # + min_freq_khz (cach_version 4)
SFFT_HDR_SIZE_V1 = struct.calcsize(SFFT_HDR_FMT_V1)
SFFT_HDR_SIZE_V2 = struct.calcsize(SFFT_HDR_FMT_V2)
SFFT_HDR_SIZE_V3 = struct.calcsize(SFFT_HDR_FMT_V3)
SFFT_HDR_SIZE = struct.calcsize(SFFT_HDR_FMT)
MAX_PAD_FACTOR = 0xFFFF # the H field above
# min_freq_khz is fixed-point (u32, units of 0.001 MHz), not a float32:
# the viewer's floor spinbox has 0.001 MHz granularity, and
# round(mhz * 1000) / 1000.0 reproduces the exact float64 the user typed,
# so the accept rule in sras_compute.cache_mismatch_reasons can compare
# with plain integer ordering. A ">f" float32 of e.g. 20.1 would read back
# as 20.10000038… > 20.1 and report a spurious mismatch forever.
MAX_MIN_FREQ_KHZ = 0xFFFFFFFF # the I field above; 0 = no floor
SFFT_FLAG_BG_SUB = 0x01
SFFT_FLAG_ROW_AVG = 0x02 # peak_freq_mhz came from same-row,
# distance-weighted averaged CH1
@@ -134,6 +146,13 @@ def adc_to_mv(adc, ymult_mv: float = _FALLBACK_YMULT_MV,
return (adc - yoff_adc) * ymult_mv + yzero_mv
def _axes_extent(x_axis, y_axis, dx: float, dy: float) -> list[float]:
"""Matplotlib imshow extent with half-pixel margins, Y flipped so row 0
renders at the top."""
return [x_axis[0] - dx / 2, x_axis[-1] + dx / 2,
y_axis[-1] + dy / 2, y_axis[0] - dy / 2]
# ---------------------------------------------------------------------------
# Binary read helpers
# ---------------------------------------------------------------------------
@@ -188,9 +207,10 @@ class SrasFile:
always as ragged per-angle lists (``list[np.ndarray | None]``, one entry
per angle, ``None`` where that angle was never stored) regardless of
source version. The scalars ``precomputed_bg_sub`` /
``precomputed_row_avg_n`` / ``precomputed_pad_factor`` record the
settings the stored FFT images were computed under, so a reader can tell
whether they answer the question it is actually asking.
``precomputed_row_avg_n`` / ``precomputed_pad_factor`` /
``precomputed_min_freq_mhz`` record the settings the stored FFT images
were computed under, so a reader can tell whether they answer the
question it is actually asking.
"""
def __init__(self, path: str):
@@ -253,6 +273,13 @@ class SrasFile:
# FFT resolves peaks a padded view would, and only such a view can
# be served from it — see sras_compute.cached_rf_image.
self.precomputed_pad_factor: int = 1
# Min peak frequency floor (MHz) the stored peak search excluded
# bins below; 0.0 = no floor. Unlike bg-sub/pad/row-avg it is
# tighten-only re-applicable: a *higher* floor can be re-imposed on
# a stored image by masking pixels below it, but bins below the
# stored floor were never searched, so a lower floor needs a real
# recompute — see sras_compute.cache_mismatch_reasons.
self.precomputed_min_freq_mhz: float = 0.0
def encoded_preambles(self) -> bytes:
"""This file's Preamble Blocks section, as bytes a writer can emit.
@@ -595,30 +622,35 @@ class SrasFile:
store[angle_idx] = _read_f32_image(f, shape)
return flags
def _read_sfft_block(self, f, cach_version: int) -> tuple[int, int, int] | None:
def _read_sfft_block(self, f, cach_version: int) -> tuple[int, int, int, float] | None:
"""Read the SFFT block header — its layout depends on cach_version,
since each bump appended a trailing field (v2 row_avg_n, v3
pad_factor) — then n_stored per-angle peak_freq_mhz entries
(unchanged across versions).
pad_factor, v4 min_freq_khz) — then n_stored per-angle
peak_freq_mhz entries (unchanged across versions).
Returns (flags, row_avg_n, pad_factor), or None if the block is
malformed. The absent fields of an older tail take the value that
describes what such a tail can only have been: row_avg_n=0 for v1,
which predates row-averaged FFT caching, and pad_factor=1 for v1/v2,
which predate padded caching and so are natural-resolution.
Returns (flags, row_avg_n, pad_factor, min_freq_mhz), or None if
the block is malformed. The absent fields of an older tail take the
value that describes what such a tail can only have been:
row_avg_n=0 for v1, which predates row-averaged FFT caching;
pad_factor=1 for v1/v2, which predate padded caching and so are
natural-resolution; and min_freq_mhz=0.0 for v1-v3, which predate
the min peak frequency floor and so searched every bin above DC.
"""
hdr_fmt = {1: SFFT_HDR_FMT_V1, 2: SFFT_HDR_FMT_V2}.get(
cach_version, SFFT_HDR_FMT)
hdr_fmt = {1: SFFT_HDR_FMT_V1, 2: SFFT_HDR_FMT_V2,
3: SFFT_HDR_FMT_V3}.get(cach_version, SFFT_HDR_FMT)
raw = f.read(struct.calcsize(hdr_fmt))
if len(raw) < struct.calcsize(hdr_fmt):
return None
row_avg_n, pad_factor = 0, 1
row_avg_n, pad_factor, min_freq_khz = 0, 1, 0
if cach_version == 1:
magic, flags, n_stored = struct.unpack(hdr_fmt, raw)
elif cach_version == 2:
magic, flags, n_stored, row_avg_n = struct.unpack(hdr_fmt, raw)
else:
elif cach_version == 3:
magic, flags, n_stored, row_avg_n, pad_factor = struct.unpack(hdr_fmt, raw)
else:
(magic, flags, n_stored, row_avg_n, pad_factor,
min_freq_khz) = struct.unpack(hdr_fmt, raw)
if magic != SFFT_MAGIC:
return None
for _ in range(n_stored):
@@ -627,7 +659,7 @@ class SrasFile:
break
self.precomputed_freq_mhz[angle_idx] = _read_f32_image(
f, self.image_shape(angle_idx))
return flags, row_avg_n, pad_factor
return flags, row_avg_n, pad_factor, min_freq_khz / 1000.0
def _parse_cach_section(self, offset: int):
"""Parse the v7 CACH tail that holds precomputed DC/FFT images."""
@@ -650,10 +682,13 @@ class SrasFile:
result = self._read_sfft_block(f, cach_version)
if result is None:
return
flags, row_avg_n, pad_factor = result
flags, row_avg_n, pad_factor, min_freq_mhz = result
self.precomputed_bg_sub = bool(flags & SFFT_FLAG_BG_SUB)
self.precomputed_row_avg_n = row_avg_n if (flags & SFFT_FLAG_ROW_AVG) else 0
self.precomputed_pad_factor = max(1, pad_factor)
# No flag bit gates the floor: 0 (= no floor) is already the
# value every pre-v4 tail reads as.
self.precomputed_min_freq_mhz = min_freq_mhz
def write_v7_cache(self, *,
new_dc3_mv: list[np.ndarray | None] | None = None,
@@ -661,7 +696,8 @@ class SrasFile:
new_freq_mhz: list[np.ndarray | None] | None = None,
new_bg_sub: bool | None = None,
new_row_avg_n: int | None = None,
new_pad_factor: int | None = None):
new_pad_factor: int | None = None,
new_min_freq_mhz: float | None = None):
"""Store computed DC and/or FFT images into this file's CACH tail,
in place, converting a v6 source to v7 (or updating an existing v7
file). Only the block(s) passed in are recomputed; whichever block
@@ -681,6 +717,13 @@ class SrasFile:
different pad resolves different peaks, so recording it is what lets
a reader refuse the cache instead of showing the wrong numbers.
*new_min_freq_mhz* is the min peak frequency floor the passed
*new_freq_mhz*'s peak search excluded bins below (0.0 = no floor),
carried forward the same way. It is stored fixed-point (whole kHz),
so the value is quantized to 0.001 MHz on write and
``precomputed_min_freq_mhz`` is updated to the quantized value —
what a reload would see, never a float the header can't represent.
The waveform data itself is never touched: the cache tail always
starts at ``_cache_tail_offset()``, a fixed offset derived from the
header and geometry table alone.
@@ -697,11 +740,21 @@ class SrasFile:
else self.precomputed_row_avg_n)
final_pad_factor = (new_pad_factor if new_pad_factor is not None
else self.precomputed_pad_factor)
final_min_freq_mhz = (new_min_freq_mhz if new_min_freq_mhz is not None
else self.precomputed_min_freq_mhz)
if not (0 <= final_row_avg_n <= 255):
raise ValueError(f"row_avg_n must fit in a byte (0-255), got {final_row_avg_n}")
if not (1 <= final_pad_factor <= MAX_PAD_FACTOR):
raise ValueError(
f"pad_factor must be 1-{MAX_PAD_FACTOR}, got {final_pad_factor}")
if not (np.isfinite(final_min_freq_mhz) and final_min_freq_mhz >= 0):
raise ValueError(
f"min_freq_mhz must be a finite value >= 0, got {final_min_freq_mhz}")
final_min_freq_khz = int(round(final_min_freq_mhz * 1000))
if final_min_freq_khz > MAX_MIN_FREQ_KHZ:
raise ValueError(
f"min_freq_mhz too large for the u32 kHz header field: "
f"{final_min_freq_mhz}")
# dc3/dc4 are always populated together by every current caller, but
# guard the per-angle pairing explicitly rather than assume it: an
@@ -730,7 +783,7 @@ class SrasFile:
fft_flags |= SFFT_FLAG_ROW_AVG if final_row_avg_n else 0
payload += struct.pack(SFFT_HDR_FMT, SFFT_MAGIC, fft_flags,
len(fft_entries), final_row_avg_n,
final_pad_factor)
final_pad_factor, final_min_freq_khz)
for a in fft_entries:
payload += struct.pack(">H", a)
payload += final_freq[a].astype(">f4").tobytes()
@@ -770,6 +823,7 @@ class SrasFile:
self.precomputed_bg_sub = final_bg_sub
self.precomputed_row_avg_n = final_row_avg_n
self.precomputed_pad_factor = final_pad_factor
self.precomputed_min_freq_mhz = final_min_freq_khz / 1000.0
# ------------------------------------------------------------------
# Axes helpers