The two clones had diverged from origin/main (191d1b8) and both had
grown uncommitted work. This merges the other clone's three commits and
reconciles four conflicting files.
Both clones independently grew a feature called "batch export", but they
are different sweeps and both are kept:
- BatchExportWorker (this clone) — every angle x channel of the one open
file, fixed colorbar per channel, under a new Export menu.
- BatchExportImagesWorker (other clone) — the current view across many
selected files, process-pooled, under Convert.
Conflict resolutions of note:
- sras_average.py: the other clone's memory-bounded rewrite had silently
reverted this clone's float32 -> int16 accumulator fix, which exists to
keep averaged output from landing 1 ADC count off the original tool.
Kept the rewrite, re-applied the fix, and corrected the two docstrings
that still claimed float32.
- tests/test_compute.py: kept the other clone's meta["waveforms"] key
(meta["data"] no longer exists for this fixture and would KeyError)
and its laser_freq_hz assertions, plus this clone's int16 expectation
and its dropped subprocess returncode assertions.
- ComputeWorker: row_avg_n and min_freq_mhz were added independently on
either side; both are now threaded through.
- compute_rf_image's `exact` parameter is gone, removed by the other
clone's PyFFTW peak-search rewrite. It had no remaining callers.
Verified: 149 passed with a complete dependency set. The failures seen
with this clone's own .venv are a missing scikit-image, which predates
this merge and reproduces identically on the pre-merge commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add Export -> Batch Export Images..., which renders and saves one PNG
per angle for whichever of CH1 (RF), CH3 (DC-A), CH4 (DC-B), and
Velocity the user selects, for the currently open file. Each channel
gets its own fixed min/max colorbar range, entered once in the dialog
and held constant across every exported angle, so the resulting images
are directly comparable to each other instead of each auto-scaling to
its own data (today's live-view default).
BatchExportDialog (sras_viewer.py) collects the output folder, file
prefix, and per-channel checkbox + range, seeded from whatever's
already cached (session DC/FFT caches, or the file's own v7 cache
blocks) so opening the dialog never triggers a fresh compute. Export
runs on a background thread via a new BatchExportWorker
(sras_workers.py), which computes each angle's channel image with the
existing dc_image_mv/compute_rf_image helpers (reusing one FFT per
angle for both RF and Velocity) and renders it with a headless
matplotlib Agg canvas.
Also includes several small correctness/robustness fixes that were
already staged in the working tree: an int16-vs-float32 accumulator
mismatch in sras_average.py's row averaging, a DC4-mask cache reuse and
atomic sidecar write in sras_compute.py, a dc3/dc4 pairing guard in
sras_format.py's v7 cache writer, and matching updates to the
tools/ test fixtures.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The old tool only understood the legacy uniform-geometry header and had no
bound on peak RAM, making it unusable on the ragged v6/v7 files the scanner
now produces at up to ~560GB. Reads via SrasFile's memmap and writes in
row-chunks sized to a memory budget (SRAS_MEM_BUDGET_MB), stages to .part and
os.replace()s per scan_format.md's atomic-write requirement, always writes
plain v6 (a v7 cache tail is frame-indexed and invalid after averaging), and
divides laser_freq_hz by N so x_axis_mm() stays correct after the X axis gets
spatially binned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- SrasFile: _parse_v6 now retains per-angle x_delta and the verbatim
preamble/background byte spans, and gains public data_offset,
y_pos_per_angle, and iter_angle_blocks() (which now owns the ragged
block-offset walk used three separate places before).
- sras_edit_scans: the 70-line re-parse of the v6 header sections
(_read_v6_sections/_reread_span/_v6_angle_offsets) collapses into a
_write_v6 that consumes SrasFile directly — verified byte-identical
round-trip on v6 int8/int16 and legacy files. Eight print-and-exit
pairs become _die().
- tools/make_test_sras imports the struct layouts from sras_format and
the rotation matrix from sras_compute instead of re-declaring them
(byte assembly stays independent of the reader).
- sras_compute: block_mean_2d, pixel_pitch_mm, nominal_delta_deg made
public (they were GUI-facing); registration_workers() and
default_max_workers() wrap the remaining private reach-throughs from
sras_workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Structure
sras_format.py parsing/writing, calibration, axes (numpy + struct)
sras_compute.py DC/FFT images, alignment, batch cache (+ scipy)
sras_workers.py Qt background workers
sras_viewer.py ROI, canvases, dialog, main window
format+compute import in 0.59s with no Qt or matplotlib (vs 2.96s for the
full app), which is what makes a spawn-based process pool worth using.
Deduplication
- SrasFile.cal() and compute.dc_image_mv() replace the ADC->mV
calibration incantation that appeared at ten call sites.
- _read_preambles/_read_background/_set_calibration are shared by the
legacy and v6 parsers instead of duplicated.
- v5 PREC images are normalised into the same ragged per-angle list
v6/v7 uses, removing four isinstance() branches and shortening
compute_rf_image's fast path.
- _run_worker replaces five copies of the QThread setup and five
near-identical teardown methods; the two lifetime hazards they
guarded against are now documented once, authoritatively.
- _on_channel_changed defers to _update_controls_enabled rather than
re-deriving the same six enable rules.
- _corners_in_ref_frame, _CHANNEL_DISPLAY dict, unified progress-dialog
helper, dead _on_roi_angle_edited stub removed.
- sras_average.py builds on SrasFile instead of carrying a second copy
of the header format, and streams per angle instead of loading the
whole file (was a ~3x file-size peak).
Executable lines: 2390 -> 2254, despite adding all of the below.
Parallelism
- compute_rf_image/compute_dc_image map row chunks over a thread pool.
Worker count is derived from the memory budget rather than the core
count: on a large scan chunk_rows is already floored at 1 row (~75 MB
of float32 at 7507x2500), so only the worker count can bound peak RAM.
- DcPrecomputeWorker computes angles on a pool, emitting each result
from its own QThread as it lands.
- BatchCacheWorker runs one process per file via cache_file(); only
paths and scalars cross the boundary. Per-process thread counts are
divided so the two levels don't oversubscribe. Drops the old pass 1,
which fully parsed every file just to weight a progress bar.
- dc_threshold_mv=None means "no mask" and skips the CH4 read entirely
— the batch FFT job previously read all of CH4 to compare against a
threshold of -1e9.
- Alignment mask and correlation stages map over angles; fft2/ifft2 use
workers=-1.
Fixes found on the way
- A partially-cached v5 file showed an all-zero image for uncached
angles: the fast-path check was file-wide, not per-angle.
- v7 cached images were read-only big-endian views; now native float32.
Verified: tools/check_equivalence.py produces byte-identical hashes for
167 outputs (DC/FFT across channels, angles, bg-sub, pad factors and
thresholds, plus the full alignment result) against ed0eba4, on both
synthetic files and a real 496 GB 17-angle v6 scan.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thomas Ales [M S E]