The two clones had diverged from origin/main (191d1b8) and both had
grown uncommitted work. This merges the other clone's three commits and
reconciles four conflicting files.
Both clones independently grew a feature called "batch export", but they
are different sweeps and both are kept:
- BatchExportWorker (this clone) — every angle x channel of the one open
file, fixed colorbar per channel, under a new Export menu.
- BatchExportImagesWorker (other clone) — the current view across many
selected files, process-pooled, under Convert.
Conflict resolutions of note:
- sras_average.py: the other clone's memory-bounded rewrite had silently
reverted this clone's float32 -> int16 accumulator fix, which exists to
keep averaged output from landing 1 ADC count off the original tool.
Kept the rewrite, re-applied the fix, and corrected the two docstrings
that still claimed float32.
- tests/test_compute.py: kept the other clone's meta["waveforms"] key
(meta["data"] no longer exists for this fixture and would KeyError)
and its laser_freq_hz assertions, plus this clone's int16 expectation
and its dropped subprocess returncode assertions.
- ComputeWorker: row_avg_n and min_freq_mhz were added independently on
either side; both are now threaded through.
- compute_rf_image's `exact` parameter is gone, removed by the other
clone's PyFFTW peak-search rewrite. It had no remaining callers.
Verified: 149 passed with a complete dependency set. The failures seen
with this clone's own .venv are a missing scikit-image, which predates
this merge and reproduces identically on the pre-merge commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add Export -> Batch Export Images..., which renders and saves one PNG
per angle for whichever of CH1 (RF), CH3 (DC-A), CH4 (DC-B), and
Velocity the user selects, for the currently open file. Each channel
gets its own fixed min/max colorbar range, entered once in the dialog
and held constant across every exported angle, so the resulting images
are directly comparable to each other instead of each auto-scaling to
its own data (today's live-view default).
BatchExportDialog (sras_viewer.py) collects the output folder, file
prefix, and per-channel checkbox + range, seeded from whatever's
already cached (session DC/FFT caches, or the file's own v7 cache
blocks) so opening the dialog never triggers a fresh compute. Export
runs on a background thread via a new BatchExportWorker
(sras_workers.py), which computes each angle's channel image with the
existing dc_image_mv/compute_rf_image helpers (reusing one FFT per
angle for both RF and Velocity) and renders it with a headless
matplotlib Agg canvas.
Also includes several small correctness/robustness fixes that were
already staged in the working tree: an int16-vs-float32 accumulator
mismatch in sras_average.py's row averaging, a DC4-mask cache reuse and
atomic sidecar write in sras_compute.py, a dc3/dc4 pairing guard in
sras_format.py's v7 cache writer, and matching updates to the
tools/ test fixtures.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The old tool only understood the legacy uniform-geometry header and had no
bound on peak RAM, making it unusable on the ragged v6/v7 files the scanner
now produces at up to ~560GB. Reads via SrasFile's memmap and writes in
row-chunks sized to a memory budget (SRAS_MEM_BUDGET_MB), stages to .part and
os.replace()s per scan_format.md's atomic-write requirement, always writes
plain v6 (a v7 cache tail is frame-indexed and invalid after averaging), and
divides laser_freq_hz by N so x_axis_mm() stays correct after the X axis gets
spatially binned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- SrasFile: _parse_v6 now retains per-angle x_delta and the verbatim
preamble/background byte spans, and gains public data_offset,
y_pos_per_angle, and iter_angle_blocks() (which now owns the ragged
block-offset walk used three separate places before).
- sras_edit_scans: the 70-line re-parse of the v6 header sections
(_read_v6_sections/_reread_span/_v6_angle_offsets) collapses into a
_write_v6 that consumes SrasFile directly — verified byte-identical
round-trip on v6 int8/int16 and legacy files. Eight print-and-exit
pairs become _die().
- tools/make_test_sras imports the struct layouts from sras_format and
the rotation matrix from sras_compute instead of re-declaring them
(byte assembly stays independent of the reader).
- sras_compute: block_mean_2d, pixel_pitch_mm, nominal_delta_deg made
public (they were GUI-facing); registration_workers() and
default_max_workers() wrap the remaining private reach-throughs from
sras_workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- pyproject.toml replaces sras_viewer_requirements.txt (same pins) and
adds a dev extra with pytest.
- tools/test_refactor.py, test_alignment.py, test_gui.py become
tests/test_compute.py, tests/test_alignment.py, tests/test_gui.py with
assertions preserved verbatim. test_gui.py stays one ordered
integration sequence over a shared module-scoped window.
- check_equivalence.py: drop the dead pre-refactor monolith shim (and the
_compute_angle_alignment alias it consumed), extend the pad sweep to
(1, 2, 4, 8, 40), add legacy-v4 and big-endian int16 legs (new bps=2
option in make_test_sras) so the padded FFT path and the >i2 memmap
path are in the baseline before the FFT rewrite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Continues the Manual Alignment work: refines the FFT cross-correlation and
mask handling, adds sras_edit_scans.py (drop/renumber bad angle scans),
tools/test_alignment.py (registration ground-truth suite), and a rotating
test fixture in make_test_sras.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three defects found by testing the parallel paths end to end.
Nested parallelism was unbounded. DcPrecomputeWorker and the alignment
driver both parallelise over angles, and each angle's compute_dc_image
then parallelised over rows again: 14 x 14 threads, and — worse — every
angle sized its chunk against the *whole* memory budget, so worst-case
live buffers were ~13.9 GB against a 1 GB budget. Capping the inner
worker count alone does not fix it; the chunk is still sized against the
full budget. plan_angle_level now hands each angle both max_workers=1
and its share of the budget, which bounds the real file at 1002 MB.
Chunk planning could not produce concurrency at all. _plan_chunks sized
chunk_rows first, and _chunk_rows_for spends whatever budget it is given
— so a single chunk consumed the lot and budget//chunk_bytes came back
as one worker. It now picks the worker count first and sizes the chunk to
it, giving 7-16 workers on the real file instead of 1.
Cancellation was too coarse to shut down. stop() was only checked between
angles, and one angle of the real scan is ~40 s, so closing the window
sat through it and returned with threads still running. should_stop is
now polled per row chunk and closeEvent signals every worker before
waiting: close went from 4.0 s with two live threads to 1.04 s with both
finished. A cancelled ComputeWorker emits None so a half-filled image is
never cached.
Also: BatchCacheWorker no longer reports every file as failed when the
process pool itself dies (unguarded __main__, frozen build, sandbox) —
it retries those files in-process. Batches below 512 MB skip the pool
entirely, since interpreter startup would otherwise make small jobs
slower. cache_file rejects an unknown mode instead of silently treating
it as "fft".
Measured on the real 496 GB scan, angle 3, 32 rows:
DC 2.94s -> 0.75s RF 6.10s -> 1.99s peak RSS 1.40 -> 1.93 GB
Batch convert, 24 files / 1 GB: 4.03s -> 2.36s.
Memory budget defaults to 1024 MB (SRAS_MEM_BUDGET_MB), the measured knee.
Testing: tools/test_refactor.py (70 checks: cache round-trip and block
carry-forward, partial-cache handling, parallel-vs-serial identity,
no-mask path, ROI mask vs full-grid, v2/v3/v4 parsing, sras_average
round-trip incl. remainder handling) and tools/test_gui.py (46 checks
driving the real widgets and workers headlessly). check_equivalence.py
still reports all 167 outputs identical to ed0eba4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/make_test_sras.py writes small v6 files with per-angle-varying
geometry, distinct per-channel calibration, and deterministic waveform
content (predictable FFT peak per pixel, predictable DC mean per pixel).
tools/check_equivalence.py hashes DC/FFT/alignment outputs across
channels, angles, bg-sub, pad factors, and thresholds. It imports from
either the monolith or the split modules, so the same script captures
both sides of a refactor. Hashes canonicalise to native float64 so a
dtype or byte-order change that preserves values is not a false failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thomas Ales [M S E]