The old tool only understood the legacy uniform-geometry header and had no
bound on peak RAM, making it unusable on the ragged v6/v7 files the scanner
now produces at up to ~560GB. Reads via SrasFile's memmap and writes in
row-chunks sized to a memory budget (SRAS_MEM_BUDGET_MB), stages to .part and
os.replace()s per scan_format.md's atomic-write requirement, always writes
plain v6 (a v7 cache tail is frame-indexed and invalid after averaging), and
divides laser_freq_hz by N so x_axis_mm() stays correct after the X axis gets
spatially binned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- SrasFile: _parse_v6 now retains per-angle x_delta and the verbatim
preamble/background byte spans, and gains public data_offset,
y_pos_per_angle, and iter_angle_blocks() (which now owns the ragged
block-offset walk used three separate places before).
- sras_edit_scans: the 70-line re-parse of the v6 header sections
(_read_v6_sections/_reread_span/_v6_angle_offsets) collapses into a
_write_v6 that consumes SrasFile directly — verified byte-identical
round-trip on v6 int8/int16 and legacy files. Eight print-and-exit
pairs become _die().
- tools/make_test_sras imports the struct layouts from sras_format and
the rotation matrix from sras_compute instead of re-declaring them
(byte assembly stays independent of the reader).
- sras_compute: block_mean_2d, pixel_pitch_mm, nominal_delta_deg made
public (they were GUI-facing); registration_workers() and
default_max_workers() wrap the remaining private reach-throughs from
sras_workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Structure
sras_format.py parsing/writing, calibration, axes (numpy + struct)
sras_compute.py DC/FFT images, alignment, batch cache (+ scipy)
sras_workers.py Qt background workers
sras_viewer.py ROI, canvases, dialog, main window
format+compute import in 0.59s with no Qt or matplotlib (vs 2.96s for the
full app), which is what makes a spawn-based process pool worth using.
Deduplication
- SrasFile.cal() and compute.dc_image_mv() replace the ADC->mV
calibration incantation that appeared at ten call sites.
- _read_preambles/_read_background/_set_calibration are shared by the
legacy and v6 parsers instead of duplicated.
- v5 PREC images are normalised into the same ragged per-angle list
v6/v7 uses, removing four isinstance() branches and shortening
compute_rf_image's fast path.
- _run_worker replaces five copies of the QThread setup and five
near-identical teardown methods; the two lifetime hazards they
guarded against are now documented once, authoritatively.
- _on_channel_changed defers to _update_controls_enabled rather than
re-deriving the same six enable rules.
- _corners_in_ref_frame, _CHANNEL_DISPLAY dict, unified progress-dialog
helper, dead _on_roi_angle_edited stub removed.
- sras_average.py builds on SrasFile instead of carrying a second copy
of the header format, and streams per angle instead of loading the
whole file (was a ~3x file-size peak).
Executable lines: 2390 -> 2254, despite adding all of the below.
Parallelism
- compute_rf_image/compute_dc_image map row chunks over a thread pool.
Worker count is derived from the memory budget rather than the core
count: on a large scan chunk_rows is already floored at 1 row (~75 MB
of float32 at 7507x2500), so only the worker count can bound peak RAM.
- DcPrecomputeWorker computes angles on a pool, emitting each result
from its own QThread as it lands.
- BatchCacheWorker runs one process per file via cache_file(); only
paths and scalars cross the boundary. Per-process thread counts are
divided so the two levels don't oversubscribe. Drops the old pass 1,
which fully parsed every file just to weight a progress bar.
- dc_threshold_mv=None means "no mask" and skips the CH4 read entirely
— the batch FFT job previously read all of CH4 to compare against a
threshold of -1e9.
- Alignment mask and correlation stages map over angles; fft2/ifft2 use
workers=-1.
Fixes found on the way
- A partially-cached v5 file showed an all-zero image for uncached
angles: the fast-path check was file-wide, not per-angle.
- v7 cached images were read-only big-endian views; now native float32.
Verified: tools/check_equivalence.py produces byte-identical hashes for
167 outputs (DC/FFT across channels, angles, bg-sub, pad factors and
thresholds, plus the full alignment result) against ed0eba4, on both
synthetic files and a real 496 GB 17-angle v6 scan.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thomas Ales [M S E]