The two clones had diverged from origin/main (191d1b8) and both had
grown uncommitted work. This merges the other clone's three commits and
reconciles four conflicting files.
Both clones independently grew a feature called "batch export", but they
are different sweeps and both are kept:
- BatchExportWorker (this clone) — every angle x channel of the one open
file, fixed colorbar per channel, under a new Export menu.
- BatchExportImagesWorker (other clone) — the current view across many
selected files, process-pooled, under Convert.
Conflict resolutions of note:
- sras_average.py: the other clone's memory-bounded rewrite had silently
reverted this clone's float32 -> int16 accumulator fix, which exists to
keep averaged output from landing 1 ADC count off the original tool.
Kept the rewrite, re-applied the fix, and corrected the two docstrings
that still claimed float32.
- tests/test_compute.py: kept the other clone's meta["waveforms"] key
(meta["data"] no longer exists for this fixture and would KeyError)
and its laser_freq_hz assertions, plus this clone's int16 expectation
and its dropped subprocess returncode assertions.
- ComputeWorker: row_avg_n and min_freq_mhz were added independently on
either side; both are now threaded through.
- compute_rf_image's `exact` parameter is gone, removed by the other
clone's PyFFTW peak-search rewrite. It had no remaining callers.
Verified: 149 passed with a complete dependency set. The failures seen
with this clone's own .venv are a missing scikit-image, which predates
this merge and reproduces identically on the pre-merge commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two strands of in-progress work, committed together because they overlap
in sras_workers.py and main_window.py.
Min peak frequency floor:
- CACH tail bumped to version 4, adding u32 min_freq_khz provenance in
fixed-point kHz (a float32 20.1 reads back as 20.10000038 and would
report a spurious mismatch forever). v1-v3 tails read as no floor.
- Stored FFT caches are accepted when the reader's floor is at or above
the stored one, since a higher floor is re-applicable by masking.
- Floor plumbed through compute_rf_image, BatchCacheWorker and the viewer.
Batch Export View as Images:
- New sras_render.py holds draw_view_image, shared by the Qt canvas and
the headless exporter so a PNG cannot drift from what the GUI shows.
Deliberately Qt-free so it is importable in a pool subprocess.
- BatchExportImagesWorker renders the current view settings across many
files, process-pooled with an inline fallback, reporting per-file
output names so the caller can flag same-stem collisions.
- _axes_extent extracted into sras_format for both render paths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ComputeWorker dropped row_avg_n, so a stored row-averaged cache was
ineligible whenever the fallback compute served the image instead of the
DC precompute path. Thread the viewer's precomputed_row_avg_n through
ComputeWorker and the batch export worker's compute_rf_image call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
origin/main split sras_viewer.py into the sras_viewer/ package (dialogs,
main_window, common, canvases) after the batch-export feature was built
against the old monolith, so merging origin/main into main lost the feature
to the sras_viewer.py deletion. Re-add it in the new structure:
BatchExportDialog in dialogs.py, a Jobs.EXPORT job key in common.py, and
the menu action/worker wiring in main_window.py, following the Jobs-key
and dialog-class conventions the split already established rather than the
old ad hoc "main" progress-key naming the original commit used.
Also carries forward two test fixes bundled in the original batch-export
commit that the parallel pytest conversion (007089d) had ported from
tools/test_refactor.py without them: the int16-vs-float32 accumulator
expectation in test_sras_average, and asserting the sras_average.py
subprocess calls actually succeed.
The old tool only understood the legacy uniform-geometry header and had no
bound on peak RAM, making it unusable on the ragged v6/v7 files the scanner
now produces at up to ~560GB. Reads via SrasFile's memmap and writes in
row-chunks sized to a memory budget (SRAS_MEM_BUDGET_MB), stages to .part and
os.replace()s per scan_format.md's atomic-write requirement, always writes
plain v6 (a v7 cache tail is frame-indexed and invalid after averaging), and
divides laser_freq_hz by N so x_axis_mm() stays correct after the X axis gets
spatially binned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Drop the coarse+fine zoom refinement, the SciPy FFT backend, and the
exact= audit path in favor of a single always-on full-transform peak
search (_peak_bins). Block size is now derived from a per-thread memory
budget (_fft_block_for/SRAS_FFT_PLAN_BUDGET_MB) instead of a fixed
constant, so the existing block-parallel PyFFTW pool stays memory-safe
at high pad factors without the zoom algorithm's bookkeeping. Also
removes the now-unused threadpoolctl dependency and the FFT backend
selector from the UI.
Also includes a pre-existing min_freq_mhz peak-search floor (excludes
bins below a caller-supplied frequency from the argmax) that was
already implemented and tested in the working tree.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Lets a user export the current ROI as one CSV with a column per selected
angle's value (RF peak freq, Bias A, Bias B, or velocity), once angles share
a common (x, y) grid -- either via a live Fusion alignment result or because
the open file is itself a previous Alignment Wizard export whose angles
already share one grid on disk. Never triggers a background compute; angles
without cached/stored data for the chosen value type are simply unavailable
in the picker.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The display path hardcoded row_avg_n=0 when checking a file's stored FFT
cache, since the main window has no row-averaging control (only the
batch-only RowAverageFftOptionsDialog). Once a file was batch-computed with
row-averaging, every view switch saw a phantom provenance mismatch and
silently launched a full raw recompute, discarding the precomputed data.
The display now asks for the stored image at the settings it was actually
computed under (bg-sub/pad/row-averaging), rather than the window's live
controls, so a stored or already-computed image is always shown once
present. Those controls now only affect a first-time compute for an
uncached angle or an explicit batch recompute -- never what's already on
screen. DC threshold is unaffected: it stays live, since re-masking a
cached image is free.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Extract cache mismatch logic into reusable cache_mismatch_reasons() function
for cleaner validation and consistent miss reporting
- Expose memory_budget_bytes() for external callers (aligned exporter)
- Optimize rebuild_stack() with incremental layer updates when only one angle
parameters change, avoiding full reprojection overhead
- Remove unused seed_per_angle parameter from alignment wizard
- Simplify cached_rf_image() DC masking with cleaner early exit logic
- Clean up internal state tracking with explicit origin caching
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
tests/test_stored_cache.py exercised two features that were never built,
so six of its tests had been failing on main. Both are now implemented.
Pad-factor caching. A padded view could not use a stored FFT cache at all:
the store was pad 1 by definition and cached_rf_image rejected any n_fft
outright, so a user working at a pad factor got nothing from batch-computing
a file. CACH tail v3 records the pad the images were resolved at, cache_file
computes at a requested pad, and the batch actions pass the viewer's own pad
down — while still refusing a store resolved at a different pad, since a
padded FFT interpolates between the natural bins and so resolves genuinely
different peak frequencies. v1/v2 tails read as pad 1 and keep working.
Stored-cache dispatch. _refresh_display only ever consulted this window's
in-session dicts, so after a batch every angle change still queued a worker
and a progress popup for an image already on disk — the exact cost the batch
was run to avoid. It now checks the file's own DC/FFT blocks first, asking
with allow_dc_recompute=False so the GUI thread never touches I/O. When the
stored cache genuinely cannot serve the view, the scan info panel says why
rather than leaving the silent recompute a mystery.
Two bugs surfaced on the way:
- The angle spinbox was wired on editingFinished, which QAbstractSpinBox
emits only on Return or focus-out — never on a step. Clicking its arrows,
the ordinary way to walk a scan, moved the number and left the image
behind. Now valueChanged with keyboard tracking off, which fires on a step
and once on commit, but not per keystroke mid-typing. Every existing GUI
test called _on_view_changed() by hand and so could not have caught this;
two new tests pin it and fail against the old wiring.
- A plain "fft" batch over a file previously cached with fft_rowavg carried
the old row_avg_n forward, labelling raw images as row-averaged. It now
writes row_avg_n=0 explicitly.
Verified: 111 passed (was 103 passed / 6 failed), and
tools/check_equivalence.py is byte-identical to the pre-change baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Alignment was two disconnected menu actions. `Angle Alignment` ran the
registration with ref_angle_idx hard-coded to 0, no exposed parameters and
no way to retry; `Manual Alignment...` was a separate dialog that
deliberately refused to inherit the automatic result, so a bad fit meant
starting over by hand. Neither told you whether the alignment was any
good, and neither produced anything durable beyond an in-memory result.
Fusion -> Alignment Wizard... now covers all of it in three steps:
1. Correlate. Reference angle, DC threshold, source, rotation seed and
sign, search window, coarse step and fine grid are all on the page,
and Run/Re-run is repeatable. Angles start pre-rotated from the
stage angles in the file, so the page is informative before any
correlation runs. The verdict is a picture: every angle's DC mask
reprojected onto the shared canvas and summed, coloured by how many
angles cover each pixel, so a good alignment reads as one saturated
plateau and a bad one as a fringe of low-count halos. A per-angle
fit table flags angles that did not register or that disagree with
their stage angle by more than a degree.
2. Crop. An axis-aligned rectangle on that canvas, with numeric
canvas-pixel boxes synced both ways and a live size estimate.
"Fit to full overlap" uses a largest-rectangle sweep rather than a
bounding box: the overlap region of several rotated scans is roughly
a disc, whose bounding box has corners no angle covers.
3. Save. Writes the aligned, cropped stack to a new .sras.
Because the wizard replaces both actions it absorbs the old dialog's
by-eye nudge editor — without it, a scan the search cannot fit would
have no fallback at all. ManualAlignmentDialog is therefore deleted
rather than left orphaned, and its tests move to the wizard.
Two bugs found while driving it end to end and fixed here: QSpinBox
setRange clamps and emits valueChanged, which committed a 1x1 crop
before the default preset could run; and the mm round trip returns an
exact pixel boundary as 11.000000000000002, so a bare ceil() added a
spurious column on every rectangle edit.
Verified: 103 pass, the 6 test_stored_cache failures are pre-existing on
main, and tools/check_equivalence.py is byte-identical to main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The alignment machinery never modified a scan: it produced an
AlignmentResult and every consumer resampled on the fly. That is right
for a viewer but means the aligned stack cannot leave the process, so no
other tool can read it and reopening the scan redoes the registration.
sras_align_export.write_aligned_sras bakes an alignment into a new v6
file: each angle is gathered onto the shared canvas by nearest neighbour,
so every output angle shares one grid and the file opens already aligned.
Registering the export against itself returns identity, which is the test
that pins the whole index chain.
Three details that are easy to get wrong and are now covered:
* Rounding must be floor(x + 0.5), not np.rint. scipy's order=0 rounds
halves away from zero, and the canvas is snapped to the reference's
pixel grid, so exact halves are common rather than hypothetical.
* Out-of-bounds must be tested on the fractional coordinate against
[0, n-1], not on the rounded index, or a one-pixel rim gets real data
everywhere the Aligned View shows padding.
* ...but with a tolerance, because the mm-space affine chain lands an
exactly-integer transform a few times 1e-13 off. A bare >= 0 drops
the *reference* angle's entire first row and last column.
Padding is the per-channel ADC code nearest 0 mV, not zero: zero ADC
decodes to ~+100 mV on real calibration and would masquerade as sample.
Source rows are served from sliding in-RAM bands. A rotated angle maps
one output row to a diagonal across the source, so indexing a memmap in
output order refaults nearly the whole angle per row — terabytes of
paging for a gigabyte of data.
Also in compute: crop_alignment_result (a crop is pure index translation,
so it folds into the affine's offset rather than becoming a second
transform), overlap_stats, largest_rect_at_least for a crop that stays
inside the overlap region, and seed/sign/refine knobs on
register_angle_to_reference whose defaults leave existing behaviour and
tests byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 2790-line module becomes sras_viewer/: common.py (constants +
layout helpers), canvases.py (RoiQuad, ImageCanvas, WaveformCanvas,
ManualAlignOverlayCanvas), dialogs.py (FftOptionsDialog,
ManualAlignmentDialog), main_window.py (SrasViewerWindow + main), with
__init__ re-exporting the public names and __main__ keeping
`python -m sras_viewer` working. pyproject gains a `sras-viewer`
console script.
Code moved verbatim; only import headers are new (pyflakes-clean).
tests/test_gui.py patch targets follow the classes to their new
modules.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- SrasFile: _parse_v6 now retains per-angle x_delta and the verbatim
preamble/background byte spans, and gains public data_offset,
y_pos_per_angle, and iter_angle_blocks() (which now owns the ragged
block-offset walk used three separate places before).
- sras_edit_scans: the 70-line re-parse of the v6 header sections
(_read_v6_sections/_reread_span/_v6_angle_offsets) collapses into a
_write_v6 that consumes SrasFile directly — verified byte-identical
round-trip on v6 int8/int16 and legacy files. Eight print-and-exit
pairs become _die().
- tools/make_test_sras imports the struct layouts from sras_format and
the rotation matrix from sras_compute instead of re-declaring them
(byte assembly stays independent of the reader).
- sras_compute: block_mean_2d, pixel_pitch_mm, nominal_delta_deg made
public (they were GUI-facing); registration_workers() and
default_max_workers() wrap the remaining private reach-throughs from
sras_workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- FFT backend and pad factor persist across sessions via QSettings
(IniFormat; tests redirect the settings path for hermeticity).
- The default backend was labelled "NumPy FFT" but always dispatched to
scipy.fft — rename the canonical value to "scipy" ("numpy" stays as a
legacy alias) and fix the dialog label.
- cache_file: DC caching fans out over angles via _parallel_map with
per-angle budgets (the DcPrecomputeWorker pattern); FFT caching stays
serial per angle because compute_rf_image now parallelises internally
over blocks. Documented that the v7 FFT cache is natural-resolution
(pad 1) by design.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
At pad 40 the old path materialised a ~9 GB padded spectrum per row,
which collapsed the chunk planner to one worker and one rfft call with
workers=1 — synthesis ran single-threaded, ~1 hour per angle on real
files.
The padded spectrum is never materialised now. Each block of 512
waveforms gets a coarse rfft at next_fast_len(2*spf); every coarse bin
within 0.7 of its row's max (plus the DC-adjacent window, which coarse
DC suppression would otherwise blind) is refined onto the exact n_fft
grid by a small complex gemm. The selected bin is bit-identical to the
full padded argmax — enforced by test_zoom_identity, a 25-seed fuzz
test over adversarial spectra, and a clean golden-hash diff against the
pre-rewrite baseline across pads {1,2,4,8,40}, masked/unmasked, bg
on/off, int8/int16, and both backends.
Blocks fan out over a persistent thread pool; pyFFTW runs through
per-thread FFTW_MEASURE builder plans with wisdom persisted to
~/.cache/sras-viewer, and threadpoolctl clamps BLAS under the pool.
compute_rf_image(exact=True) (or SRAS_FFT_EXACT=1) keeps the reference
padded path for audits.
tools/bench_fft.py measures: pad 40, 16 cores, 8192x2500 synthetic —
exact serial 717 wf/s -> zoom pool 25100 wf/s (35x, pyFFTW backend;
19x scipy), every variant verified equal to the reference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- pyproject.toml replaces sras_viewer_requirements.txt (same pins) and
adds a dev extra with pytest.
- tools/test_refactor.py, test_alignment.py, test_gui.py become
tests/test_compute.py, tests/test_alignment.py, tests/test_gui.py with
assertions preserved verbatim. test_gui.py stays one ordered
integration sequence over a shared module-scoped window.
- check_equivalence.py: drop the dead pre-refactor monolith shim (and the
_compute_angle_alignment alias it consumed), extend the pad sweep to
(1, 2, 4, 8, 40), add legacy-v4 and big-endian int16 legs (new bps=2
option in make_test_sras) so the padded FFT path and the >i2 memmap
path are in the baseline before the FFT rewrite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Thomas Ales [M S E]