44 Commits

Author SHA1 Message Date
Thomas Ales [M S E] 48d560fb00 Make both batch image exports reproduce the view on screen
Two strands, committed together because they touch the same four files
and answer the same question: an exported PNG must be the image the user
was looking at when they triggered the export.

Dropped view settings (bug fix):
- BatchExportWorker forwarded the DC threshold, bg-sub, pad and row-
  averaging to compute_rf_image but never min_freq_mhz, so every RF and
  Velocity map was exported with no min peak frequency floor. The pixels
  the floor exists to reject came back at their pre-floor peaks.
- That also decided *which source* the image came from: requesting floor
  0 makes a stored v7 cache look like a match (cache_mismatch_reasons
  only rejects a higher stored floor), so a batch-computed file exported
  the cached peak-frequency block verbatim - numbers from an earlier
  Batch Compute run, while the screen showed that same cache re-masked
  against the live floor. The symptom was a newly loaded file exporting
  a stale-looking velocity map, since the floor is a display control
  that survives a load while the window's own FFT cache does not.
- export_view_image had the same class of defect with a different
  parameter: it dropped row_avg_n, so a file whose stored cache is row-
  averaged displayed from the store but exported as a full raw
  recompute - a different, and much slower, image. It now reads
  sras.precomputed_row_avg_n per file, mirroring _stored_fft_image and
  _start_compute, and derived per file for the same reason n_fft is.

Export at the live canvas size:
- Both exporters now take the canvas's current figure size instead of a
  hard-coded 7x5. draw_view_image uses aspect="auto", so the figure box
  is what sets the map's proportions - a view sized wide on screen was
  being squeezed into a different shape on disk. Read at trigger time,
  in inches, so a resize mid-batch cannot change images later in the
  same run and a HiDPI display exports like a standard one.
- sanitize_figsize clamps a degenerate size (collapsed splitter pane,
  minimized window): a wrong-looking aspect ratio must never be the
  reason a batch loses an image.

Tests: BatchExportWorker had no coverage at all. Four new tests cover
the floor being applied, a stored cache never being served unfloored,
the menu-to-worker wiring, and the row-averaged cache case; each was
verified to fail against the unfixed code. 166 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 12:16:45 -05:00
Thomas Ales [M S E] 2937c017a7 Merge the Documents/sras-viewer working copy into this repo
The two clones had diverged from origin/main (191d1b8) and both had
grown uncommitted work. This merges the other clone's three commits and
reconciles four conflicting files.

Both clones independently grew a feature called "batch export", but they
are different sweeps and both are kept:
- BatchExportWorker (this clone) — every angle x channel of the one open
  file, fixed colorbar per channel, under a new Export menu.
- BatchExportImagesWorker (other clone) — the current view across many
  selected files, process-pooled, under Convert.

Conflict resolutions of note:
- sras_average.py: the other clone's memory-bounded rewrite had silently
  reverted this clone's float32 -> int16 accumulator fix, which exists to
  keep averaged output from landing 1 ADC count off the original tool.
  Kept the rewrite, re-applied the fix, and corrected the two docstrings
  that still claimed float32.
- tests/test_compute.py: kept the other clone's meta["waveforms"] key
  (meta["data"] no longer exists for this fixture and would KeyError)
  and its laser_freq_hz assertions, plus this clone's int16 expectation
  and its dropped subprocess returncode assertions.
- ComputeWorker: row_avg_n and min_freq_mhz were added independently on
  either side; both are now threaded through.
- compute_rf_image's `exact` parameter is gone, removed by the other
  clone's PyFFTW peak-search rewrite. It had no remaining callers.

Verified: 149 passed with a complete dependency set. The failures seen
with this clone's own .venv are a missing scikit-image, which predates
this merge and reproduces identically on the pre-merge commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 10:36:25 -05:00
Thomas Ales [M S E] a8b962e317 Add min peak frequency floor (CACH v4) and Batch Export View as Images
Two strands of in-progress work, committed together because they overlap
in sras_workers.py and main_window.py.

Min peak frequency floor:
- CACH tail bumped to version 4, adding u32 min_freq_khz provenance in
  fixed-point kHz (a float32 20.1 reads back as 20.10000038 and would
  report a spurious mismatch forever). v1-v3 tails read as no floor.
- Stored FFT caches are accepted when the reader's floor is at or above
  the stored one, since a higher floor is re-applicable by masking.
- Floor plumbed through compute_rf_image, BatchCacheWorker and the viewer.

Batch Export View as Images:
- New sras_render.py holds draw_view_image, shared by the Qt canvas and
  the headless exporter so a PNG cannot drift from what the GUI shows.
  Deliberately Qt-free so it is importable in a pool subprocess.
- BatchExportImagesWorker renders the current view settings across many
  files, process-pooled with an inline fallback, reporting per-file
  output names so the caller can flag same-stem collisions.
- _axes_extent extracted into sras_format for both render paths.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 10:17:43 -05:00
Thomas Ales [M S E] efecc154fe Keep stored row-averaged caches eligible on the fallback compute path
ComputeWorker dropped row_avg_n, so a stored row-averaged cache was
ineligible whenever the fallback compute served the image instead of the
DC precompute path. Thread the viewer's precomputed_row_avg_n through
ComputeWorker and the batch export worker's compute_rf_image call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 10:17:16 -05:00
Thomas Ales [M S E] ec0d322871 Merge remote-tracking branch 'origin/refactor/parallel-and-dedup' into main
Its only unmerged commit, "pre merge cleanup" (1a81871), branched off
refactor/parallel-and-dedup after PR #1 (b0bfe6e) had already merged that
branch into main and was never rebased onto main's later work. Every change
it made was independently reimplemented, more completely, by main's own
subsequent commits:

  - FFT pad-factor caching + the stored-cache display fast path
    (cached_rf_image, precomputed_pad_factor, _stored_fft_image,
    _stored_dc_image, PAD_FACTOR_MAX) -> 8348ad3, refined by c30c8b1.
  - The FftOptionsDialog "stored cache unusable at this pad" warning
    -> also 8348ad3.
  - spin_angle redrawing on arrows/keyboard via setKeyboardTracking(False)
    + valueChanged, and the corresponding tests
    (test_stepping_the_angle_spinbox_redraws,
    test_typing_an_angle_does_not_compute_intermediate_angles) -> already
    on main before the refactor/parallel-and-dedup merge point.

A real 3-way merge of 1a81871 confirmed this: it produced no conflicts in
sras_format.py/sras_compute.py only because both sides independently wrote
equivalent code, and silently duplicated a method definition
(_stored_fft_image) in sras_viewer/main_window.py where the two
implementations diverged textually. Recording this as a merge with main's
tree kept (-s ours) marks the branch as fully incorporated without
reintroducing its now-superseded, lower cache-version (CACH_VERSION 2 vs
main's 3, missing row-averaged-FFT support) implementation.
2026-08-10 19:01:21 -05:00
Thomas Ales [M S E] aca501ae5c Port batch image export onto the sras_viewer package split
origin/main split sras_viewer.py into the sras_viewer/ package (dialogs,
main_window, common, canvases) after the batch-export feature was built
against the old monolith, so merging origin/main into main lost the feature
to the sras_viewer.py deletion. Re-add it in the new structure:
BatchExportDialog in dialogs.py, a Jobs.EXPORT job key in common.py, and
the menu action/worker wiring in main_window.py, following the Jobs-key
and dialog-class conventions the split already established rather than the
old ad hoc "main" progress-key naming the original commit used.

Also carries forward two test fixes bundled in the original batch-export
commit that the parallel pytest conversion (007089d) had ported from
tools/test_refactor.py without them: the int16-vs-float32 accumulator
expectation in test_sras_average, and asserting the sras_average.py
subprocess calls actually succeed.
2026-08-10 18:58:56 -05:00
Thomas Ales [M S E] 105b9514a5 Merge remote-tracking branch 'origin/main' into main
# Conflicts:
#	sras_compute.py
#	sras_viewer.py
#	sras_workers.py
#	tools/make_test_sras.py
#	tools/test_refactor.py
2026-08-10 18:58:21 -05:00
Thomas Ales [M S E] 4212b8c313 Merge branch 'feature/batch-export-images' into main
Adds Export -> Batch Export Images... (see 6976cbf for details), plus
the small correctness fixes bundled in that commit.
2026-08-10 18:30:29 -05:00
Thomas Ales [M S E] 6976cbf767 Add batch export of DC/RF/Velocity map images
Add Export -> Batch Export Images..., which renders and saves one PNG
per angle for whichever of CH1 (RF), CH3 (DC-A), CH4 (DC-B), and
Velocity the user selects, for the currently open file. Each channel
gets its own fixed min/max colorbar range, entered once in the dialog
and held constant across every exported angle, so the resulting images
are directly comparable to each other instead of each auto-scaling to
its own data (today's live-view default).

BatchExportDialog (sras_viewer.py) collects the output folder, file
prefix, and per-channel checkbox + range, seeded from whatever's
already cached (session DC/FFT caches, or the file's own v7 cache
blocks) so opening the dialog never triggers a fresh compute. Export
runs on a background thread via a new BatchExportWorker
(sras_workers.py), which computes each angle's channel image with the
existing dc_image_mv/compute_rf_image helpers (reusing one FFT per
angle for both RF and Velocity) and renders it with a headless
matplotlib Agg canvas.

Also includes several small correctness/robustness fixes that were
already staged in the working tree: an int16-vs-float32 accumulator
mismatch in sras_average.py's row averaging, a DC4-mask cache reuse and
atomic sidecar write in sras_compute.py, a dc3/dc4 pairing guard in
sras_format.py's v7 cache writer, and matching updates to the
tools/ test fixtures.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 18:30:00 -05:00
Thomas Ales [M S E] c54cce453c Rewrite sras_average.py for v6/v7, memory-bounded for multi-hundred-GB scans
The old tool only understood the legacy uniform-geometry header and had no
bound on peak RAM, making it unusable on the ragged v6/v7 files the scanner
now produces at up to ~560GB. Reads via SrasFile's memmap and writes in
row-chunks sized to a memory budget (SRAS_MEM_BUDGET_MB), stages to .part and
os.replace()s per scan_format.md's atomic-write requirement, always writes
plain v6 (a v7 cache tail is frame-indexed and invalid after averaging), and
divides laser_freq_hz by N so x_axis_mm() stays correct after the X axis gets
spatially binned.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 17:24:41 -05:00
Thomas Ales [M S E] 1caf6373cb Replace zoom FFT peak search with a budget-bounded PyFFTW direct transform
Drop the coarse+fine zoom refinement, the SciPy FFT backend, and the
exact= audit path in favor of a single always-on full-transform peak
search (_peak_bins). Block size is now derived from a per-thread memory
budget (_fft_block_for/SRAS_FFT_PLAN_BUDGET_MB) instead of a fixed
constant, so the existing block-parallel PyFFTW pool stays memory-safe
at high pad factors without the zoom algorithm's bookkeeping. Also
removes the now-unused threadpoolctl dependency and the FFT backend
selector from the UI.

Also includes a pre-existing min_freq_mhz peak-search floor (excludes
bins below a caller-supplied frequency from the argmax) that was
already implemented and tested in the working tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-10 14:14:38 -05:00
Thomas Ales 191d1b8946 Add Export Fused ROI CSV export
Lets a user export the current ROI as one CSV with a column per selected
angle's value (RF peak freq, Bias A, Bias B, or velocity), once angles share
a common (x, y) grid -- either via a live Fusion alignment result or because
the open file is itself a previous Alignment Wizard export whose angles
already share one grid on disk. Never triggers a background compute; angles
without cached/stored data for the chosen value type are simply unavailable
in the picker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-09 19:11:11 -05:00
Thomas Ales d5914b5793 Stop discarding precomputed FFT data on view switches
The display path hardcoded row_avg_n=0 when checking a file's stored FFT
cache, since the main window has no row-averaging control (only the
batch-only RowAverageFftOptionsDialog). Once a file was batch-computed with
row-averaging, every view switch saw a phantom provenance mismatch and
silently launched a full raw recompute, discarding the precomputed data.

The display now asks for the stored image at the settings it was actually
computed under (bg-sub/pad/row-averaging), rather than the window's live
controls, so a stored or already-computed image is always shown once
present. Those controls now only affect a first-time compute for an
uncached angle or an explicit batch recompute -- never what's already on
screen. DC threshold is unaffected: it stays live, since re-masking a
cached image is free.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-09 18:21:34 -05:00
Thomas Ales c30c8b1815 Refactor cache validation and optimize alignment wizard incremental updates
- Extract cache mismatch logic into reusable cache_mismatch_reasons() function
  for cleaner validation and consistent miss reporting
- Expose memory_budget_bytes() for external callers (aligned exporter)
- Optimize rebuild_stack() with incremental layer updates when only one angle
  parameters change, avoiding full reprojection overhead
- Remove unused seed_per_angle parameter from alignment wizard
- Simplify cached_rf_image() DC masking with cleaner early exit logic
- Clean up internal state tracking with explicit origin caching

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-08-09 14:30:02 -05:00
Thomas Ales 8348ad313c Implement FFT pad-factor caching and the stored-cache display fast path
tests/test_stored_cache.py exercised two features that were never built,
so six of its tests had been failing on main. Both are now implemented.

Pad-factor caching. A padded view could not use a stored FFT cache at all:
the store was pad 1 by definition and cached_rf_image rejected any n_fft
outright, so a user working at a pad factor got nothing from batch-computing
a file. CACH tail v3 records the pad the images were resolved at, cache_file
computes at a requested pad, and the batch actions pass the viewer's own pad
down — while still refusing a store resolved at a different pad, since a
padded FFT interpolates between the natural bins and so resolves genuinely
different peak frequencies. v1/v2 tails read as pad 1 and keep working.

Stored-cache dispatch. _refresh_display only ever consulted this window's
in-session dicts, so after a batch every angle change still queued a worker
and a progress popup for an image already on disk — the exact cost the batch
was run to avoid. It now checks the file's own DC/FFT blocks first, asking
with allow_dc_recompute=False so the GUI thread never touches I/O. When the
stored cache genuinely cannot serve the view, the scan info panel says why
rather than leaving the silent recompute a mystery.

Two bugs surfaced on the way:

- The angle spinbox was wired on editingFinished, which QAbstractSpinBox
  emits only on Return or focus-out — never on a step. Clicking its arrows,
  the ordinary way to walk a scan, moved the number and left the image
  behind. Now valueChanged with keyboard tracking off, which fires on a step
  and once on commit, but not per keystroke mid-typing. Every existing GUI
  test called _on_view_changed() by hand and so could not have caught this;
  two new tests pin it and fail against the old wiring.

- A plain "fft" batch over a file previously cached with fft_rowavg carried
  the old row_avg_n forward, labelling raw images as row-averaged. It now
  writes row_avg_n=0 explicitly.

Verified: 111 passed (was 103 passed / 6 failed), and
tools/check_equivalence.py is byte-identical to the pre-change baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 13:50:37 -05:00
Thomas Ales f40c965b74 Replace the two Fusion alignment actions with a three-step wizard
Alignment was two disconnected menu actions. `Angle Alignment` ran the
registration with ref_angle_idx hard-coded to 0, no exposed parameters and
no way to retry; `Manual Alignment...` was a separate dialog that
deliberately refused to inherit the automatic result, so a bad fit meant
starting over by hand. Neither told you whether the alignment was any
good, and neither produced anything durable beyond an in-memory result.

Fusion -> Alignment Wizard... now covers all of it in three steps:

  1. Correlate. Reference angle, DC threshold, source, rotation seed and
     sign, search window, coarse step and fine grid are all on the page,
     and Run/Re-run is repeatable. Angles start pre-rotated from the
     stage angles in the file, so the page is informative before any
     correlation runs. The verdict is a picture: every angle's DC mask
     reprojected onto the shared canvas and summed, coloured by how many
     angles cover each pixel, so a good alignment reads as one saturated
     plateau and a bad one as a fringe of low-count halos. A per-angle
     fit table flags angles that did not register or that disagree with
     their stage angle by more than a degree.
  2. Crop. An axis-aligned rectangle on that canvas, with numeric
     canvas-pixel boxes synced both ways and a live size estimate.
     "Fit to full overlap" uses a largest-rectangle sweep rather than a
     bounding box: the overlap region of several rotated scans is roughly
     a disc, whose bounding box has corners no angle covers.
  3. Save. Writes the aligned, cropped stack to a new .sras.

Because the wizard replaces both actions it absorbs the old dialog's
by-eye nudge editor — without it, a scan the search cannot fit would
have no fallback at all. ManualAlignmentDialog is therefore deleted
rather than left orphaned, and its tests move to the wizard.

Two bugs found while driving it end to end and fixed here: QSpinBox
setRange clamps and emits valueChanged, which committed a 1x1 crop
before the default preset could run; and the mm round trip returns an
exact pixel boundary as 11.000000000000002, so a bare ceil() added a
spurious column on every rectangle edit.

Verified: 103 pass, the 6 test_stored_cache failures are pre-existing on
main, and tools/check_equivalence.py is byte-identical to main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 23:00:59 -05:00
Thomas Ales 6d0e30b9ce Add aligned/cropped .sras export and the compute pieces behind it
The alignment machinery never modified a scan: it produced an
AlignmentResult and every consumer resampled on the fly. That is right
for a viewer but means the aligned stack cannot leave the process, so no
other tool can read it and reopening the scan redoes the registration.

sras_align_export.write_aligned_sras bakes an alignment into a new v6
file: each angle is gathered onto the shared canvas by nearest neighbour,
so every output angle shares one grid and the file opens already aligned.
Registering the export against itself returns identity, which is the test
that pins the whole index chain.

Three details that are easy to get wrong and are now covered:

  * Rounding must be floor(x + 0.5), not np.rint. scipy's order=0 rounds
    halves away from zero, and the canvas is snapped to the reference's
    pixel grid, so exact halves are common rather than hypothetical.
  * Out-of-bounds must be tested on the fractional coordinate against
    [0, n-1], not on the rounded index, or a one-pixel rim gets real data
    everywhere the Aligned View shows padding.
  * ...but with a tolerance, because the mm-space affine chain lands an
    exactly-integer transform a few times 1e-13 off. A bare >= 0 drops
    the *reference* angle's entire first row and last column.

Padding is the per-channel ADC code nearest 0 mV, not zero: zero ADC
decodes to ~+100 mV on real calibration and would masquerade as sample.

Source rows are served from sliding in-RAM bands. A rotated angle maps
one output row to a diagonal across the source, so indexing a memmap in
output order refaults nearly the whole angle per row — terabytes of
paging for a gigabyte of data.

Also in compute: crop_alignment_result (a crop is pure index translation,
so it folds into the affine's offset rather than becoming a second
transform), overlap_stats, largest_rect_at_least for a crop that stays
inside the overlap region, and seed/sign/refine knobs on
register_angle_to_reference whose defaults leave existing behaviour and
tests byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 22:34:18 -05:00
Thomas Ales 3989c2a1b8 Implement row-averaged FFT feature for same-row SNR cleanup
Add row-averaged FFT feature with configurable window size (row_avg_n
parameter) for improved signal-to-noise ratio on noisy scans. Includes:

- Gaussian-weighted same-row neighbor averaging (never crosses rows)
- Masked/renormalized convolution handling for edge cases and masked samples
- Cache format v2 with row_avg_n tracking to prevent silent cache mismatches
- GUI dialog option for row-average window configuration
- Comprehensive tests validating kernel properties, background subtraction
  invariance, and cache dispatch

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-08-07 21:50:05 -05:00
Thomas Ales eecb0e3a82 Merge remote-tracking branch 'origin/refactor/parallel-and-dedup' 2026-08-06 21:20:58 -05:00
Thomas Ales 1a8187138f pre merge cleanup 2026-08-06 21:20:35 -05:00
Thomas Ales 8154c57066 Move design essays to docs/design.md, leave pointers
The three block essays in sras_compute.py — memory budget / row
chunking, angle-alignment coordinate frames, and the sidecar placement
+ schema history — move to docs/design.md (joined by a new section on
the zoom FFT peak search), each replaced by a 2-4 line pointer. The
save_manual_alignment docstring no longer restates the JSON schema its
own eight lines of code construct.

Load-bearing trap notes (normalization=None, subpixel-score mixing,
memmap lazy reads) stay in place. Zero code changes — golden-hash diff
verified empty.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 11:01:49 -05:00
Thomas Ales f0f622b9ab Dedup viewer Qt boilerplate after the split
- One _apply_alignment_result() replaces the five copies of the
  cache-clear / generation-bump / Aligned-View-checkbox dance, and
  _on_manual_alignment_saved/_cleared collapse into a shared
  _manual_alignment_changed.
- QSignalBlocker context managers replace every hand-rolled
  blockSignals(True)/set/blockSignals(False) triple (11 sites).
- _make_dspin() replaces the 11 four-to-seven-line QDoubleSpinBox
  constructions; _axes_extent() replaces the duplicated imshow-extent
  formula; _is_fft_mode() replaces the repeated CH1-mode guard.
- Jobs constants replace the stringly-typed background-job keys.
- The two 160-line widget builders split along their section banners:
  _build_left_panel -> file/info/view/roi groups, the manual-alignment
  _build_ui -> six per-group builders + _connect_controls.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:56:50 -05:00
Thomas Ales 26a34f7436 Split sras_viewer.py into a package (pure move)
The 2790-line module becomes sras_viewer/: common.py (constants +
layout helpers), canvases.py (RoiQuad, ImageCanvas, WaveformCanvas,
ManualAlignOverlayCanvas), dialogs.py (FftOptionsDialog,
ManualAlignmentDialog), main_window.py (SrasViewerWindow + main), with
__init__ re-exporting the public names and __main__ keeping
`python -m sras_viewer` working. pyproject gains a `sras-viewer`
console script.

Code moved verbatim; only import headers are new (pyflakes-clean).
tests/test_gui.py patch targets follow the classes to their new
modules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:47:34 -05:00
Thomas Ales 00a7afade0 Dedup format layer; public accessors replace private reach-throughs
- SrasFile: _parse_v6 now retains per-angle x_delta and the verbatim
  preamble/background byte spans, and gains public data_offset,
  y_pos_per_angle, and iter_angle_blocks() (which now owns the ragged
  block-offset walk used three separate places before).
- sras_edit_scans: the 70-line re-parse of the v6 header sections
  (_read_v6_sections/_reread_span/_v6_angle_offsets) collapses into a
  _write_v6 that consumes SrasFile directly — verified byte-identical
  round-trip on v6 int8/int16 and legacy files. Eight print-and-exit
  pairs become _die().
- tools/make_test_sras imports the struct layouts from sras_format and
  the rotation matrix from sras_compute instead of re-declaring them
  (byte assembly stays independent of the reader).
- sras_compute: block_mean_2d, pixel_pitch_mm, nominal_delta_deg made
  public (they were GUI-facing); registration_workers() and
  default_max_workers() wrap the remaining private reach-throughs from
  sras_workers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:40:42 -05:00
Thomas Ales e80717d5c5 Dedup workers: one _PooledWorker base for the three pool fan-outs
DcPrecomputeWorker, Ch4MaskWorker and CrossCorrelateWorker shared the
same submit/as_completed/emit/shutdown skeleton; they are now
constructors plus _plan/_items/_one/_emit hooks. All three inherit the
cancellation-aware shutdown (wait=not stopped, cancel_futures) that
only DcPrecomputeWorker had before, and the per-run budget attributes
are initialised in __init__ instead of appearing mid-run.

Also removes the viewer's write-only _dc_precompute_worker plumbing
(job tracking already owns worker lifetime via _jobs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:32:06 -05:00
Thomas Ales 9a9a2557d6 Persist FFT settings, fix backend naming, parallelise batch DC caching
- FFT backend and pad factor persist across sessions via QSettings
  (IniFormat; tests redirect the settings path for hermeticity).
- The default backend was labelled "NumPy FFT" but always dispatched to
  scipy.fft — rename the canonical value to "scipy" ("numpy" stays as a
  legacy alias) and fix the dialog label.
- cache_file: DC caching fans out over angles via _parallel_map with
  per-angle budgets (the DcPrecomputeWorker pattern); FFT caching stays
  serial per angle because compute_rf_image now parallelises internally
  over blocks. Documented that the v7 FFT cache is natural-resolution
  (pad 1) by design.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:26:50 -05:00
Thomas Ales 11ff3b62e2 Rewrite the FFT peak search: block-parallel zoom refinement
At pad 40 the old path materialised a ~9 GB padded spectrum per row,
which collapsed the chunk planner to one worker and one rfft call with
workers=1 — synthesis ran single-threaded, ~1 hour per angle on real
files.

The padded spectrum is never materialised now. Each block of 512
waveforms gets a coarse rfft at next_fast_len(2*spf); every coarse bin
within 0.7 of its row's max (plus the DC-adjacent window, which coarse
DC suppression would otherwise blind) is refined onto the exact n_fft
grid by a small complex gemm. The selected bin is bit-identical to the
full padded argmax — enforced by test_zoom_identity, a 25-seed fuzz
test over adversarial spectra, and a clean golden-hash diff against the
pre-rewrite baseline across pads {1,2,4,8,40}, masked/unmasked, bg
on/off, int8/int16, and both backends.

Blocks fan out over a persistent thread pool; pyFFTW runs through
per-thread FFTW_MEASURE builder plans with wisdom persisted to
~/.cache/sras-viewer, and threadpoolctl clamps BLAS under the pool.
compute_rf_image(exact=True) (or SRAS_FFT_EXACT=1) keeps the reference
padded path for audits.

tools/bench_fft.py measures: pad 40, 16 cores, 8192x2500 synthetic —
exact serial 717 wf/s -> zoom pool 25100 wf/s (35x, pyFFTW backend;
19x scipy), every variant verified equal to the reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 10:20:14 -05:00
Thomas Ales 007089dd48 Convert test scripts to pytest; extend the equivalence harness
- pyproject.toml replaces sras_viewer_requirements.txt (same pins) and
  adds a dev extra with pytest.
- tools/test_refactor.py, test_alignment.py, test_gui.py become
  tests/test_compute.py, tests/test_alignment.py, tests/test_gui.py with
  assertions preserved verbatim. test_gui.py stays one ordered
  integration sequence over a shared module-scoped window.
- check_equivalence.py: drop the dead pre-refactor monolith shim (and the
  _compute_angle_alignment alias it consumed), extend the pad sweep to
  (1, 2, 4, 8, 40), add legacy-v4 and big-endian int16 legs (new bps=2
  option in make_test_sras) so the padded FFT path and the >i2 memmap
  path are in the baseline before the FFT rewrite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 09:45:35 -05:00
Thomas Ales dc513e3fd0 Housekeeping: extend .gitignore, drop stale v2-v5 format docs
SRAS_FORMAT.md/.html documented only v2-v5 and were superseded by
scan_format.md in July. Ignore .DS_Store, screenshots, pytest cache,
and test scratch outputs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 09:36:02 -05:00
Thomas Ales d5028db445 Add scan-editing CLI and alignment tests; extend Manual Alignment correlation
Continues the Manual Alignment work: refines the FFT cross-correlation and
mask handling, adds sras_edit_scans.py (drop/renumber bad angle scans),
tools/test_alignment.py (registration ground-truth suite), and a rotating
test fixture in make_test_sras.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 09:35:30 -05:00
tka b0bfe6e8c6 Merge pull request 'Refactor/parallel and dedup' (#1) from refactor/parallel-and-dedup into main
Reviewed-on: #1
2026-07-31 15:20:33 -05:00
Thomas Ales bbb075ff34 Add FFT cross-correlation to Manual Alignment, with tunable options
Manual mode previously offered only Auto De-rotate (rotation from known
scan angles) plus keyboard nudging, so every angle's translation had to
be found entirely by eye — fine for a small correction, unworkable for
the tens-of-mm scatter a real many-angle scan can have between angles
before any translation search runs at all.

Add an "Auto Cross-Correlate (vs Reference)" action that, for every
non-reference angle, sets rotation to the known analytic angle (same as
Auto De-rotate) and translation to the FFT-phase-correlation best fit
against angle #0 (always the reference/ground truth). This is meant to
land every angle's outline roughly stacked on the others so keyboard
nudging only has to make small corrections afterward, per the intended
workflow: cross-correlate first, then nudge.

Exposes two tunable options, since real data may correlate better one
way than the other: "Correlate on" (raw CH4 signal, minus its own
minimum -- the default, robust to a shared threshold not suiting every
angle's real signal level -- or the thresholded binary mask), and
"Search margin" (how far a shift the phase correlation searches before
wraparound bias becomes a risk).

Runs on a background thread (new CrossCorrelateWorker in
sras_workers.py) since correlating many high-resolution angles can take
long enough to visibly freeze the dialog otherwise.

The underlying per-angle correlation math is factored out of
compute_angle_alignment's Step 2 into a new, reusable
correlate_translation_mm, so the existing automatic Fusion -> Angle
Alignment action and the new manual button now share one implementation
instead of two copies of the same phase-correlation logic.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 11:31:14 -05:00
Thomas Ales 52e91d46fd Stop seeding Manual Alignment translation from the automatic result
The dialog was seeding rotation_deg/shift_mm from self._alignment_result
whenever it was non-None, with no way to tell whether that result came
from a user's own manual Save or from the automatic Fusion -> Angle
Alignment run. The automatic path's translation comes from FFT phase
correlation - the exact thing manual mode exists to work around - so
opening Manual Alignment after running the automatic pass silently
inherited its unreliable shifts on top of the (correct) analytic
rotation, reproducing the same scattered-scan symptom under a "manual"
label.

Manual mode should always start from identity (each angle's centroid
already coincides with every other's, per the prior pivot fix) unless
the user previously used this same dialog to save their own deliberate
translation - which is exactly what the sidecar file already records.
Seed exclusively from load_manual_alignment() and drop the
self._alignment_result branch entirely; the automatic result is never
an appropriate seed for a manual editing session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 11:07:55 -05:00
Thomas Ales 8312668c02 Make alignment pivot independent of the DC mask threshold
The pivot was still derived from the binary CH4 >= dc_threshold_mv mask,
so a threshold that happens to leave a real angle's mask empty (signal
levels vary scan to scan across a many-angle acquisition) silently fell
back to the raw scan-window bbox center — reproducing the exact "aligned
to the scan window, not the sample" scatter the centroid pivot exists to
fix, without any visible error.

Replace the binary-mask centroid with an intensity-weighted centroid of
the continuous CH4 signal, which is always well-defined regardless of the
RF-mask threshold in effect. The threshold still controls what the manual
dialog's overlay and CH1 masking show; it no longer has any bearing on
where alignment pivots.

Also bump the sidecar schema version: a file saved before today's pivot
and rotation-sign fixes stores numbers for a since-corrected transform, so
loading it unchanged would reintroduce the same scatter. Old sidecars are
now treated as absent rather than silently reused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 09:58:15 -05:00
Thomas Ales ca0c736c28 Fix alignment rotation pivot and sign convention
Rotation was pivoting on each angle's raw scan-window bbox center, which
ties alignment to wherever the window happened to sit in microscope/global
XY space rather than to the sample itself, leaving a residual orbital drift
between angles. Pivot on each angle's own CH4-threshold mask centroid
instead (its own sample footprint), for both the automatic phase-
correlation path and the manual dialog's Auto De-rotate/live preview.

Also negate the rotation used everywhere (_theta_deg): the GR stage's
reported angle increases in the opposite rotational sense from this
module's CCW math convention, so de-rotating by the raw angles_deg delta
was making misalignment worse rather than better.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 09:36:12 -05:00
Thomas Ales 3c835f4592 Add manual angle alignment mode with overlay, keyboard nudge, and save/clear
Automatic alignment's phase-correlation translation search is unreliable, so
add a Fusion -> Manual Alignment dialog: every angle's CH4 threshold mask
overlaid at once with distinct colors/opacity, a selector for the active
angle, arrow keys to nudge translation and Q/E to nudge rotation, an Auto
De-rotate button that snaps to the known scan angle, and Save/Clear
controls backed by a JSON sidecar that's restored automatically on reload.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 09:19:17 -05:00
Thomas Ales 013c7739a1 Fix clipped and overlapping controls in the side panels
The left panel was a plain fixed-width widget sized by the window, so once
the Scan Info notes grew past the window height Qt compressed every control
below its minimum: the DC threshold spin box was sliced in half, the ADC
hint collided with it, and the cache note was cut off mid-line.  Word-wrapped
labels made it worse by advertising a single-line minimum height, which
clipped the filename and the format/cache notes.

- Put both side panels in fixed-width QScrollAreas so a short window scrolls
  instead of crushing the controls.
- Add _wrap_label(), which enables height-for-width so wrapped labels report
  their real height; use it for the filename, Scan Info rows, format/cache
  notes, DC-precompute status, ADC hint and ROI readouts.
- Replace the ad-hoc QHBoxLayout label/field rows with a shared QFormLayout
  (Angle, Channel, DC threshold, Grating size, Colormap, min, max) so labels
  align and fields keep a usable minimum width.
- Add _group() for consistent group-box margins; widen the left column
  260 -> 288 px so the channel combo and checkbox text stop eliding.
- Keep the canvases usable: non-collapsible splitter, minimum canvas heights
  and a 960x560 window minimum.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 08:18:57 -05:00
Thomas Ales 9a7cc396e7 Bound nested parallelism, make compute cancellable, harden batch pool
Three defects found by testing the parallel paths end to end.

Nested parallelism was unbounded. DcPrecomputeWorker and the alignment
driver both parallelise over angles, and each angle's compute_dc_image
then parallelised over rows again: 14 x 14 threads, and — worse — every
angle sized its chunk against the *whole* memory budget, so worst-case
live buffers were ~13.9 GB against a 1 GB budget. Capping the inner
worker count alone does not fix it; the chunk is still sized against the
full budget. plan_angle_level now hands each angle both max_workers=1
and its share of the budget, which bounds the real file at 1002 MB.

Chunk planning could not produce concurrency at all. _plan_chunks sized
chunk_rows first, and _chunk_rows_for spends whatever budget it is given
— so a single chunk consumed the lot and budget//chunk_bytes came back
as one worker. It now picks the worker count first and sizes the chunk to
it, giving 7-16 workers on the real file instead of 1.

Cancellation was too coarse to shut down. stop() was only checked between
angles, and one angle of the real scan is ~40 s, so closing the window
sat through it and returned with threads still running. should_stop is
now polled per row chunk and closeEvent signals every worker before
waiting: close went from 4.0 s with two live threads to 1.04 s with both
finished. A cancelled ComputeWorker emits None so a half-filled image is
never cached.

Also: BatchCacheWorker no longer reports every file as failed when the
process pool itself dies (unguarded __main__, frozen build, sandbox) —
it retries those files in-process. Batches below 512 MB skip the pool
entirely, since interpreter startup would otherwise make small jobs
slower. cache_file rejects an unknown mode instead of silently treating
it as "fft".

Measured on the real 496 GB scan, angle 3, 32 rows:
  DC   2.94s -> 0.75s   RF  6.10s -> 1.99s   peak RSS 1.40 -> 1.93 GB
Batch convert, 24 files / 1 GB: 4.03s -> 2.36s.
Memory budget defaults to 1024 MB (SRAS_MEM_BUDGET_MB), the measured knee.

Testing: tools/test_refactor.py (70 checks: cache round-trip and block
carry-forward, partial-cache handling, parallel-vs-serial identity,
no-mask path, ROI mask vs full-grid, v2/v3/v4 parsing, sras_average
round-trip incl. remainder handling) and tools/test_gui.py (46 checks
driving the real widgets and workers headlessly). check_equivalence.py
still reports all 167 outputs identical to ed0eba4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 00:09:46 -05:00
Thomas Ales 55a4c5e42f Split into four modules, deduplicate, and parallelize the compute paths
Structure
  sras_format.py   parsing/writing, calibration, axes  (numpy + struct)
  sras_compute.py  DC/FFT images, alignment, batch cache  (+ scipy)
  sras_workers.py  Qt background workers
  sras_viewer.py   ROI, canvases, dialog, main window

format+compute import in 0.59s with no Qt or matplotlib (vs 2.96s for the
full app), which is what makes a spawn-based process pool worth using.

Deduplication
  - SrasFile.cal() and compute.dc_image_mv() replace the ADC->mV
    calibration incantation that appeared at ten call sites.
  - _read_preambles/_read_background/_set_calibration are shared by the
    legacy and v6 parsers instead of duplicated.
  - v5 PREC images are normalised into the same ragged per-angle list
    v6/v7 uses, removing four isinstance() branches and shortening
    compute_rf_image's fast path.
  - _run_worker replaces five copies of the QThread setup and five
    near-identical teardown methods; the two lifetime hazards they
    guarded against are now documented once, authoritatively.
  - _on_channel_changed defers to _update_controls_enabled rather than
    re-deriving the same six enable rules.
  - _corners_in_ref_frame, _CHANNEL_DISPLAY dict, unified progress-dialog
    helper, dead _on_roi_angle_edited stub removed.
  - sras_average.py builds on SrasFile instead of carrying a second copy
    of the header format, and streams per angle instead of loading the
    whole file (was a ~3x file-size peak).
  Executable lines: 2390 -> 2254, despite adding all of the below.

Parallelism
  - compute_rf_image/compute_dc_image map row chunks over a thread pool.
    Worker count is derived from the memory budget rather than the core
    count: on a large scan chunk_rows is already floored at 1 row (~75 MB
    of float32 at 7507x2500), so only the worker count can bound peak RAM.
  - DcPrecomputeWorker computes angles on a pool, emitting each result
    from its own QThread as it lands.
  - BatchCacheWorker runs one process per file via cache_file(); only
    paths and scalars cross the boundary. Per-process thread counts are
    divided so the two levels don't oversubscribe. Drops the old pass 1,
    which fully parsed every file just to weight a progress bar.
  - dc_threshold_mv=None means "no mask" and skips the CH4 read entirely
    — the batch FFT job previously read all of CH4 to compare against a
    threshold of -1e9.
  - Alignment mask and correlation stages map over angles; fft2/ifft2 use
    workers=-1.

Fixes found on the way
  - A partially-cached v5 file showed an all-zero image for uncached
    angles: the fast-path check was file-wide, not per-angle.
  - v7 cached images were read-only big-endian views; now native float32.

Verified: tools/check_equivalence.py produces byte-identical hashes for
167 outputs (DC/FFT across channels, angles, bg-sub, pad factors and
thresholds, plus the full alignment result) against ed0eba4, on both
synthetic files and a real 496 GB 17-angle v6 scan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 23:38:08 -05:00
Thomas Ales 95c273dce5 Add test tooling: synthetic .sras generator and equivalence harness
tools/make_test_sras.py writes small v6 files with per-angle-varying
geometry, distinct per-channel calibration, and deterministic waveform
content (predictable FFT peak per pixel, predictable DC mean per pixel).

tools/check_equivalence.py hashes DC/FFT/alignment outputs across
channels, angles, bg-sub, pad factors, and thresholds. It imports from
either the monolith or the split modules, so the same script captures
both sides of a refactor. Hashes canonicalise to native float64 so a
dtype or byte-order change that preserves values is not a false failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 23:25:45 -05:00
Thomas Ales ed0eba4bae Add v7 format with on-disk DC/FFT cache, replace v5 pre-process with Convert menu
Scans come off the scope as v6; a new trailing CACH section (independent
SDCB/SFFT blocks, sized per-angle from the existing geometry table) lets
computed DC and FFT images be cached in place, bumping the file to v7 the
first time either is stored. The Convert menu's "Batch Compute DC and
Store"/"Batch Compute FFT and Store" actions run this across multiple
files in the background. compute_rf_image and the DC compute workers now
reuse cached v7 data instead of recomputing it.

Removes "Pre-process and Save as v5", which never supported v6 sources
and is superseded by in-place v7 caching. Legacy v2-v5 reading, including
v5's PREC section, is unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-30 22:27:56 -05:00
Thomas Ales a5770491d5 Add Fusion menu with Angle Alignment (rotation+translation registration)
Adds a background worker that aligns every scan angle onto one shared,
zero-padded canvas using a rigid transform only (no scaling): rotation
is taken analytically from the known scan angle, and only the residual
translation is found via FFT phase correlation of each angle's
binarized CH4 dc-mask. An "Aligned View" toggle then redisplays the
currently selected angle/channel resampled onto that shared canvas,
with ROI, CSV export, and waveform-click inspection all kept working.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-30 21:57:39 -05:00
Thomas Ales f745efe7e0 Skip FFT compute and disk I/O for below-threshold pixels in RF images
Previously the FFT was always run unmasked (dc_threshold_mv=-1e9) so the
result could be cached independent of the threshold, with masking applied
afterward as a display-time step. That meant every pixel's CH1 waveform
was read and FFT'd even when most of the scan is background — wasted work
now that the DC map (and its threshold mask) is already known ahead of
time via the DC precompute cache.

compute_rf_image now takes the real threshold and an optional precomputed
dc4_mv array (reused from the DC cache, avoiding a redundant CH4 read),
and skips the FFT for masked-out pixels entirely, same as before this
threshold-caching detour was introduced. The masked-out CH1 samples are
also never read from disk: the boolean valid-pixel mask is applied to the
raw memmap slice before any dtype conversion, so numpy only pages in the
bytes for pixels that pass the threshold.

Threshold is therefore back to being part of the FFT cache key (changing
it now decides which pixels get computed at all, so it can't be satisfied
from a cache built for a different threshold) — but the recompute it
triggers reuses the cached DC4 image and skips both the FFT and the I/O
for masked pixels, so it's much cheaper than the original full-image
compute. Grating and colormap remain pure post-processing with no
recompute.

Verified bit-identical output against the un-optimized reference path on
synthetic data (including scattered, non-row-aligned masking), and timed
on the real 532 GB / 17-angle file: angle 0's FFT (75.2% of pixels above
the default 50 mV threshold) went from ~121s (fully unmasked) to 114s
(FFT skipped, I/O not skipped) to 90.2s with this change (both skipped).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-30 20:59:02 -05:00
Thomas Ales 158f60d2a7 Stop recomputing images on display-only setting changes; cache and lazily compute
Colormap, DC threshold, velocity grating, and FFT pad-factor changes were
all routing through a full recompute, which is unworkable on large v6
scans where a single angle's FFT can take minutes. These are now applied
as cheap post-processing on cached data instead:

- ComputeWorker computes CH1/Velocity as an unmasked raw FFT plus its DC4
  mask image, cached per (angle, bg_sub, n_fft). Threshold masking and
  grating scaling are applied to the cached array on redraw, so neither
  needs a recompute; colormap changes only touch matplotlib.
- New DcPrecomputeWorker fills a per-angle DC (CH3/CH4) cache in the
  background right after load, since DC images are cheap (mean, no FFT)
  and make switching angles instant once cached. Progress is shown in the
  Scan Info panel.
- New _refresh_display()/_show_image_now() route every settings-changed
  handler through the cache first, falling back to the existing threaded
  _start_compute() (with a progress popup, now FFT- vs DC-specific) only
  on a genuine cache miss.
- File load now defaults to a DC channel instead of CH1/FFT, so opening a
  large file shows something in seconds instead of minutes.

Verified against a real 532 GB / 17-angle file: DC compute for one angle
takes ~100s+ (I/O-bound over USB) the first time, but switching to an
already-cached angle takes 0.4s with no compute thread spun up at all.

Audited every widget signal connection for this pass; the spinboxes
already correctly used editingFinished rather than valueChanged — the
recompute-on-every-change bug was in the handlers, not the event type.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-30 20:36:44 -05:00
35 changed files with 15087 additions and 3660 deletions
+7
View File
@@ -1,2 +1,9 @@
__pycache__/ __pycache__/
.venv/ .venv/
.DS_Store
.pytest_cache/
*.png
*.sras
baseline*.txt
after*.txt
sras_viewer.egg-info/
-761
View File
@@ -1,761 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>SRAS File Format Specification</title>
<style>
/* From extension vscode.github */
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
.vscode-dark img[src$=\#gh-light-mode-only],
.vscode-light img[src$=\#gh-dark-mode-only],
.vscode-high-contrast:not(.vscode-high-contrast-light) img[src$=\#gh-light-mode-only],
.vscode-high-contrast-light img[src$=\#gh-dark-mode-only] {
display: none;
}
</style>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Microsoft/vscode/extensions/markdown-language-features/media/markdown.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Microsoft/vscode/extensions/markdown-language-features/media/highlight.css">
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe WPC', 'Segoe UI', system-ui, 'Ubuntu', 'Droid Sans', sans-serif;
font-size: 14px;
line-height: 1.6;
}
</style>
<style>
.task-list-item {
list-style-type: none;
}
.task-list-item-checkbox {
margin-left: -20px;
vertical-align: middle;
pointer-events: none;
}
</style>
<style>
:root {
--color-note: #0969da;
--color-tip: #1a7f37;
--color-warning: #9a6700;
--color-severe: #bc4c00;
--color-caution: #d1242f;
--color-important: #8250df;
}
</style>
<style>
@media (prefers-color-scheme: dark) {
:root {
--color-note: #2f81f7;
--color-tip: #3fb950;
--color-warning: #d29922;
--color-severe: #db6d28;
--color-caution: #f85149;
--color-important: #a371f7;
}
}
</style>
<style>
.markdown-alert {
padding: 0.5rem 1rem;
margin-bottom: 16px;
color: inherit;
border-left: .25em solid #888;
}
.markdown-alert>:first-child {
margin-top: 0
}
.markdown-alert>:last-child {
margin-bottom: 0
}
.markdown-alert .markdown-alert-title {
display: flex;
font-weight: 500;
align-items: center;
line-height: 1
}
.markdown-alert .markdown-alert-title .octicon {
margin-right: 0.5rem;
display: inline-block;
overflow: visible !important;
vertical-align: text-bottom;
fill: currentColor;
}
.markdown-alert.markdown-alert-note {
border-left-color: var(--color-note);
}
.markdown-alert.markdown-alert-note .markdown-alert-title {
color: var(--color-note);
}
.markdown-alert.markdown-alert-important {
border-left-color: var(--color-important);
}
.markdown-alert.markdown-alert-important .markdown-alert-title {
color: var(--color-important);
}
.markdown-alert.markdown-alert-warning {
border-left-color: var(--color-warning);
}
.markdown-alert.markdown-alert-warning .markdown-alert-title {
color: var(--color-warning);
}
.markdown-alert.markdown-alert-tip {
border-left-color: var(--color-tip);
}
.markdown-alert.markdown-alert-tip .markdown-alert-title {
color: var(--color-tip);
}
.markdown-alert.markdown-alert-caution {
border-left-color: var(--color-caution);
}
.markdown-alert.markdown-alert-caution .markdown-alert-title {
color: var(--color-caution);
}
</style>
</head>
<body class="vscode-body vscode-light">
<h1 id="sras-file-format-specification">SRAS File Format Specification</h1>
<p><strong>Format family:</strong> <code>.sras</code><br>
<strong>Byte order:</strong> Big-endian (network byte order) throughout, unless noted.<br>
<strong>Version history:</strong> v2 (baseline), v3 (scope calibration), v4 (background waveform), v5 (precomputed images + guaranteed frame count).</p>
<hr>
<h2 id="table-of-contents">Table of Contents</h2>
<ol>
<li><a href="#overview">Overview</a></li>
<li><a href="#type-notation">Type notation</a></li>
<li><a href="#version-history">Version history</a></li>
<li><a href="#file-structure">File structure</a>
<ul>
<li><a href="#1-fixed-header-43-bytes-all-versions">Fixed header (all versions)</a></li>
<li><a href="#2-angle-table-all-versions">Angle table (all versions)</a></li>
<li><a href="#3-row-position-table-all-versions">Row position table (all versions)</a></li>
<li><a href="#4-channel-preambles-v3">Channel preambles (v3+)</a></li>
<li><a href="#5-background-waveform-v4">Background waveform (v4+)</a></li>
<li><a href="#6-waveform-data-all-versions">Waveform data (all versions)</a></li>
<li><a href="#7-prec-section-v5">PREC section (v5)</a></li>
</ul>
</li>
<li><a href="#derived-quantities">Derived quantities</a></li>
<li><a href="#adc-calibration">ADC calibration</a></li>
<li><a href="#waveform-data-layout-detail">Waveform data layout detail</a></li>
<li><a href="#size-reference">Size reference</a></li>
<li><a href="#compatibility-notes">Compatibility notes</a></li>
</ol>
<hr>
<h2 id="overview">Overview</h2>
<p>An SRAS file stores the raw RF waveforms captured during a Surface-acoustic-wave Resonance And Spectroscopy (SRAS) scan, along with the scan geometry and scope calibration metadata needed to interpret them.</p>
<p>A scan consists of one or more <strong>angles</strong> (rotation positions of the sample), each containing a 2-D raster of <strong>rows</strong> × <strong>frames</strong>. At every pixel, <code>n_channels</code> waveforms of <code>samples_per_frame</code> ADC counts are stored. Channel order is fixed:</p>
<table>
<thead>
<tr>
<th>Index</th>
<th>Hardware channel</th>
<th>Signal</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>CH1</td>
<td>RF acoustic packet (AC-coupled)</td>
</tr>
<tr>
<td>1</td>
<td>CH3</td>
<td>Bias A — DC mean used for masking</td>
</tr>
<tr>
<td>2</td>
<td>CH4</td>
<td>Bias B — DC mean used for masking</td>
</tr>
</tbody>
</table>
<hr>
<h2 id="type-notation">Type notation</h2>
<table>
<thead>
<tr>
<th>Symbol</th>
<th>C type</th>
<th>Size</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>u8</code></td>
<td><code>uint8_t</code></td>
<td>1 byte</td>
<td>unsigned</td>
</tr>
<tr>
<td><code>u16</code></td>
<td><code>uint16_t</code></td>
<td>2 bytes</td>
<td>big-endian</td>
</tr>
<tr>
<td><code>u32</code></td>
<td><code>uint32_t</code></td>
<td>4 bytes</td>
<td>big-endian</td>
</tr>
<tr>
<td><code>i8</code></td>
<td><code>int8_t</code></td>
<td>1 byte</td>
<td>signed, used for ADC samples when <code>bytes_per_sample == 1</code></td>
</tr>
<tr>
<td><code>i16</code></td>
<td><code>int16_t</code></td>
<td>2 bytes</td>
<td>big-endian signed, used when <code>bytes_per_sample == 2</code></td>
</tr>
<tr>
<td><code>f32</code></td>
<td><code>float</code></td>
<td>4 bytes</td>
<td>big-endian IEEE 754 single</td>
</tr>
<tr>
<td><code>f64</code></td>
<td><code>double</code></td>
<td>8 bytes</td>
<td>big-endian IEEE 754 double</td>
</tr>
<tr>
<td><code>char[N]</code></td>
<td>—</td>
<td>N bytes</td>
<td>raw bytes, no null terminator unless noted</td>
</tr>
<tr>
<td><code>utf8[N]</code></td>
<td>—</td>
<td>N bytes</td>
<td>UTF-8 string, length-prefixed (see preamble section)</td>
</tr>
</tbody>
</table>
<hr>
<h2 id="version-history">Version history</h2>
<table>
<thead>
<tr>
<th>Version</th>
<th>Added</th>
</tr>
</thead>
<tbody>
<tr>
<td>2</td>
<td>Baseline: fixed header, angle table, row table, raw waveform data. No scope calibration (fallback constants used by readers).</td>
</tr>
<tr>
<td>3</td>
<td>Per-channel Tektronix WFMOutpre preamble strings carrying YMULT / YOFF / YZERO calibration.</td>
</tr>
<tr>
<td>4</td>
<td>Background waveform section: one CH1 reference shot subtracted from each CH1 frame before FFT.</td>
</tr>
<tr>
<td>5</td>
<td><strong>(this document)</strong> Version byte incremented to 5. <code>n_frames_hdr</code> is now the <em>actual</em> acquired frame count (authoritative). PREC section appended after waveform data with precomputed FFT-peak and DC images for instant re-display.</td>
</tr>
</tbody>
</table>
<blockquote>
<p><strong>v2 note:</strong> Version 1 is not defined; version 2 is the lowest observed in the field.</p>
</blockquote>
<hr>
<h2 id="file-structure">File structure</h2>
<pre><code>┌─────────────────────────────────────────────┐
│ 1. Fixed header (43 bytes) │ all versions
├─────────────────────────────────────────────┤
│ 2. Angle table (n_angles × 4 bytes)│ all versions
├─────────────────────────────────────────────┤
│ 3. Row position table (n_rows × 4 bytes)│ all versions
├─────────────────────────────────────────────┤
│ 4. Channel preambles (variable) │ v3+
├─────────────────────────────────────────────┤
│ 5. Background waveform (variable) │ v4+
├─────────────────────────────────────────────┤
│ 6. Waveform data (variable) │ all versions
├─────────────────────────────────────────────┤
│ 7. PREC section (variable) │ v5 only
└─────────────────────────────────────────────┘
</code></pre>
<hr>
<h3 id="1-fixed-header-43-bytes-all-versions">1. Fixed header (43 bytes, all versions)</h3>
<table>
<thead>
<tr>
<th>Offset</th>
<th>Size</th>
<th>Type</th>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>4</td>
<td><code>char[4]</code></td>
<td><code>magic</code></td>
<td><code>SRAS</code> (ASCII, no null terminator). Reject file if this does not match.</td>
</tr>
<tr>
<td>4</td>
<td>1</td>
<td><code>u8</code></td>
<td><code>version</code></td>
<td>Format version. This document describes version <strong>5</strong>.</td>
</tr>
<tr>
<td>5</td>
<td>2</td>
<td><code>u16</code></td>
<td><code>n_angles</code></td>
<td>Number of scan angles (rotation positions). ≥ 1.</td>
</tr>
<tr>
<td>7</td>
<td>2</td>
<td><code>u16</code></td>
<td><code>n_rows</code></td>
<td>Number of scan rows (Y positions). ≥ 1.</td>
</tr>
<tr>
<td>9</td>
<td>4</td>
<td><code>f32</code></td>
<td><code>x_start_mm</code></td>
<td>X position of the first frame in the first row, in millimetres.</td>
</tr>
<tr>
<td>13</td>
<td>4</td>
<td><code>f32</code></td>
<td><code>x_delta_mm</code></td>
<td>Pre-computed pixel pitch in mm (<code>velocity_mm_s / laser_freq_hz</code>). Provided for convenience; readers should prefer the derived value.</td>
</tr>
<tr>
<td>17</td>
<td>4</td>
<td><code>f32</code></td>
<td><code>velocity_mm_s</code></td>
<td>Scanner stage velocity, mm s⁻¹. Used together with <code>laser_freq_hz</code> to compute pixel pitch.</td>
</tr>
<tr>
<td>21</td>
<td>4</td>
<td><code>f32</code></td>
<td><code>laser_freq_hz</code></td>
<td>Laser repetition rate, Hz.</td>
</tr>
<tr>
<td>25</td>
<td>4</td>
<td><code>u32</code></td>
<td><code>n_frames_hdr</code></td>
<td><strong>v2–v4:</strong> the <em>configured</em> frame count written before acquisition; may exceed actual frames acquired (use file-size arithmetic to obtain the true count). <strong>v5:</strong> the <em>actual</em> acquired frame count — authoritative; readers must not re-derive it from file size.</td>
</tr>
<tr>
<td>29</td>
<td>4</td>
<td><code>u32</code></td>
<td><code>samples_per_frame</code></td>
<td>ADC samples per waveform (<code>spf</code>).</td>
</tr>
<tr>
<td>33</td>
<td>8</td>
<td><code>f64</code></td>
<td><code>sample_rate_hz</code></td>
<td>Oscilloscope sample rate, Hz (e.g. 5 × 10⁹ for 5 GS/s).</td>
</tr>
<tr>
<td>41</td>
<td>1</td>
<td><code>u8</code></td>
<td><code>bytes_per_sample</code></td>
<td>ADC word size: <code>1</code> → <code>i8</code>, <code>2</code> → <code>i16</code> (big-endian).</td>
</tr>
<tr>
<td>42</td>
<td>1</td>
<td><code>u8</code></td>
<td><code>n_channels</code></td>
<td>Number of channels per frame. Currently always <code>3</code>.</td>
</tr>
</tbody>
</table>
<hr>
<h3 id="2-angle-table-all-versions">2. Angle table (all versions)</h3>
<p>Immediately follows the fixed header.</p>
<pre><code>n_angles × f32 — scan angle in degrees
</code></pre>
<p>Each entry is a big-endian <code>f32</code> giving the sample rotation angle in degrees at which that angle index was acquired.</p>
<hr>
<h3 id="3-row-position-table-all-versions">3. Row position table (all versions)</h3>
<p>Immediately follows the angle table.</p>
<pre><code>n_rows × f32 — Y position of each row, in millimetres
</code></pre>
<hr>
<h3 id="4-channel-preambles-v3">4. Channel preambles (v3+)</h3>
<p>One entry per channel, in channel-index order (CH1 first).</p>
<pre><code>for each channel:
u16 preamble_length — byte count of the UTF-8 string that follows
utf8[N] preamble — Tektronix WFMOutpre string
</code></pre>
<p>The preamble is the oscilloscope's <code>WFMOutpre</code> response string. Readers extract the following keys (case-insensitive, space-separated value):</p>
<table>
<thead>
<tr>
<th>Key</th>
<th>Stored unit</th>
<th>Conversion to mV</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>YMULT</code></td>
<td>V count⁻¹</td>
<td>multiply by 1000</td>
</tr>
<tr>
<td><code>YOFF</code></td>
<td>ADC counts</td>
<td>used directly</td>
</tr>
<tr>
<td><code>YZERO</code></td>
<td>V</td>
<td>multiply by 1000</td>
</tr>
</tbody>
</table>
<p><strong>v2 fallback:</strong> when preambles are absent, readers use:</p>
<ul>
<li><code>YMULT</code> = 1.5625 mV count⁻¹ (50 mV/div, 8 div, 8-bit ADC)</li>
<li><code>YOFF</code> = −87.04 ADC counts (scope position = −2.72 div)</li>
<li><code>YZERO</code> = 0 mV</li>
</ul>
<hr>
<h3 id="5-background-waveform-v4">5. Background waveform (v4+)</h3>
<pre><code>u32 n_bg_samples — number of i8 ADC samples that follow
i8[n_bg] background — one representative CH1 background shot
</code></pre>
<p>The background waveform has the same <code>samples_per_frame</code> length as a normal CH1 waveform. It is subtracted from each CH1 waveform before FFT processing when background subtraction is enabled. When <code>n_bg_samples == 0</code> the section is present but empty.</p>
<hr>
<h3 id="6-waveform-data-all-versions">6. Waveform data (all versions)</h3>
<p>Begins immediately after the fixed header (v2), preambles (v3), or background waveform (v4+). The waveform data is a flat, contiguous array with the following logical shape, stored in row-major (C) order:</p>
<pre><code>waveform_data[n_angles][n_rows][n_channels][n_frames][samples_per_frame]
</code></pre>
<p>Each element is a signed ADC count of size <code>bytes_per_sample</code>:</p>
<ul>
<li><code>bytes_per_sample == 1</code> → <code>i8</code></li>
<li><code>bytes_per_sample == 2</code> → <code>i16</code> big-endian</li>
</ul>
<p><strong>Total byte count:</strong></p>
<pre><code>waveform_bytes = n_angles × n_rows × n_channels × n_frames × samples_per_frame × bytes_per_sample
</code></pre>
<h4 id="index-semantics">Index semantics</h4>
<table>
<thead>
<tr>
<th>Dimension</th>
<th>Range</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>[a]</code></td>
<td>0 … n_angles−1</td>
<td>Scan angle (rotation position)</td>
</tr>
<tr>
<td><code>[r]</code></td>
<td>0 … n_rows−1</td>
<td>Row (Y position); row 0 is the first acquired</td>
</tr>
<tr>
<td><code>[c]</code></td>
<td>0 … n_channels−1</td>
<td>Channel (0=CH1 RF, 1=CH3 Bias A, 2=CH4 Bias B)</td>
</tr>
<tr>
<td><code>[f]</code></td>
<td>0 … n_frames−1</td>
<td>Frame (X position) within the row</td>
</tr>
<tr>
<td><code>[s]</code></td>
<td>0 … spf−1</td>
<td>Sample index within the waveform</td>
</tr>
</tbody>
</table>
<h4 id="frame-count-determination">Frame-count determination</h4>
<ul>
<li><strong>v5:</strong> use <code>n_frames_hdr</code> directly; do not use file-size arithmetic.</li>
<li><strong>v2–v4:</strong> <code>n_frames = floor((file_bytes_after_header_sections) / (bytes_per_sample × n_angles × n_rows × n_channels × samples_per_frame))</code>. Any remainder bytes are a partial trailing row and are discarded.</li>
</ul>
<hr>
<h3 id="7-prec-section-v5">7. PREC section (v5)</h3>
<p>The PREC section is appended immediately after the waveform data and is present if and only if <code>version == 5</code> and the file size exceeds <code>waveform_end_offset</code>.</p>
<pre><code>waveform_end_offset = data_offset + waveform_bytes
</code></pre>
<p>where <code>data_offset</code> is the file offset of the first waveform byte (the byte immediately after the background waveform, or after the angle/row tables for v2 files).</p>
<h4 id="prec-header-8-bytes">PREC header (8 bytes)</h4>
<table>
<thead>
<tr>
<th>Offset (relative)</th>
<th>Size</th>
<th>Type</th>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>4</td>
<td><code>char[4]</code></td>
<td><code>prec_magic</code></td>
<td><code>PREC</code> (ASCII). Absent or wrong magic → ignore section.</td>
</tr>
<tr>
<td>4</td>
<td>1</td>
<td><code>u8</code></td>
<td><code>prec_version</code></td>
<td>PREC format version. Currently <code>1</code>.</td>
</tr>
<tr>
<td>5</td>
<td>1</td>
<td><code>u8</code></td>
<td><code>flags</code></td>
<td>Bitmask (see below).</td>
</tr>
<tr>
<td>6</td>
<td>2</td>
<td><code>u16</code></td>
<td><code>n_stored</code></td>
<td>Number of angle entries that follow. 0 ≤ <code>n_stored</code> ≤ <code>n_angles</code>.</td>
</tr>
</tbody>
</table>
<h5 id="flags-byte">Flags byte</h5>
<table>
<thead>
<tr>
<th>Bit</th>
<th>Mask</th>
<th>Meaning when set</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td><code>0x01</code></td>
<td><code>bg_sub_applied</code> — background waveform was subtracted from CH1 before the FFT when these images were computed.</td>
</tr>
<tr>
<td>1–7</td>
<td>—</td>
<td>Reserved, must be zero on write; readers must ignore.</td>
</tr>
</tbody>
</table>
<h4 id="prec-angle-entries">PREC angle entries</h4>
<p>Repeated <code>n_stored</code> times, in arbitrary angle-index order:</p>
<pre><code>for each stored angle:
u16 angle_idx — index into the angle table (0-based)
f32[n_rows×n_frames] peak_freq_mhz — CH1 FFT peak frequency, MHz, row-major
f32[n_rows×n_frames] dc4_mv — CH4 waveform mean, mV, row-major
f32[n_rows×n_frames] dc3_mv — CH3 waveform mean, mV, row-major
</code></pre>
<p>All image arrays are <code>f32</code> big-endian, stored in row-major order: element <code>[r][f]</code> is at offset <code>(r × n_frames + f) × 4</code> bytes within the array.</p>
<p><strong><code>peak_freq_mhz</code></strong> is computed without any DC-threshold masking (i.e. the FFT is run on every pixel unconditionally). Readers apply the <code>dc4_mv</code> threshold at display time:</p>
<pre><code>pixel is valid ⟺ dc4_mv[r][f] ≥ threshold_mv
display_value = peak_freq_mhz[r][f] if valid, else 0
</code></pre>
<p><strong><code>dc4_mv</code> / <code>dc3_mv</code></strong> are the mean of all ADC samples in the respective channel waveform, converted to millivolts using the channel calibration:</p>
<pre><code>dc_mv = (adc_mean − YOFF) × YMULT + YZERO
</code></pre>
<h4 id="when-readers-must-bypass-the-prec-fast-path">When readers must bypass the PREC fast path</h4>
<p>Readers must fall back to real-time FFT computation (ignoring stored <code>peak_freq_mhz</code>) when any of the following are true:</p>
<ul>
<li>Time-domain gating is active (zeroing samples outside a time window changes peak frequency).</li>
<li>Zero-padding (<code>n_fft ≠ samples_per_frame</code>) is requested (changes bin spacing).</li>
<li>The reader's background-subtraction setting does not match <code>flags.bg_sub_applied</code>.</li>
</ul>
<hr>
<h2 id="derived-quantities">Derived quantities</h2>
<pre><code>pixel_pitch_mm = velocity_mm_s / laser_freq_hz
x_axis_mm[f] = x_start_mm + f × pixel_pitch_mm (f = 0 … n_frames−1)
time_axis_ns[s] = s / sample_rate_hz × 1e9 (s = 0 … spf−1)
freq_axis_mhz[k] = k × sample_rate_hz / (n_fft × 1e6) (k = 0 … n_fft/2)
where n_fft = samples_per_frame unless zero-padding is active
velocity_ms[r][f] = peak_freq_mhz[r][f] × grating_um (grating_um user-supplied)
</code></pre>
<hr>
<h2 id="adc-calibration">ADC calibration</h2>
<p>Convert raw ADC counts to millivolts:</p>
<pre><code>voltage_mv = (adc_count − YOFF) × YMULT_mv + YZERO_mv
</code></pre>
<p>Invert (mV → ADC count):</p>
<pre><code>adc_count = (voltage_mv − YZERO_mv) / YMULT_mv + YOFF
</code></pre>
<p>where <code>YMULT_mv</code> is YMULT in mV count⁻¹ (= scope YMULT in V count⁻¹ × 1000).</p>
<hr>
<h2 id="waveform-data-layout-detail">Waveform data layout detail</h2>
<p>For a scan with <code>n_angles=2</code>, <code>n_rows=3</code>, <code>n_channels=3</code>, <code>n_frames=4</code>, <code>spf=5</code> the layout is:</p>
<pre><code>angle 0
row 0
CH1: [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4]
frame 0 frame 1 frame 2 frame 3
CH3: …(same layout)…
CH4: …(same layout)…
row 1
…
row 2
…
angle 1
…
</code></pre>
<p>The flat byte offset of sample <code>s</code> of frame <code>f</code>, channel <code>c</code>, row <code>r</code>, angle <code>a</code> is:</p>
<pre><code>offset = data_offset
+ (a × n_rows × n_channels × n_frames × spf
+ r × n_channels × n_frames × spf
+ c × n_frames × spf
+ f × spf
+ s)
× bytes_per_sample
</code></pre>
<hr>
<h2 id="size-reference">Size reference</h2>
<p>Approximate sizes for representative scans (<code>bytes_per_sample = 1</code>, <code>n_channels = 3</code>).</p>
<table>
<thead>
<tr>
<th>n_angles</th>
<th>n_rows</th>
<th>n_frames</th>
<th>spf</th>
<th>Waveform data</th>
<th>PREC section</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>500</td>
<td>500</td>
<td>400</td>
<td>300 MB</td>
<td>12 MB</td>
</tr>
<tr>
<td>4</td>
<td>500</td>
<td>500</td>
<td>400</td>
<td>1.2 GB</td>
<td>48 MB</td>
</tr>
<tr>
<td>1</td>
<td>2000</td>
<td>2000</td>
<td>400</td>
<td>4.8 GB</td>
<td>48 MB</td>
</tr>
<tr>
<td>4</td>
<td>2000</td>
<td>2000</td>
<td>400</td>
<td>19.2 GB</td>
<td>192 MB</td>
</tr>
<tr>
<td>8</td>
<td>2000</td>
<td>2000</td>
<td>400</td>
<td>38.4 GB</td>
<td>384 MB</td>
</tr>
<tr>
<td>16</td>
<td>2000</td>
<td>2000</td>
<td>400</td>
<td>76.8 GB</td>
<td>768 MB</td>
</tr>
</tbody>
</table>
<p><strong>PREC section size formula:</strong></p>
<pre><code>prec_bytes = 8 + n_stored × (2 + 3 × n_rows × n_frames × 4)
</code></pre>
<hr>
<h2 id="compatibility-notes">Compatibility notes</h2>
<h3 id="reading-v5-files-with-a-v4-reader">Reading v5 files with a v4 reader</h3>
<p>A v4 reader that only accepts versions <code>{2, 3, 4}</code> will reject a v5 file with an &quot;unsupported version&quot; error. This is intentional: a v4 reader would derive <code>n_frames</code> from the file size, incorrectly including the PREC bytes in the sample count, producing a silently wrong reshape.</p>
<h3 id="producing-v5-files">Producing v5 files</h3>
<p>v5 files are produced by the SRAS viewer's <strong>&quot;Pre-process and Save as v5&quot;</strong> action. The procedure is:</p>
<ol>
<li>Copy the source file (any version) verbatim.</li>
<li>Set <code>version = 5</code> at byte offset 4.</li>
<li>Set <code>n_frames_hdr</code> at byte offset 25 to the actual acquired frame count.</li>
<li>Truncate the copy to <code>data_offset + waveform_bytes</code> (removes any pre-existing stale PREC tail).</li>
<li>Compute <code>peak_freq_mhz</code>, <code>dc4_mv</code>, and <code>dc3_mv</code> for every angle using chunked FFT.</li>
<li>Append the PREC section.</li>
</ol>
<h3 id="partially-written-prec-sections">Partially-written PREC sections</h3>
<p>If <code>n_stored &lt; n_angles</code> (e.g. pre-processing was interrupted), the file is still valid. Readers use stored images for the angles present in the PREC section and fall back to real-time FFT for the remainder. Readers must check <code>angle_idx</code> bounds on each entry and stop parsing on an out-of-range value.</p>
</body>
</html>
-379
View File
@@ -1,379 +0,0 @@
# SRAS File Format Specification
**Format family:** `.sras`
**Byte order:** Big-endian (network byte order) throughout, unless noted.
**Version history:** v2 (baseline), v3 (scope calibration), v4 (background waveform), v5 (precomputed images + guaranteed frame count).
---
## Table of Contents
1. [Overview](#overview)
2. [Type notation](#type-notation)
3. [Version history](#version-history)
4. [File structure](#file-structure)
- [Fixed header (all versions)](#1-fixed-header-43-bytes-all-versions)
- [Angle table (all versions)](#2-angle-table-all-versions)
- [Row position table (all versions)](#3-row-position-table-all-versions)
- [Channel preambles (v3+)](#4-channel-preambles-v3)
- [Background waveform (v4+)](#5-background-waveform-v4)
- [Waveform data (all versions)](#6-waveform-data-all-versions)
- [PREC section (v5)](#7-prec-section-v5)
5. [Derived quantities](#derived-quantities)
6. [ADC calibration](#adc-calibration)
7. [Waveform data layout detail](#waveform-data-layout-detail)
8. [Size reference](#size-reference)
9. [Compatibility notes](#compatibility-notes)
---
## Overview
An SRAS file stores the raw RF waveforms captured during a Surface-acoustic-wave Resonance And Spectroscopy (SRAS) scan, along with the scan geometry and scope calibration metadata needed to interpret them.
A scan consists of one or more **angles** (rotation positions of the sample), each containing a 2-D raster of **rows** × **frames**. At every pixel, `n_channels` waveforms of `samples_per_frame` ADC counts are stored. Channel order is fixed:
| Index | Hardware channel | Signal |
|-------|-----------------|--------|
| 0 | CH1 | RF acoustic packet (AC-coupled) |
| 1 | CH3 | Bias A — DC mean used for masking |
| 2 | CH4 | Bias B — DC mean used for masking |
---
## Type notation
| Symbol | C type | Size | Notes |
|--------|--------|------|-------|
| `u8` | `uint8_t` | 1 byte | unsigned |
| `u16` | `uint16_t` | 2 bytes | big-endian |
| `u32` | `uint32_t` | 4 bytes | big-endian |
| `i8` | `int8_t` | 1 byte | signed, used for ADC samples when `bytes_per_sample == 1` |
| `i16` | `int16_t` | 2 bytes | big-endian signed, used when `bytes_per_sample == 2` |
| `f32` | `float` | 4 bytes | big-endian IEEE 754 single |
| `f64` | `double` | 8 bytes | big-endian IEEE 754 double |
| `char[N]` | — | N bytes | raw bytes, no null terminator unless noted |
| `utf8[N]` | — | N bytes | UTF-8 string, length-prefixed (see preamble section) |
---
## Version history
| Version | Added |
|---------|-------|
| 2 | Baseline: fixed header, angle table, row table, raw waveform data. No scope calibration (fallback constants used by readers). |
| 3 | Per-channel Tektronix WFMOutpre preamble strings carrying YMULT / YOFF / YZERO calibration. |
| 4 | Background waveform section: one CH1 reference shot subtracted from each CH1 frame before FFT. |
| 5 | **(this document)** Version byte incremented to 5. `n_frames_hdr` is now the *actual* acquired frame count (authoritative). PREC section appended after waveform data with precomputed FFT-peak and DC images for instant re-display. |
> **v2 note:** Version 1 is not defined; version 2 is the lowest observed in the field.
---
## File structure
```
┌─────────────────────────────────────────────┐
│ 1. Fixed header (43 bytes) │ all versions
├─────────────────────────────────────────────┤
│ 2. Angle table (n_angles × 4 bytes)│ all versions
├─────────────────────────────────────────────┤
│ 3. Row position table (n_rows × 4 bytes)│ all versions
├─────────────────────────────────────────────┤
│ 4. Channel preambles (variable) │ v3+
├─────────────────────────────────────────────┤
│ 5. Background waveform (variable) │ v4+
├─────────────────────────────────────────────┤
│ 6. Waveform data (variable) │ all versions
├─────────────────────────────────────────────┤
│ 7. PREC section (variable) │ v5 only
└─────────────────────────────────────────────┘
```
---
### 1. Fixed header (43 bytes, all versions)
| Offset | Size | Type | Field | Description |
|--------|------|------|-------|-------------|
| 0 | 4 | `char[4]` | `magic` | `SRAS` (ASCII, no null terminator). Reject file if this does not match. |
| 4 | 1 | `u8` | `version` | Format version. This document describes version **5**. |
| 5 | 2 | `u16` | `n_angles` | Number of scan angles (rotation positions). ≥ 1. |
| 7 | 2 | `u16` | `n_rows` | Number of scan rows (Y positions). ≥ 1. |
| 9 | 4 | `f32` | `x_start_mm` | X position of the first frame in the first row, in millimetres. |
| 13 | 4 | `f32` | `x_delta_mm` | Pre-computed pixel pitch in mm (`velocity_mm_s / laser_freq_hz`). Provided for convenience; readers should prefer the derived value. |
| 17 | 4 | `f32` | `velocity_mm_s` | Scanner stage velocity, mm s⁻¹. Used together with `laser_freq_hz` to compute pixel pitch. |
| 21 | 4 | `f32` | `laser_freq_hz` | Laser repetition rate, Hz. |
| 25 | 4 | `u32` | `n_frames_hdr` | **v2–v4:** the *configured* frame count written before acquisition; may exceed actual frames acquired (use file-size arithmetic to obtain the true count). **v5:** the *actual* acquired frame count — authoritative; readers must not re-derive it from file size. |
| 29 | 4 | `u32` | `samples_per_frame` | ADC samples per waveform (`spf`). |
| 33 | 8 | `f64` | `sample_rate_hz` | Oscilloscope sample rate, Hz (e.g. 5 × 10⁹ for 5 GS/s). |
| 41 | 1 | `u8` | `bytes_per_sample` | ADC word size: `1` → `i8`, `2` → `i16` (big-endian). |
| 42 | 1 | `u8` | `n_channels` | Number of channels per frame. Currently always `3`. |
---
### 2. Angle table (all versions)
Immediately follows the fixed header.
```
n_angles × f32 — scan angle in degrees
```
Each entry is a big-endian `f32` giving the sample rotation angle in degrees at which that angle index was acquired.
---
### 3. Row position table (all versions)
Immediately follows the angle table.
```
n_rows × f32 — Y position of each row, in millimetres
```
---
### 4. Channel preambles (v3+)
One entry per channel, in channel-index order (CH1 first).
```
for each channel:
u16 preamble_length — byte count of the UTF-8 string that follows
utf8[N] preamble — Tektronix WFMOutpre string
```
The preamble is the oscilloscope's `WFMOutpre` response string. Readers extract the following keys (case-insensitive, space-separated value):
| Key | Stored unit | Conversion to mV |
|-----|-------------|-----------------|
| `YMULT` | V count⁻¹ | multiply by 1000 |
| `YOFF` | ADC counts | used directly |
| `YZERO` | V | multiply by 1000 |
**v2 fallback:** when preambles are absent, readers use:
- `YMULT` = 1.5625 mV count⁻¹ (50 mV/div, 8 div, 8-bit ADC)
- `YOFF` = −87.04 ADC counts (scope position = −2.72 div)
- `YZERO` = 0 mV
---
### 5. Background waveform (v4+)
```
u32 n_bg_samples — number of i8 ADC samples that follow
i8[n_bg] background — one representative CH1 background shot
```
The background waveform has the same `samples_per_frame` length as a normal CH1 waveform. It is subtracted from each CH1 waveform before FFT processing when background subtraction is enabled. When `n_bg_samples == 0` the section is present but empty.
---
### 6. Waveform data (all versions)
Begins immediately after the fixed header (v2), preambles (v3), or background waveform (v4+). The waveform data is a flat, contiguous array with the following logical shape, stored in row-major (C) order:
```
waveform_data[n_angles][n_rows][n_channels][n_frames][samples_per_frame]
```
Each element is a signed ADC count of size `bytes_per_sample`:
- `bytes_per_sample == 1` → `i8`
- `bytes_per_sample == 2` → `i16` big-endian
**Total byte count:**
```
waveform_bytes = n_angles × n_rows × n_channels × n_frames × samples_per_frame × bytes_per_sample
```
#### Index semantics
| Dimension | Range | Meaning |
|-----------|-------|---------|
| `[a]` | 0 … n_angles−1 | Scan angle (rotation position) |
| `[r]` | 0 … n_rows−1 | Row (Y position); row 0 is the first acquired |
| `[c]` | 0 … n_channels−1 | Channel (0=CH1 RF, 1=CH3 Bias A, 2=CH4 Bias B) |
| `[f]` | 0 … n_frames−1 | Frame (X position) within the row |
| `[s]` | 0 … spf−1 | Sample index within the waveform |
#### Frame-count determination
- **v5:** use `n_frames_hdr` directly; do not use file-size arithmetic.
- **v2–v4:** `n_frames = floor((file_bytes_after_header_sections) / (bytes_per_sample × n_angles × n_rows × n_channels × samples_per_frame))`. Any remainder bytes are a partial trailing row and are discarded.
---
### 7. PREC section (v5)
The PREC section is appended immediately after the waveform data and is present if and only if `version == 5` and the file size exceeds `waveform_end_offset`.
```
waveform_end_offset = data_offset + waveform_bytes
```
where `data_offset` is the file offset of the first waveform byte (the byte immediately after the background waveform, or after the angle/row tables for v2 files).
#### PREC header (8 bytes)
| Offset (relative) | Size | Type | Field | Description |
|-------------------|------|------|-------|-------------|
| 0 | 4 | `char[4]` | `prec_magic` | `PREC` (ASCII). Absent or wrong magic → ignore section. |
| 4 | 1 | `u8` | `prec_version` | PREC format version. Currently `1`. |
| 5 | 1 | `u8` | `flags` | Bitmask (see below). |
| 6 | 2 | `u16` | `n_stored` | Number of angle entries that follow. 0 ≤ `n_stored` ≤ `n_angles`. |
##### Flags byte
| Bit | Mask | Meaning when set |
|-----|------|-----------------|
| 0 | `0x01` | `bg_sub_applied` — background waveform was subtracted from CH1 before the FFT when these images were computed. |
| 1–7 | — | Reserved, must be zero on write; readers must ignore. |
#### PREC angle entries
Repeated `n_stored` times, in arbitrary angle-index order:
```
for each stored angle:
u16 angle_idx — index into the angle table (0-based)
f32[n_rows×n_frames] peak_freq_mhz — CH1 FFT peak frequency, MHz, row-major
f32[n_rows×n_frames] dc4_mv — CH4 waveform mean, mV, row-major
f32[n_rows×n_frames] dc3_mv — CH3 waveform mean, mV, row-major
```
All image arrays are `f32` big-endian, stored in row-major order: element `[r][f]` is at offset `(r × n_frames + f) × 4` bytes within the array.
**`peak_freq_mhz`** is computed without any DC-threshold masking (i.e. the FFT is run on every pixel unconditionally). Readers apply the `dc4_mv` threshold at display time:
```
pixel is valid ⟺ dc4_mv[r][f] ≥ threshold_mv
display_value = peak_freq_mhz[r][f] if valid, else 0
```
**`dc4_mv` / `dc3_mv`** are the mean of all ADC samples in the respective channel waveform, converted to millivolts using the channel calibration:
```
dc_mv = (adc_mean − YOFF) × YMULT + YZERO
```
#### When readers must bypass the PREC fast path
Readers must fall back to real-time FFT computation (ignoring stored `peak_freq_mhz`) when any of the following are true:
- Time-domain gating is active (zeroing samples outside a time window changes peak frequency).
- Zero-padding (`n_fft ≠ samples_per_frame`) is requested (changes bin spacing).
- The reader's background-subtraction setting does not match `flags.bg_sub_applied`.
---
## Derived quantities
```
pixel_pitch_mm = velocity_mm_s / laser_freq_hz
x_axis_mm[f] = x_start_mm + f × pixel_pitch_mm (f = 0 … n_frames−1)
time_axis_ns[s] = s / sample_rate_hz × 1e9 (s = 0 … spf−1)
freq_axis_mhz[k] = k × sample_rate_hz / (n_fft × 1e6) (k = 0 … n_fft/2)
where n_fft = samples_per_frame unless zero-padding is active
velocity_ms[r][f] = peak_freq_mhz[r][f] × grating_um (grating_um user-supplied)
```
---
## ADC calibration
Convert raw ADC counts to millivolts:
```
voltage_mv = (adc_count − YOFF) × YMULT_mv + YZERO_mv
```
Invert (mV → ADC count):
```
adc_count = (voltage_mv − YZERO_mv) / YMULT_mv + YOFF
```
where `YMULT_mv` is YMULT in mV count⁻¹ (= scope YMULT in V count⁻¹ × 1000).
---
## Waveform data layout detail
For a scan with `n_angles=2`, `n_rows=3`, `n_channels=3`, `n_frames=4`, `spf=5` the layout is:
```
angle 0
row 0
CH1: [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4] [s0 s1 s2 s3 s4]
frame 0 frame 1 frame 2 frame 3
CH3: …(same layout)…
CH4: …(same layout)…
row 1
…
row 2
…
angle 1
…
```
The flat byte offset of sample `s` of frame `f`, channel `c`, row `r`, angle `a` is:
```
offset = data_offset
+ (a × n_rows × n_channels × n_frames × spf
+ r × n_channels × n_frames × spf
+ c × n_frames × spf
+ f × spf
+ s)
× bytes_per_sample
```
---
## Size reference
Approximate sizes for representative scans (`bytes_per_sample = 1`, `n_channels = 3`).
| n_angles | n_rows | n_frames | spf | Waveform data | PREC section |
|----------|--------|----------|-----|---------------|-------------|
| 1 | 500 | 500 | 400 | 300 MB | 12 MB |
| 4 | 500 | 500 | 400 | 1.2 GB | 48 MB |
| 1 | 2000 | 2000 | 400 | 4.8 GB | 48 MB |
| 4 | 2000 | 2000 | 400 | 19.2 GB | 192 MB |
| 8 | 2000 | 2000 | 400 | 38.4 GB | 384 MB |
| 16 | 2000 | 2000 | 400 | 76.8 GB | 768 MB |
**PREC section size formula:**
```
prec_bytes = 8 + n_stored × (2 + 3 × n_rows × n_frames × 4)
```
---
## Compatibility notes
### Reading v5 files with a v4 reader
A v4 reader that only accepts versions `{2, 3, 4}` will reject a v5 file with an "unsupported version" error. This is intentional: a v4 reader would derive `n_frames` from the file size, incorrectly including the PREC bytes in the sample count, producing a silently wrong reshape.
### Producing v5 files
v5 files are produced by the SRAS viewer's **"Pre-process and Save as v5"** action. The procedure is:
1. Copy the source file (any version) verbatim.
2. Set `version = 5` at byte offset 4.
3. Set `n_frames_hdr` at byte offset 25 to the actual acquired frame count.
4. Truncate the copy to `data_offset + waveform_bytes` (removes any pre-existing stale PREC tail).
5. Compute `peak_freq_mhz`, `dc4_mv`, and `dc3_mv` for every angle using chunked FFT.
6. Append the PREC section.
### Partially-written PREC sections
If `n_stored < n_angles` (e.g. pre-processing was interrupted), the file is still valid. Readers use stored images for the angles present in the PREC section and fall back to real-time FFT for the remainder. Readers must check `angle_idx` bounds on each entry and stop parsing on an out-of-range value.
+427
View File
@@ -0,0 +1,427 @@
# sras-viewer design notes
Rationale that outgrew code comments. Each section is referenced by a short
pointer comment at the relevant definition, so the code stays scannable and
the reasoning stays findable.
## Memory budget and row chunking (`sras_compute.py`)
DC images are computed over row chunks so the float32 working buffers for one
chunk stay under a memory budget. A fixed row count (the original design)
works fine for small legacy scans but is catastrophic for a v6 scan with a
large per-angle frame/sample count — e.g. a 7500-frame × 2500-sample angle
needs ~2.4 GB for a single 32-row chunk.
With chunks running concurrently the budget has to cover *all* live chunks at
once. On a large scan `chunk_rows` is already clamped to its floor of one row
(one row alone is ~75 MB of float32 at 7507×2500), so shrinking the per-chunk
size cannot buy more concurrency — the worker count must be derived from the
budget instead: `_plan_chunks` picks the worker count *first* and sizes the
chunk to it. Sizing the chunk first is the trap: a single chunk would always
consume the whole budget and leave room for exactly one worker, precisely on
the large scans that need concurrency most.
The 1024 MB default (`SRAS_MEM_BUDGET_MB`) is the measured knee on a 16-core
machine against a 7507-frame × 2500-sample angle: 512 MB left ~20% of the
speedup on the table, and 1536+ MB cost ~0.4 GB more resident memory for no
further gain.
A caller that itself runs several computations concurrently (angle-level
parallelism, `plan_angle_level`) must pass *both* `max_workers=1` and its
share of the budget. Capping the workers alone is not enough: the chunk would
still be sized against the whole budget, and N concurrent callers would each
allocate all of it.
## FFT peak search: block-parallel direct transform (`sras_compute.py`)
The displayed RF value per pixel is the argmax of the zero-padded power
spectrum of that pixel's CH1 waveform. This used to run through `_peak_bins_zoom`,
a coarse-rfft-plus-local-fine-DFT refinement that avoided ever materialising
a padded spectrum — at the pad factor of 40 needed for mapping resolution, a
full padded spectrum is ~9 GB per scan row, which used to collapse the old
row-chunk planner to one worker and make synthesis single-threaded. That
refinement was removed once pyFFTW became the sole, mandatory FFT backend
(SciPy dropped as a compute backend entirely): `_peak_bins` now always runs
the real full transform, and the memory problem zoom dodged is instead
solved by bounding *per-block* spectrum memory rather than avoiding full
spectra altogether.
`_peak_bins` runs the full transform (`_block_rfft`, a cached pyFFTW
`builders.rfft` plan, FFTW_MEASURE, wisdom persisted under
`~/.cache/sras-viewer/`) and argmaxes the power spectrum, in blocks fanned
out task-parallel over a persistent thread pool (`_fft_pool()`) — each pool
thread runs one single-threaded transform at a time, so aggregate
parallelism equals the pool's worker count. This block-streaming structure
is not just about parallelism: it is also what keeps memory bounded, by
never materialising more than one block's worth of full padded spectrum at
a time, regardless of how many waveforms a chunk holds.
The block size is the part that has to adapt to pad factor.
`_fft_block_for(spf, n_len)` derives waveforms-per-task from a fixed
per-thread byte budget (`_FFT_PLAN_BYTES_BUDGET`, `SRAS_FFT_PLAN_BUDGET_MB`,
default 16 MB) rather than a fixed constant, because a cached pyFFTW plan's
input+output buffers are *permanent* per-thread memory (the plan cache is
never evicted) — with a fixed 512-waveform block, pad 40 at a 2500-sample
frame costs ~210 MB per pool thread (~3.3 GB total across 16 threads);
`_fft_block_for` bounds that to ~16 MB per thread (~260 MB across 16
threads) at the same pad factor, while still reproducing the old tuned 512
exactly at natural resolution (pad 1), where it cost nothing to begin with.
`_FFT_BLOCK_MAX` (512) and `_FFT_BLOCK_MIN` (32) cap and floor the result:
the ceiling is the measured knee on a 16-core machine at natural resolution
(smaller blocks serialise on GIL-held numpy dispatch, larger ones lose cache
residency and task granularity); the floor keeps task granularity from
collapsing at extreme pad factors, at the cost of exceeding the byte budget
there.
The outer row-chunk sizing (`_plan_fft_rows`) needed no companion change.
It only ever budgets the raw float32 waveform *read* buffer, which this
change doesn't touch — spectrum memory is bounded independently by
`_fft_block_for`, and since the pool only ever runs as many blocks
concurrently as it has workers, peak transient spectrum memory during a
chunk's FFT phase is `_MAX_WORKERS * block * bytes_per_wf`, the same bound
whether the chunk holds 10 rows or 10,000. Queuing more rows into one chunk
to keep the read-row pool busy therefore can't blow up spectrum memory.
`tools/check_equivalence.py`'s golden-hash harness remains the end-to-end
regression baseline for this path, unaffected by this change.
## Row-averaged FFT: same-row, distance-weighted SNR cleanup (`sras_compute.py`)
`compute_rf_image`'s `row_avg_n` parameter averages each pixel's CH1
waveform with its up-to-n same-row neighbors before the FFT peak search, to
improve SNR on noisy scans. Never crosses rows: pixel pitch is strongly
anisotropic and varies by scan (5 µm × 50 µm on a typical scan, but as
stretched as 5 µm × 1 mm on others), so a physically meaningful "neighbor"
set can't be a fixed-shape 2-D window — but the X pitch *within one row* is
a single file-wide constant (`SrasFile.pixel_x_mm`), so restricting to the
row axis sidesteps the anisotropy question entirely rather than solving it
with an elliptical or physically-scaled 2-D kernel.
`_row_average_weights` is a Gaussian in pixel-index distance, not physical
mm distance — deliberately: within one row those are the same function up
to a fixed scale factor (`pixel_x_mm` is constant along a row), so the
kernel itself needs no pitch at all. `pixel_x_mm` is used for real exactly
once, in the GUI's options dialog, to show the window's physical width —
not in the kernel math, where it would only ever cancel out.
`_row_average_waveforms` is a masked/renormalized convolution (two
`correlate1d` calls, numerator and denominator, divided) rather than a
single fixed-normalized convolution, because a masked neighbor must
contribute *zero weight*, not a zero-amplitude sample at full weight — the
latter would bias every average near a masked run or a row's own edge
toward zero. The same two-correlation trick handles row-edge truncation for
free: `mode="constant", cval=0.0` zero-pads both the numerator and the
denominator beyond a row's own ends, so the output renormalizes by whatever
weight sum actually landed inside the row, no separate edge case.
Background subtraction stays exactly where it already was (subtracted once
from the fully-assembled `waves` buffer) rather than being threaded into the
per-neighbor gather. This is exact, not an approximation: because
`_row_average_waveforms`'s denominator is always the *actual* sum of
included, valid weights (never a fixed total), `Σwᵢ·(rawᵢ−bg) / Σwᵢ`
distributes to `avg − bg·(Σwᵢ/Σwᵢ) = avg − bg` regardless of which or how
many neighbors were included — subtracting background from the averaged
waveform is identical to subtracting it from every neighbor first, for any
window, at any row edge, with any number of masked-out neighbors.
No cross-row halo is needed: `compute_rf_image`'s chunk loop already splits
on rows only, and `read_row` already reads one row's complete
`(n_frames, spf)` slice at a time — averaging happens entirely inside that
one row's own frame axis, so a chunk boundary (which falls between rows)
can never truncate a window. Only a row's own start/end can, and that's the
same edge case the masked convolution already handles.
The averaging step doubles the live per-row scratch memory (a full-width
`(n_frames, spf)` buffer on top of the existing compacted `waves` buffer),
so `compute_rf_image` halves its byte budget when `row_avg_n > 0` before
`_plan_fft_rows` runs — see "Memory budget and row chunking" above. On the
largest real scans `_plan_chunks` is already
clamped to its floor of one row regardless, so this costs no concurrency
where it matters most; it mainly protects moderate-sized scans from an
unexpected regression.
Persistence: `cached_rf_image` (the extracted fast-path check) requires
`sras.precomputed_row_avg_n == row_avg_n` exactly, so a raw request can
never be silently served a row-averaged cache or vice versa, and a request
at one window size can never be served a cache at another — see
`scan_format.md`'s Cache Tail / CACH tail version history sections for the
on-disk `row_avg_n` field this depends on.
## Serving a stored cache: provenance, not just presence
A stored `peak_freq_mhz` image is only interchangeable with a live compute
for the *exact* settings it was computed under. Three of them are baked
irreversibly into the numbers — background subtraction, row-averaging window,
and zero-padding — so all three are recorded in the `SFFT` block and checked
by `cached_rf_image` before it hands the image back. Getting this wrong is
not a slow display, it is a *wrong* display, which is why the check is a
single predicate in one place rather than spread across callers.
The min peak frequency floor is the fourth recorded setting, and it sits
between the DC threshold (fully re-applicable at display time) and the
baked-in three: it is *tighten-only* re-applicable. A floor at or above the
stored one is served by masking stored pixels below it to the 0.0 "no valid
peak" sentinel — invalidated, not re-resolved; only a real recompute can
find their true above-floor peak. A floor *below* the stored one is a
genuine mismatch: the stored search never looked at those bins. Because a
served-then-masked image is a lossy stand-in for a real floored compute,
`cache_file` passes `use_stored=False` so a batch recompute always runs the
real FFT — otherwise re-batching a cached file would silently bake the
masked copy of its own old image back into the store.
Padding is the subtlest of the three, because a padded FFT looks like it
should be a refinement of the unpadded one. It isn't: zero-padding
interpolates between the natural bins, so it resolves a different peak
frequency for the same waveform. `precomputed_pad_factor` exists so a padded
view can be served from a cache computed at *its* pad while still refusing
one computed at any other, including pad 1. Before it existed the store was
pad 1 by definition and any `n_fft` was rejected outright — correct, but it
meant a user working at a pad factor got nothing at all from batch-computing
a file, which is most of the point of the feature. `pad_factor_for` maps an
`n_fft` request onto the integer factor a store could have recorded, and
returns 0 for a request that is not a whole multiple of `samples_per_frame`
— unmatchable by construction, since only integer factors are representable.
The batch actions therefore have to cache at the *viewer's* current pad
factor, not a fixed one: a cache stored at a pad nobody is viewing at is
dead weight. When the two do diverge (the user changes the pad after
batching), `_cache_mismatch_notes` says so in the scan info panel, because
the symptom otherwise is just "the file I pre-computed got slow again" with
no visible cause.
That divergence note is informational only, not a warning of an impending
recompute. The *display* path (`_stored_fft_image`) never asks
`cached_rf_image` whether a stored image matches the window's live
bg-sub/pad/row-averaging controls — it asks whether the image matches its
*own* recorded settings (`sras.precomputed_bg_sub`/`precomputed_pad_factor`/
`precomputed_row_avg_n`), which is always true whenever a stored image
exists. Two settings are taken live instead: the DC threshold and the min
peak frequency floor, both cheaply re-appliable as masks on serve. So for
bg-sub/pad/row-avg, presence alone decides whether a stored image is shown;
the floor is the one case where a live control can gate it — a live floor
*below* the stored one can't be answered by masking, so `cached_rf_image`
declines and the caller falls through to a real compute. They still matter for two things: a genuinely never-computed
angle's first live compute, and an explicit batch recompute — both of which
read the live controls and produce new stored data, at which point it's the
new data's *own* settings that get self-matched from then on. This is what
keeps a view switch (angle, channel, or flipping bg-sub/pad) from ever
discarding precomputed data — only an explicit batch recompute does, and it
already reloads the file afterward so the new data displays immediately.
Row-averaging has no live control to diverge from in the first place (it's
only ever set inside the batch dialog), so it never appears in the
divergence note — the "Cached images" line's own `row-averaged n=…` phrase
already covers it.
### Two caches, in cost order
`_refresh_display` consults this window's in-session `_fft_cache`/`_dc_cache`
dicts first, then the open file's stored v5/v7 blocks, and only then
dispatches a `ComputeWorker`. The second tier is what makes a batch-computed
file worth having; without it every angle change queued a worker and a
progress popup for an image already sitting on disk — precisely the cost the
batch was run to avoid. (The stored image *was* reachable before, but only
from inside `ComputeWorker`, i.e. after paying for the thread and the popup.)
The file tier asks with `allow_dc_recompute=False`. If applying the DC4 mask
would mean reading a whole CH4 channel, it declines rather than blocking the
GUI thread, and the fall-through worker reaches the same stored image via
`compute_rf_image` and pays for the mask off-thread. So the GUI thread never
does I/O, and the slow path is still a fast path.
## Angle alignment coordinate frames (`sras_compute.py`)
Alignment puts every angle's images onto one shared, zero-padded pixel grid
using a rigid transform only — rotation + translation, never scale.
Angle 0 (the reference) is the sole coordinate authority: it is the only
angle whose stage XY (`x_start_mm` / `y_positions_mm`) is ever read, and the
shared canvas is literally an extension of angle 0's own pixel grid, so the
aligned view carries angle 0's real X/Y axes. Every *other* angle is placed
purely by content — its rotation and translation come from cross-correlating
its CH4 image against angle 0's (`register_angle_to_reference`) — and its own
stage XY is deliberately never consulted. That is not an oversight: the
rotation stage moves the sample relative to the scan window, so where a
window sat in stage coordinates says nothing about where the sample is, and
an earlier design that pivoted each angle on a signal-weighted centroid of
its own window put every angle on a ~20 mm circle around the optical center
instead of stacking them into one shape.
Only two coordinate frames exist:
* **local mm** — one angle's own physical frame: origin at the *center of its
own pixel array*, x along +column, y along +row, scaled by that angle's own
pitches. Carries no stage position whatsoever.
* **ref mm** — the reference angle's local mm. A registration result
`(rotation_deg, shift_mm)` is exactly the rigid map from an angle's local
mm to ref mm: `q = R(rotation_deg) @ l + shift_mm`. Stage coordinates
re-enter once, at the very end, when the canvas origin is converted to
angle 0's stage mm (`AlignmentResult.canvas_origin_mm`).
Rotation is done in mm, never on raw pixel indices: the x pitch
(`SrasFile.pixel_x_mm`, 5 µm on a real scan) and the y/row pitch (50 µm)
differ by 10×, so rotating the raw index grid would shear the image — an
unwanted anisotropic scale. Registration runs on a resampled *isotropic* grid
for the same reason, and every affine maps shared-grid index → mm → undo
rotation/shift → that angle's own local mm → that angle's own raw index,
matching the output→input convention `scipy.ndimage.affine_transform` wants.
### Cropping the canvas is index translation, not a second transform
`crop_alignment_result` restricts an `AlignmentResult` to a rectangular window
of its canvas by folding the crop into each angle's existing affine rather than
composing a new one. From `_affine_out_to_src`, `matrix = D @ Rinv @ A_out`
depends only on the pitches and the rotation, and `A_out @ [row0, col0]` is
exactly the mm displacement of the new origin, so
```
matrix @ [r', c'] + (offset + matrix @ [row0, col0])
== matrix @ [r' + row0, c' + col0] + offset
```
identically. `matrix` is untouched and `offset` — which already absorbs the
origin — absorbs the crop too.
Two things follow, and both are relied on. `apply_alignment`, `reproject_mask`
and the aligned exporter all work on a cropped result with no special-casing:
resampling a cropped result is *exactly* a slice of resampling the full one
(`tests/test_align_export.py::test_crop_is_a_window_of_the_full_canvas` asserts
bit equality). And because the crop offset is a whole number of canvas pixels,
`canvas_for_params`' snap invariant — the reference angle lands on integer
canvas pixels — survives the crop, which is what keeps the reference exportable
as a verbatim block.
## Aligned export (`sras_align_export.py`)
`write_aligned_sras` bakes an alignment into a new v6 file: every angle
resampled onto the cropped shared canvas, so all of them end up with identical
geometry and the file opens already aligned. It is the only place in the
codebase that *resamples* waveform data — `sras_edit_scans` and `sras_average`
copy waveform bytes verbatim — which is why it is its own top-level module
rather than part of `sras_format` (scoped to the versioned binary spec, per the
sidecar section's own rule) or `sras_compute` (imported by every
multiprocessing child).
**Nearest neighbour, never interpolation.** Each output pixel gets exactly one
source pixel's three waveforms, verbatim. Averaging two neighbouring CH1
packets would synthesise a waveform the instrument never measured, whose FFT
peak is the peak of neither — meaningless for a technique whose entire output is
that peak frequency. The cost is that some source pixels are duplicated and
others dropped, which is the same trade `apply_alignment`'s `order=0` already
makes for the display.
**The rounding rule is `floor(x + 0.5)`, not `np.rint`.** `scipy.ndimage`'s
`order=0` rounds halves away from zero while `np.rint` rounds them to even. The
canvas is snapped to the reference's own pixel grid, so an angle whose row pitch
differs from the reference's lands on exact half-integers across whole rows —
this is the common case, not a corner case. Getting it wrong shifts those rows
by one source pixel relative to what the Aligned View drew.
**Out-of-bounds is tested on the fractional coordinate, not the rounded index.**
`scipy`'s `mode="constant"` writes `cval` wherever the coordinate leaves the
range of sample *centres*, `[0, n-1]` — a coordinate of −0.4 rounds to a
perfectly valid index 0 and is still padding. Testing the rounded index instead
puts a one-pixel rim of real data everywhere the preview shows padding.
**...but with a tolerance (`_EDGE_TOL`).** The affine is built from a chain of
mm-space multiplications, so an exactly-integer transform comes out a few times
1e-13 off: the reference angle's offset is `-20 - 7e-15`, not `-20`. A bare
`>= 0.0` therefore rejects that angle's entire first row, and `<= n-1` its last
column — for the *reference* angle, whose whole job is to pass through as an
exact integer crop. The tolerance is ~7 orders of magnitude above that noise and
~7 below the half-pixel scale at which a rounding decision means anything, so it
can only ever change pixels whose scipy answer was itself decided by noise.
**Padding is the per-channel ADC code nearest 0 mV, not 0.** Zero ADC decodes to
`(0 - yoff) * ymult + yzero`, which on real calibration is around +100 mV —
above any sensible CH4 mask threshold, so a zero fill would paint a solid
rectangle of "valid" pixels around the sample and corrupt every DC image and ROI
statistic downstream.
**Source rows are served from sliding in-RAM bands** (`_SourceReader`). A
rotated angle maps one output row to a *diagonal* across the source array, so
the pixels of a single output row come from hundreds of different source rows —
~1.4 MB each on a full-size scan. Indexing a memmap pixel-by-pixel in output
order re-faults nearly the whole angle per output row: terabytes of paging for a
gigabyte of data. Reading a contiguous band per output chunk, with the band
advancing monotonically, costs roughly 2× the source size in total reads.
**Writes go to `.part` and are `os.replace`d into position.** Not politeness: a
truncated .sras is not detectably broken, because `_parse_v6` drops incomplete
trailing angle blocks and opens what is left as an aborted scan. A half-written
export left in place would silently look like a real file with fewer angles.
**The Angle Table is carried over unchanged.** Alignment removes the *spatial*
rotation of the sample; it does not change which acoustic propagation direction
each angle measured, and that direction is the scientific content of a
multi-angle scan. Zeroing the table would make the export self-consistent for
re-registration and useless for anisotropy work. The consequence is that
re-registering an export needs `seed_deg=0.0` to put 0° inside the coarse sweep,
since `nominal_delta_deg` is still non-zero — which is exactly what the seed
parameter exists for.
## Alignment wizard (`sras_viewer/align_wizard.py`)
A `QWizard` rather than another dialog because the three steps are genuinely
sequential and the last one is destructive: correlate, choose a crop, write a
file. It replaces both former Fusion actions, so it also absorbs the old
`ManualAlignmentDialog`'s by-eye nudge editor — otherwise a scan the search
cannot fit would have no fallback at all.
Shared state lives on the wizard object, not in `registerField`: the pages pass
numpy arrays, `ManualAngleParams` and an `AlignmentResult` between them, none of
which are scalar widget properties.
`IndependentPages` is deliberately left **off**. With it set Qt never calls
`cleanupPage`, and `cleanupPage` is how the ROI page discards a crop when the
user goes back to re-correlate — a crop is indexed in canvas pixels, and a new
rotation means a different canvas, so stale indices would silently be
reinterpreted against the wrong grid. `geometry_generation` is the belt-and-
braces check for the same hazard.
The mask-stack preview shares the **final** canvas's origin and uses a pitch
that is an integer multiple of it, unlike the old manual dialog's padded,
unsnapped preview canvas. That is what lets the crop page convert a rectangle
drawn in millimetres into an exact integer window of the real canvas, with no
second coordinate frame to reconcile.
"Fit to full overlap" uses `largest_rect_at_least`, a largest-rectangle sweep,
not a bounding box of the fully-covered pixels. The full-overlap region of
several rotated scans is roughly a disc, and its bounding box has corners no
angle covers — offering that as the crop would hand the user the padding they
were trying to avoid.
Every background launch follows the two rules `_run_worker`'s docstring
establishes: disable the trigger *before* the call (so a re-entrant click cannot
start a second thread over the first), and never ignore the returned bool.
Progress is an inline `QProgressBar` on the page rather than a `QProgressDialog`
— a window-modal popup over a wizard both looks wrong and reintroduces the
event-loop pumping hazard that ordering exists to avoid. `reject()` refuses to
close while a job is in flight, since the running worker's signals are connected
to bound methods of the pages Qt would be deleting.
## Manual-alignment sidecar (`sras_compute.py`)
`<name>.sras.align.json` lives next to the scan file. The code lives in
`sras_compute`, not `sras_format`: `sras_format` is scoped to the versioned
binary .sras spec itself (see `scan_format.md`), while a manual alignment is
a viewer-computed *derived* artifact, analogous in kind to `AlignmentResult`
— so it belongs with the alignment math it serialises. json + pathlib are
stdlib, so this adds no dependency to a module whose load-bearing constraint
is staying free of Qt/matplotlib for cheap multiprocessing-child imports.
### Schema history
The stored `rotation_deg`/`shift_mm` are meaningless without the frame they
were measured in, so `_SIDECAR_SCHEMA_VERSION` is bumped whenever that frame
changes. Each bump makes older files describe a different (and, for the bugs
each bump fixed, actively wrong) transform than the same numbers would today;
loading one unchanged would silently reproduce the very "scans show up
everywhere" symptom the bump fixed — so older sidecars are treated as absent
rather than migrated.
* **1 → 2** — pivot moved from the scan-window bbox center to a
content-derived centroid, and the rotation sign convention was corrected.
* **2 → 3** — the content centroid was abandoned entirely: rotation is now
about each angle's own array center, mapped onto the reference's array
center, with `shift_mm` in the reference's local mm frame. No angle but the
reference contributes stage coordinates any more.
+40
View File
@@ -0,0 +1,40 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "sras-viewer"
version = "0.1.0"
description = "Viewer and processing tools for SRAS .sras scan files"
requires-python = ">=3.12"
dependencies = [
"PyQt6==6.10.2",
"numpy==2.4.1",
"matplotlib==3.10.8",
"scipy==1.18.0",
# Angle alignment only: masked FFT phase correlation (skimage.registration).
"scikit-image==0.26.0",
# Mandatory rfft backend for the peak search (no SciPy fallback).
"pyFFTW==0.15.1",
]
[project.optional-dependencies]
dev = ["pytest"]
[project.scripts]
sras-viewer = "sras_viewer.main_window:main"
[tool.setuptools]
py-modules = [
"sras_format",
"sras_compute",
"sras_render",
"sras_workers",
"sras_align_export",
"sras_average",
"sras_edit_scans",
]
packages = ["sras_viewer"]
[tool.pytest.ini_options]
testpaths = ["tests"]
+221 -2
View File
@@ -1,4 +1,4 @@
# SRAS Scan Binary Format — Version 6 # SRAS Scan Binary Format — Version 6 / 7
Each `.sras` file contains **one complete scan**: all GR rotation angles and all Each `.sras` file contains **one complete scan**: all GR rotation angles and all
Y rows. Files are named `{prefix}.sras`. Y rows. Files are named `{prefix}.sras`.
@@ -10,6 +10,14 @@ rotated by that specific angle** — not the worst case across all angles — so
fewer rows than a 45° scan of the same ROI, and the file format reflects that fewer rows than a 45° scan of the same ROI, and the file format reflects that
instead of forcing every angle to the largest bounding box. instead of forcing every angle to the largest bounding box.
v7 is byte-identical to v6 (same header, angle table, geometry table, row
table, preamble blocks, background block, waveform data) plus an optional
trailing **Cache Tail** holding precomputed per-angle DC and/or FFT images
(see [Cache Tail (v7)](#cache-tail-v7) below) — only the header's `version`
field and the presence of that trailing section differ. Scans come off the
scope as v6; `sras_viewer.py`'s "Convert" menu batch actions convert a file
to v7 **in place** the first time either cache block is computed and stored.
--- ---
## File Layout ## File Layout
@@ -22,6 +30,7 @@ instead of forcing every angle to the largest bounding box.
[Preamble Blocks — n_channels × (uint16 length + UTF-8 WFMOutpre string)] [Preamble Blocks — n_channels × (uint16 length + UTF-8 WFMOutpre string)]
[Background Block — uint32 n_bg_samples + n_bg_samples × int8 bytes] [Background Block — uint32 n_bg_samples + n_bg_samples × int8 bytes]
[Waveform Data (ragged) — per angle: n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bps bytes] [Waveform Data (ragged) — per angle: n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bps bytes]
[Cache Tail (optional) — "CACH" + DC block (optional) + FFT block (optional); v7 only]
``` ```
All multi-byte integers and floats use **big-endian** byte order All multi-byte integers and floats use **big-endian** byte order
@@ -164,6 +173,41 @@ sum over angles a of: n_rows[a] × 3 × n_frames[a] × samples_per_frame × byte
> Table and check `file_size` against the running total before reshaping — > Table and check `file_size` against the running total before reshaping —
> a fixed `(n_angles, n_rows, ...)` reshape (as in pre-v6 readers) will not > a fixed `(n_angles, n_rows, ...)` reshape (as in pre-v6 readers) will not
> work since row/frame counts are no longer uniform across angles. > work since row/frame counts are no longer uniform across angles.
>
> A consequence worth stating explicitly: because a short file opens
> *successfully* as a scan with fewer angles, a truncated file is not
> detectably broken. Anything that writes a .sras must therefore stage to a
> temporary name and rename on success — the viewer's aligned export writes
> `<name>.part` and `os.replace`s it — or an interrupted write leaves behind
> something that loads without complaint and silently has the wrong angle count.
---
## Files written by the viewer's Alignment Wizard
The acquisition app is not the only producer of this format. The viewer's
`Fusion → Alignment Wizard…` writes a **v6** file holding the aligned, cropped
stack, with these properties:
* Every angle shares one grid — the cropped alignment canvas — so the
Per-Angle Geometry Table is `n_angles` identical records and the ragged Row
Table is `n_angles` identical spans. The raggedness v6 exists for is still
*expressible*, just unused, so any v6 reader works unchanged.
* `x_delta` is the reference angle's own pitch, which is exactly
`velocity_mm_s / laser_freq_hz`, so the derived X axis stays consistent with
the header.
* The `*_nominal` header fields describe the crop. Uniquely for these files they
coincide with the actual per-angle geometry, since after alignment every angle
really does scan the same box.
* The **Angle Table is unchanged**. Alignment removes the sample's spatial
rotation, not the acoustic propagation direction each angle measured — that
direction is the point of a multi-angle scan, so it is preserved.
* Output pixels with no corresponding source pixel (the canvas corners a rotated
scan cannot reach) hold the per-channel ADC code nearest **0 mV**, not zero.
Zero ADC decodes to roughly +100 mV on real calibration and would read as
signal.
* No Cache Tail is written: any cached DC/FFT is indexed by the source's grid
and would be meaningless on the new one.
--- ---
@@ -182,6 +226,180 @@ using that angle's `x_start` from the Per-Angle Geometry Table (not
--- ---
## Cache Tail (v7)
Present iff `version == 7` and `file_size > cache_offset`, where:
```
cache_offset = data_offset + Σ over angles a of:
n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bytes_per_sample
```
i.e. exactly `data_offset + waveform_bytes` — the same "Total data size"
formula as Waveform Data above. This offset is derivable from the header and
Per-Angle Geometry Table alone and does **not** depend on which cache
block(s) are present, so a writer can always seek straight there without
reading or touching any waveform byte before it.
Unlike the (removed) v5 `PREC` section, which stored one omnibus per-angle
entry (FFT + both DC channels together) in a dense, uniform-geometry array,
the v7 Cache Tail splits DC and FFT into two **independent** sub-blocks —
each sized per-angle from the Per-Angle Geometry Table, each independently
present, and each independently updatable in any order, any number of
times, without disturbing the other. This matches `sras_viewer.py`'s
"Convert" menu, which exposes DC and FFT store as two separate batch
actions.
### CACH outer header (6 bytes, `">4sBB"`)
| Offset | Size | Type | Field | Description |
|--------|------|------|-------|-------------|
| 0 | 4 | `char[4]` | `cach_magic` | `CACH` (ASCII). Missing/wrong magic → treat file as having no cache. |
| 4 | 1 | `u8` | `cach_version` | Cache format version. Currently `4`; readers also accept `1`–`3` (each older tail simply lacks the fields added since — see the `SFFT` block below and [CACH tail version history](#cach-tail-version-history)). Any other value → treat the file as uncached (unlike v5's `PREC` section, which read but never validated its version byte). |
| 5 | 1 | `u8` | `block_flags` | Bit 0 = DC block (`SDCB`) follows. Bit 1 = FFT block (`SFFT`) follows, immediately after the DC block if both are present. Bits 2–7 reserved, must be zero on write. |
### DC block `SDCB` (present iff `block_flags & 0x01`)
7-byte block header, format `">4sBH"`:
| Offset (rel) | Size | Type | Field | Description |
|--------------|------|------|-------|-------------|
| 0 | 4 | `char[4]` | `magic` | `SDCB` |
| 4 | 1 | `u8` | `reserved` | `0`, reserved for future use |
| 5 | 2 | `u16` | `n_stored` | Number of angle entries that follow, `0 ≤ n_stored ≤ n_angles` |
followed by `n_stored` entries, each:
```
u16 angle_idx — index into the angle table (0-based)
f32[n_rows[angle_idx] × n_frames[angle_idx]] dc3_mv — CH3 waveform mean, mV, row-major
f32[n_rows[angle_idx] × n_frames[angle_idx]] dc4_mv — CH4 waveform mean, mV, row-major
```
Entries may appear in any order and need not be contiguous from angle 0 —
this supports storing (or re-storing) a subset of angles, or an
interrupted batch run leaving only some angles cached. Readers bounds-check
`angle_idx < n_angles` on each entry and stop parsing on an out-of-range
value, same as v5's `PREC` section.
### FFT block `SFFT` (present iff `block_flags & 0x02`)
Block header layout depends on `cach_version`:
Each `cach_version` appended one trailing field, so the header grows but
never shifts an existing offset:
- **`cach_version` 1**: 7 bytes, format `">4sBH"` — magic, flags, n_stored.
- **`cach_version` 2**: 8 bytes, format `">4sBHB"` — + `row_avg_n`.
- **`cach_version` 3**: 10 bytes, format `">4sBHBH"` — + `pad_factor`.
- **`cach_version` 4**: 14 bytes, format `">4sBHBHI"` — + `min_freq_khz`.
Always written by current code.
An older tail is read with its absent fields taken as the only value such a
tail can describe: `row_avg_n = 0` for a `cach_version` 1 tail, which
predates row-averaged FFT caching, `pad_factor = 1` for `cach_version`
1 or 2, which predate padded caching and are therefore natural-resolution,
and `min_freq_khz = 0` (no floor) for `cach_version` 1–3, which predate the
min peak frequency floor and therefore searched every bin above DC.
Files cached before any of these changes keep working with no recompute.
| Offset (rel) | Size | Type | Field | Description |
|--------------|------|------|-------|-------------|
| 0 | 4 | `char[4]` | `magic` | `SFFT` |
| 4 | 1 | `u8` | `flags` | Bit 0 = `bg_sub_applied` — background waveform was subtracted from CH1 before the FFT when these images were computed. Bit 1 = `row_averaged` — `peak_freq_mhz` came from same-row, distance-weighted averaged CH1 waveforms rather than raw per-pixel ones; `row_avg_n` (below) is the neighbor half-width used. Bits 2–7 reserved. |
| 5 | 2 | `u16` | `n_stored` | Number of angle entries that follow |
| 7 | 1 | `u8` | `row_avg_n` | *`cach_version` ≥ 2 only.* Same-row neighbor half-width, in pixels, that `peak_freq_mhz` was averaged over before its FFT; `0` = raw (unaveraged). Meaningful only when `flags` bit 1 is set — a `cach_version` 1 tail has no such byte and is always `row_avg_n = 0`. |
| 8 | 2 | `u16` | `pad_factor` | *`cach_version` ≥ 3 only.* Zero-padding factor the stored `peak_freq_mhz` was resolved at: `n_fft = pad_factor × samples_per_frame`, so `1` = natural resolution. Never `0`; a `cach_version` 1 or 2 tail has no such field and is always `pad_factor = 1`. |
| 10 | 4 | `u32` | `min_freq_khz` | *`cach_version` ≥ 4 only.* Min peak frequency floor the stored peak search excluded bins below, fixed-point in units of 0.001 MHz (kHz); `0` = no floor. Fixed-point rather than `f32` so a value that round-trips through the file compares exactly against the same value re-requested by a reader (the viewer's floor control has 0.001 MHz granularity). A `cach_version` 1–3 tail has no such field and is always `min_freq_khz = 0`. |
followed by `n_stored` entries, each:
```
u16 angle_idx — index into the angle table (0-based)
f32[n_rows[angle_idx] × n_frames[angle_idx]] peak_freq_mhz — CH1 FFT peak frequency, MHz, row-major
```
**`peak_freq_mhz`** for a raw store (`row_avg_n == 0`) is computed without
any DC-threshold masking (i.e. the FFT is run on every pixel
unconditionally, same as v5's `PREC` convention). Readers apply the DC4
threshold at display time:
```
pixel is valid ⟺ dc4_mv[r][f] ≥ threshold_mv
display_value = peak_freq_mhz[r][f] if valid, else 0
```
using the DC4 image from the DC block if that angle is also cached there,
else computed on demand.
For a row-averaged store (`row_avg_n > 0`), the DC4 threshold is applied
*during* the store — a pixel below threshold is left at `0` and never
contributes to any neighbor's average — since neighbor validity can't be
deferred to display time the way plain masking can. The threshold value
itself is not recorded, only that averaging happened and at what window
size. Readers still apply their own live DC4 threshold at display time
exactly as for a raw store, using whatever mask they currently have.
Readers must fall back to real-time FFT computation (ignoring stored
`peak_freq_mhz`) whenever the store's recorded provenance doesn't match what
the reader is asking for: time-domain gating is active, the reader's
requested `n_fft` doesn't equal `pad_factor × samples_per_frame`, the
reader's background-subtraction setting doesn't match
`flags.bg_sub_applied`, the reader's requested `row_avg_n` doesn't match
the stored value exactly, or the reader's requested min peak frequency
floor is *below* the stored `min_freq_khz`. A raw request must never be
served a row-averaged store, or vice versa; a request at one row-averaging
window size must never be served a store at another; and a request at one
padding must never be served a store at another, since a padded FFT
interpolates between the natural bins and so resolves genuinely different
peak frequencies. An `n_fft` that is not a whole multiple of
`samples_per_frame` can never match any store, because only an integer
`pad_factor` is representable.
The min peak frequency floor is the one asymmetric provenance field. A
request at a floor *below* the stored one cannot be served: the stored
search never looked at bins below its floor, so the stored numbers cannot
say what a lower-floored search would have found. A request at a floor at
or *above* the stored one **is** servable — the difference is re-applied at
display time by masking every pixel whose stored `peak_freq_mhz` is below
the requested floor to `0` (the same sentinel as the DC threshold mask;
a genuine peak can never be `0`, since bin 0 is always excluded from the
search). Such masked pixels are *invalid*, not re-resolved — only a real
recompute can recover the strongest peak above the floor for them.
### In-place write ordering
A writer updating a file's Cache Tail must write the payload (CACH header +
whichever block(s) are present) **before** flipping the header's `version`
byte to `7`, and `truncate()` the file to the new payload's end immediately
after writing it. If the process is interrupted between the payload write
and the version-byte flip, the file is still valid v6 — v6 parsing only
bounds-checks each angle's `offset + nbytes ≤ file_size`, it never asserts
exactly how many bytes follow the last angle's waveform block — so the
interrupted write leaves harmless trailing bytes rather than a corrupt file,
and the next successful write overwrites them via the same deterministic
`cache_offset`.
### CACH tail version history
Distinct from the outer `.sras` file `version` byte (top of this document),
which has stayed `7` since the Cache Tail was introduced — this is the inner
`cach_version` byte inside the `CACH` header itself.
| cach_version | Change |
|--------------|--------|
| 1 | Initial Cache Tail: `SDCB` (DC) and `SFFT` (FFT, 7-byte header) blocks. |
| 2 | `SFFT` header grows one byte, `row_avg_n` — the same-row neighbor half-width the stored `peak_freq_mhz` was averaged over before its FFT, `0` = raw. Readers still accept a `cach_version` 1 tail, treated as `row_avg_n = 0` for every angle it stores, so files cached before this change keep working without a recompute. |
| 3 | `SFFT` header grows a `u16` `pad_factor` — the zero-padding factor the stored `peak_freq_mhz` was resolved at, `1` = natural resolution. Before this, a padded view could never use a stored cache at all (the store was pad 1 by definition and readers rejected any `n_fft ≠ samples_per_frame`), so a user working at a pad factor got no benefit from batch-computing a file. Recording the factor lets such a view be served, while still refusing a store resolved at a *different* pad. Readers accept `cach_version` 1 and 2 tails as `pad_factor = 1`. |
| 4 | `SFFT` header grows a `u32` `min_freq_khz` — the min peak frequency floor the stored peak search excluded bins below, in 0.001 MHz units, `0` = no floor. The floor exists because a pixel that passes the DC-bias threshold but carries only weak real signal can otherwise resolve to the un-subtracted background's DC-leakage skirt — an implausibly-near-zero frequency (and so an implausibly slow velocity) for a pixel that has a genuine peak higher up. Recording the floor is what makes it enforceable against a store: without it, a stored image silently bypassed the floor entirely. Unlike the other provenance fields it is asymmetric — a *higher* requested floor is servable by masking stored pixels below it, only a *lower* one forces a recompute (see above). Readers accept `cach_version` 1–3 tails as `min_freq_khz = 0`. |
A reader that does not know a `cach_version` must treat the file as
uncached — not attempt a partial parse — and the file still reads as an
ordinary v7 (byte-identical to v6) scan, so a forward-dated tail costs a
recompute and never correctness.
---
## Acquisition Settings (fixed by sc3_aui_app.py) ## Acquisition Settings (fixed by sc3_aui_app.py)
| Parameter | Value | | Parameter | Value |
@@ -206,5 +424,6 @@ using that angle's `x_start` from the Per-Angle Geometry Table (not
| 3 | Added preamble blocks (WFMOutpre strings) after the row table, one length-prefixed UTF-8 block per channel. | | 3 | Added preamble blocks (WFMOutpre strings) after the row table, one length-prefixed UTF-8 block per channel. |
| 4 | Added background waveform block (CH1, Helios ON / Genesis OFF) after the preamble blocks; stored as `uint32` sample count followed by raw `int8` ADC bytes. | | 4 | Added background waveform block (CH1, Helios ON / Genesis OFF) after the preamble blocks; stored as `uint32` sample count followed by raw `int8` ADC bytes. |
| 5 | (skipped) | | 5 | (skipped) |
| 6 | Each angle now scans only the bounding box of the nominal ROI rotated by that angle instead of the AABB-expanded worst case across all angles. Header no longer carries a single global `x_start`/`x_delta`/`n_rows` — replaced with `*_nominal` reference fields plus a new Per-Angle Geometry Table (`x_start`, `x_delta`, `n_frames`, `n_rows` per angle) and a ragged Row Table / Waveform Data block sized per angle. **Not compatible with pre-v6 readers** that assume uniform geometry. `sras_viewer.py` reads v6 natively (per-angle `n_rows`/`n_frames`/`x_start`); the "Pre-process and Save as v5" fast-path export is not offered for v6 files since the flat v5 layout cannot represent per-angle geometry. | | 6 | Each angle now scans only the bounding box of the nominal ROI rotated by that angle instead of the AABB-expanded worst case across all angles. Header no longer carries a single global `x_start`/`x_delta`/`n_rows` — replaced with `*_nominal` reference fields plus a new Per-Angle Geometry Table (`x_start`, `x_delta`, `n_frames`, `n_rows` per angle) and a ragged Row Table / Waveform Data block sized per angle. **Not compatible with pre-v6 readers** that assume uniform geometry. `sras_viewer.py` reads v6 natively (per-angle `n_rows`/`n_frames`/`x_start`). |
| 7 | Adds an optional trailing **Cache Tail** (`CACH` section, see [Cache Tail (v7)](#cache-tail-v7)) after the ragged waveform data, holding independently-present, independently-updatable per-angle DC (`SDCB`: dc3_mv + dc4_mv) and FFT (`SFFT`: peak_freq_mhz) blocks, so previously-computed images redisplay instantly instead of being recomputed. Header / angle table / geometry table / row table / preamble blocks / background block / waveform data are byte-identical to v6 — only the version byte and the optional Cache Tail differ. Scans still come off the scope as v6; `sras_viewer.py`'s "Convert" menu ("Batch Compute DC and Store" / "Batch Compute FFT and Store") converts a file to v7 **in place** on first use, or updates an existing v7 file's cache blocks, without rewriting any waveform bytes. Supersedes the removed "Pre-process and Save as v5" workflow, which was never available for v6 sources since the flat v5 `PREC` layout can't represent per-angle geometry. |
+456
View File
@@ -0,0 +1,456 @@
#!/usr/bin/env python3
"""Write an aligned, cropped .sras file from an AlignmentResult.
The alignment machinery in sras_compute never modifies a scan: it produces an
AlignmentResult, and every consumer resamples on the fly (apply_alignment for
the display, reproject_mask for the overlay). That is right for a viewer, but it
means the aligned stack cannot leave the process — no other tool can read it,
and re-opening the scan re-does the registration.
This module bakes an alignment into a new file. Each angle is resampled onto the
shared canvas that AlignmentResult already defines, cropped to the caller's
window, so every output angle ends up with *identical* geometry: same rows, same
frames, same X/Y coordinates. Rotation and translation are gone, absorbed into
where each waveform sits. The result is an ordinary v6 file that opens already
aligned, and registering it against itself returns identity.
Two deliberate choices, both about not inventing data:
* The resample is a nearest-neighbour **gather** of whole waveforms, never an
interpolation. Averaging two neighbouring pixels' CH1 packets would produce
a waveform the instrument never measured, whose FFT peak is not the peak of
either — meaningless for a technique whose entire output is that peak
frequency. So each output pixel gets exactly one source pixel's three
waveforms, verbatim, and the cost is that some source pixels are duplicated
and others dropped. This matches apply_alignment's order=0 for the same
reason.
* Output pixels with no source pixel (the canvas corners a rotated scan cannot
reach, and anything outside the crop's coverage) are filled with the ADC
code for 0 mV, not with zero. See _fill_row.
Depends only on numpy/sras_format/sras_compute — no Qt — so it is directly
unit-testable and importable from a worker thread.
"""
import os
import struct
from dataclasses import dataclass, field
from pathlib import Path
import numpy as np
import sras_compute as compute
from sras_compute import AlignmentResult
from sras_format import GEO_FMT_V6, HDR_FMT_V6, SrasFile, mv_to_adc
# GEO_FMT_V6 stores n_rows as ">H" and n_frames as ">I". A canvas that overflows
# either is not representable, and silently truncating would write a file whose
# geometry table disagrees with its waveform block.
_MAX_ROWS = 0xFFFF
_MAX_FRAMES = 0xFFFFFFFF
# Slack, in source pixels, on the in-bounds test at the very edge of a source
# array. Absorbs the ~1e-13 of float noise an exactly-integer affine picks up
# from being built in mm space; see _in_bounds.
_EDGE_TOL = 1e-6
# Output rows per write() call. One row is n_channels * n_cols * spf bytes —
# ~1.5 MB on a full-size scan — so a handful of rows keeps the peak buffer in
# the low tens of MB no matter how large the scan is.
_ROW_CHUNK = 8
@dataclass
class ExportPlan:
"""What write_aligned_sras would produce, without producing it.
Derived from the affine transforms alone — no waveform bytes are read — so
the wizard can call it on every ROI edit to keep a live size estimate and
per-angle coverage readout in front of the user *before* they commit to a
multi-gigabyte write.
"""
n_rows: int
n_frames: int
n_angles: int
bytes_per_angle: int
total_bytes: int
valid_px: dict[int, int] # output pixels with a source pixel
warnings: list[str] = field(default_factory=list)
def coverage_frac(self, angle_idx: int) -> float:
px = self.n_rows * self.n_frames
return (self.valid_px.get(angle_idx, 0) / px) if px else 0.0
def empty_angles(self) -> list[int]:
"""Angles that would be written as pure padding — no output pixel of
theirs has a source pixel."""
return [a for a in range(self.n_angles) if not self.valid_px.get(a, 0)]
def _src_coords(t, rows, n_cols: int) -> tuple[np.ndarray, np.ndarray]:
"""Fractional source (row, col) coordinates for whole output rows.
*rows* is an array of output row indices; both results are shaped
(len(rows), n_cols).
"""
cols = np.arange(n_cols, dtype=np.float64)
r = np.asarray(rows, dtype=np.float64)[:, None]
sr = t.matrix[0, 0] * r + t.matrix[0, 1] * cols + t.offset[0]
sc = t.matrix[1, 0] * r + t.matrix[1, 1] * cols + t.offset[1]
return sr, sc
def _round_idx(coord: np.ndarray) -> np.ndarray:
"""Nearest source index, rounding halves away from zero.
floor(x + 0.5), not np.rint: scipy.ndimage's order=0 rounds halves away
from zero while np.rint rounds them to even, and these have to be the same
source pixels the apply_alignment(order=0) preview drew. Exact halves are
not a corner case here — the canvas is snapped to the reference angle's own
pixel grid (see canvas_for_params), so an unrotated angle lands on
half-integers wherever its row pitch differs from the reference's.
"""
return np.floor(coord + 0.5).astype(np.int64)
def _in_bounds(sr: np.ndarray, sc: np.ndarray,
n_rows: int, n_frames: int) -> np.ndarray:
"""Which output pixels have a source pixel, by scipy's mode="constant" rule
plus a tolerance at the edge.
Tested on the *fractional* coordinate against the range of sample centres,
[0, n-1] inclusive — deliberately not on the rounded index. The two differ
around the whole rim: a coordinate of -0.4 rounds to a perfectly valid index
0, but scipy calls it out of bounds and writes cval there, so testing the
rounded index would put a one-pixel rim of real data everywhere the Aligned
View shows padding.
_EDGE_TOL is why this is not literally scipy's test. The affine is built
from a chain of mm-space multiplications, so an exactly-integer transform
comes out a few times 1e-13 off (the reference angle's offset lands on
-20 - 7e-15 rather than -20). Bare >= 0.0 then rejects that angle's entire
first row, and <= n-1 its last column — for the *reference* angle, whose
whole role is to pass through as an exact integer crop. The tolerance is
seven orders of magnitude above that noise and seven below the half-pixel
scale at which a rounding decision is ever meaningful, so it can only ever
change pixels whose scipy answer was itself decided by rounding noise.
"""
return ((sr >= -_EDGE_TOL) & (sr <= n_rows - 1 + _EDGE_TOL)
& (sc >= -_EDGE_TOL) & (sc <= n_frames - 1 + _EDGE_TOL))
# Output rows evaluated per numpy call when counting coverage. Counting row by
# row costs one small matmul per row (hundreds of milliseconds per angle on a
# full-size scan, on every ROI edit); counting the whole canvas at once needs
# hundreds of MB of index arrays. Blocking gets both: ~10 numpy calls per angle
# against ~30 MB of live index arrays.
_COUNT_BLOCK = 128
def _count_in_bounds(t, n_rows: int, n_cols: int,
src_rows: int, src_frames: int) -> int:
"""How many of the n_rows x n_cols output pixels have a source pixel."""
total = 0
for start in range(0, n_rows, _COUNT_BLOCK):
rows = np.arange(start, min(start + _COUNT_BLOCK, n_rows))
sr, sc = _src_coords(t, rows, n_cols)
total += int(np.count_nonzero(_in_bounds(sr, sc, src_rows, src_frames)))
return total
def plan_export(sras: SrasFile, result: AlignmentResult) -> ExportPlan:
"""Geometry, size and per-angle coverage of the file *result* would export.
Coverage is counted from the actual per-pixel index arrays rather than
approximated by the footprint parallelogram's area, because the two differ
exactly where it matters — a crop that clips one angle's scan window — and
this number is what tells the user an angle will come out mostly empty. No
waveform bytes are read, so it stays fast enough to call on every ROI edit.
"""
n_rows, n_cols = result.canvas_shape
n_angles = sras.n_angles
warnings: list[str] = []
valid_px: dict[int, int] = {}
for a in range(n_angles):
t = result.per_angle.get(a)
if t is None:
valid_px[a] = 0
warnings.append(f"Angle {a} has no transform and will be all padding.")
continue
src_rows, src_frames = sras.image_shape(a)
valid_px[a] = _count_in_bounds(t, n_rows, n_cols, src_rows, src_frames)
bytes_per_angle = (n_rows * sras.n_channels * n_cols
* sras.samples_per_frame * sras.bytes_per_sample)
# Built before the remaining warnings so they can be phrased with the
# plan's own coverage_frac rather than a second copy of the same division.
plan = ExportPlan(n_rows=n_rows, n_frames=n_cols, n_angles=n_angles,
bytes_per_angle=bytes_per_angle,
total_bytes=bytes_per_angle * n_angles,
valid_px=valid_px, warnings=warnings)
if n_rows > _MAX_ROWS:
warnings.append(
f"Crop is {n_rows} rows; the .sras geometry table caps rows at "
f"{_MAX_ROWS}. Narrow the ROI in Y.")
if n_cols > _MAX_FRAMES:
warnings.append(f"Crop is {n_cols} frames; the cap is {_MAX_FRAMES}.")
for a in range(n_angles):
frac = plan.coverage_frac(a)
if frac == 0.0:
warnings.append(
f"Angle {a} has no data inside this crop — it will be written "
f"as all padding.")
elif frac < 0.10:
warnings.append(
f"Angle {a} covers only {frac * 100:.1f}% of the crop.")
if sras.background is None:
warnings.append(
"Input has no background waveform (pre-v4 scan); a zero background "
"is written, which makes background subtraction a no-op.")
if sras.version != 6:
warnings.append(f"Input is v{sras.version}; the export is written as v6.")
if sras.scan_aborted:
warnings.append(
f"Input scan was aborted: only its {n_angles} complete angle(s) "
f"are exported.")
return plan
def _fill_row(sras: SrasFile, n_cols: int, dtype) -> np.ndarray:
"""One output row of pure padding, shape (n_channels, n_cols, spf).
Filled per channel with the ADC code for 0 mV, not with 0. Zero ADC decodes
to (0 - yoff) * ymult + yzero, which for a real scope preamble is a long way
from 0 mV — often far enough to sit above the CH4 mask threshold, which
would paint a solid rectangle of "valid" pixels around the sample and make
every DC image and every ROI statistic wrong. Rounding to the integer code
lands within half an ADC step of 0 mV, which is as close as the format can
represent.
"""
info = np.iinfo(dtype)
codes = [int(np.clip(round(mv_to_adc(0.0, *sras.cal(ch))), info.min, info.max))
for ch in range(sras.n_channels)]
row = np.empty((sras.n_channels, n_cols, sras.samples_per_frame), dtype=dtype)
for ch, code in enumerate(codes):
row[ch] = code
return row
class _SourceReader:
"""Gives the gather source rows without ever reading one twice.
This is the difference between a usable export and an unusable one, and it
is entirely about read amplification. A rotated angle maps one output row to
a *diagonal* line across the source array, so the pixels of a single output
row come from hundreds of different source rows — on a full-size scan, a
~1.4 MB source row each. Indexing a memmap pixel by pixel in output order
therefore re-faults nearly the whole angle for every output row: terabytes
of paging for a gigabyte of data.
So rows are served from a contiguous *band* held in RAM. The band for a
chunk of output rows is read in one sequential slice, and because output
rows advance monotonically through the source, consecutive chunks' bands
barely overlap: each source row is read about once, and the whole job costs
roughly 2x the source size in reads rather than a thousand times it.
Small angles skip the machinery — if the whole block fits the budget it is
materialized once and every band is a view of it.
One honest caveat: a chunk whose diagonal spans more source rows than the
budget allows still gets the band it asked for, so the budget can be
overshot. The overshoot is bounded by the span of _ROW_CHUNK output rows,
and in the worst case (an extreme rotation on a huge scan) that is the whole
angle — i.e. no worse than the in-RAM path above. Accepted deliberately:
correctness of the gather is not negotiable, and the alternative is the
memmap thrashing this class exists to avoid.
"""
def __init__(self, sras: SrasFile, angle_idx: int, budget: int):
self._src = sras.data[angle_idx]
self._n_rows = self._src.shape[0]
self._row_bytes = max(1, self._src[0].nbytes)
self._whole = np.asarray(self._src) if self._src.nbytes <= budget else None
# Leave room for the output buffer and the index arrays alongside.
self._max_band = max(1, int(budget * 0.5) // self._row_bytes)
self._band = None
self._lo = self._hi = 0
def band(self, lo: int, hi: int) -> tuple[np.ndarray, int]:
"""Rows [lo, hi) as an in-RAM array, plus the index its row 0 holds."""
lo = max(0, min(lo, self._n_rows))
hi = max(lo + 1, min(hi, self._n_rows))
if self._whole is not None:
return self._whole, 0
if self._band is None or lo < self._lo or hi > self._hi:
# Read a little more than asked so a chunk whose band creeps
# forward by a few rows does not re-read the whole span.
span = min(self._max_band, max(hi - lo, self._max_band // 2))
self._lo = lo
self._hi = min(self._n_rows, lo + span)
if self._hi < hi: # band cannot cover the ask
self._hi = hi
self._band = np.asarray(self._src[self._lo:self._hi])
return self._band, self._lo
def close(self):
self._whole = None
self._band = None
def write_aligned_sras(sras: SrasFile, result: AlignmentResult, out_path,
*, progress_cb=None, should_stop=None,
budget: int | None = None) -> Path:
"""Write *sras*, aligned per *result* and cropped to its canvas, to a new
v6 .sras file. Returns the path written.
*result* is used exactly as given: crop the canvas first with
compute.crop_alignment_result, whose offset shift makes the cropped result
resample precisely the window the user selected.
No cache tail is written. Any DC/FFT the input had cached is indexed by the
input's grid and is meaningless on the new one, so — like sras_edit_scans —
the export drops it and lets the viewer recompute.
Writes to a sibling ".part" file and os.replace()s it into position on
success, unlinking it on error or cancellation: a half-written .sras is not
detectably broken (the v6 parser treats a short file as an aborted scan and
opens it happily), so it must never be left where the user might load it.
*should_stop* is polled once per output row chunk; returning True aborts and
raises nothing — the partial file is removed and the returned path will not
exist, so callers must check.
"""
out_path = Path(out_path)
n_rows, n_cols = result.canvas_shape
n_ch, spf = sras.n_channels, sras.samples_per_frame
n_angles = sras.n_angles
if n_rows <= 0 or n_cols <= 0:
raise ValueError(f"empty canvas: {n_rows} x {n_cols}")
if n_rows > _MAX_ROWS:
raise ValueError(
f"{n_rows} rows exceeds the .sras per-angle geometry limit of "
f"{_MAX_ROWS}; crop further in Y")
if n_cols > _MAX_FRAMES:
raise ValueError(f"{n_cols} frames exceeds the limit of {_MAX_FRAMES}")
missing = [a for a in range(n_angles) if a not in result.per_angle]
if missing:
raise ValueError(f"alignment result has no transform for angle(s) {missing}")
# The source's waveform blocks are live read-only memmaps into sras.path,
# so writing over it would corrupt the very reads the gather is making.
if out_path.exists() and out_path.samefile(sras.path):
raise ValueError(
"refusing to export onto the source scan; choose another filename")
dtype = np.dtype(np.int8 if sras.bytes_per_sample == 1 else ">i2")
x0_mm, y0_mm = result.canvas_origin_mm
y_rows = (y0_mm + np.arange(n_rows) * result.canvas_dy_mm).astype(">f4")
# Reference-only header fields. v6/v7 inputs have real ones to carry over;
# for a legacy input describe the canvas we are actually writing.
if sras.x_start_nominal_mm is not None:
nominal = (sras.x_start_nominal_mm, sras.y_start_nominal_mm,
sras.x_delta_nominal_mm, sras.y_delta_nominal_mm,
sras.row_spacing_mm)
else:
nominal = (x0_mm, y0_mm,
n_cols * sras.pixel_x_mm, n_rows * result.canvas_dy_mm,
result.canvas_dy_mm)
header = struct.pack(
HDR_FMT_V6, b"SRAS", 6, n_angles,
float(nominal[0]), float(nominal[1]), float(nominal[2]),
float(nominal[3]), float(nominal[4]),
sras.velocity_mm_s, sras.laser_freq_hz, spf, sras.sample_rate_hz,
sras.bytes_per_sample, n_ch)
# Every angle now shares one grid, so the ragged v6 tables collapse to
# n_angles copies of the same record. x_delta is the reference angle's own
# pitch (the canvas is its grid extended), which is velocity/laser_freq
# exactly, so x_axis_mm() stays self-consistent on re-read.
geo = struct.pack(GEO_FMT_V6, float(x0_mm), float(sras.pixel_x_mm),
int(n_cols), int(n_rows)) * n_angles
budget = compute.memory_budget_bytes() if budget is None else max(1, budget)
total_chunks = max(1, n_angles * ((n_rows + _ROW_CHUNK - 1) // _ROW_CHUNK))
done_chunks = 0
cancelled = False
part_path = out_path.with_name(out_path.name + ".part")
try:
with open(part_path, "wb") as fout:
fout.write(header)
fout.write(sras.angles_deg.astype(">f4").tobytes())
fout.write(geo)
fout.write(y_rows.tobytes() * n_angles)
fout.write(sras.encoded_preambles())
fout.write(sras.encoded_background())
pad = _fill_row(sras, n_cols, dtype)
for a in range(n_angles):
t = result.per_angle[a]
reader = _SourceReader(sras, a, budget)
src_rows, src_frames = sras.image_shape(a)
try:
for chunk_start in range(0, n_rows, _ROW_CHUNK):
if should_stop is not None and should_stop():
cancelled = True
break
chunk = np.arange(chunk_start,
min(chunk_start + _ROW_CHUNK, n_rows))
sr, sc = _src_coords(t, chunk, n_cols)
ok = _in_bounds(sr, sc, src_rows, src_frames)
# Clip rather than trust: _EDGE_TOL admits coordinates a
# hair outside the array, and an index off the end here
# would silently read the wrong row of the band.
idx_r = np.clip(_round_idx(sr), 0, src_rows - 1)
idx_c = np.clip(_round_idx(sc), 0, src_frames - 1)
# One band read covers the whole chunk: every source row
# any of these output rows touches, in one sequential
# slice. See _SourceReader.
if ok.any():
band, base = reader.band(int(idx_r[ok].min()),
int(idx_r[ok].max()) + 1)
else:
band, base = None, 0
for i in range(len(chunk)):
out = pad.copy()
keep = ok[i]
if keep.any():
# The two advanced indices are separated by a
# slice, so numpy puts the gathered axis first:
# (n_sel, n_ch, spf). Move it behind channels.
out[:, keep, :] = band[
idx_r[i][keep] - base, :, idx_c[i][keep], :
].transpose(1, 0, 2)
# out is C-contiguous, so the buffer protocol
# writes it straight out — .tobytes() would
# copy a full row per row written.
fout.write(out)
done_chunks += 1
if progress_cb is not None:
progress_cb(int(done_chunks / total_chunks * 100))
finally:
reader.close()
if cancelled:
break
if not cancelled:
fout.flush()
os.fsync(fout.fileno())
if cancelled:
part_path.unlink(missing_ok=True)
return out_path
os.replace(part_path, out_path)
except BaseException:
part_path.unlink(missing_ok=True)
raise
if progress_cb is not None:
progress_cb(100)
return out_path
+188 -163
View File
@@ -3,7 +3,23 @@
sras_average.py — Waveform-averaging utility for .sras files. sras_average.py — Waveform-averaging utility for .sras files.
Reduces memory footprint by coherently averaging every N consecutive frames Reduces memory footprint by coherently averaging every N consecutive frames
along the acquisition axis, writing a new .sras file with n_frames / N frames. along the acquisition axis, writing a new v6 .sras file with (about)
n_frames / N frames per angle.
Handles v6/v7 only (per-angle ragged geometry). A v7 input's cache tail is
dropped — it's indexed by frame, which this changes — so the output is always
written as v6; the viewer recomputes DC/FFT on next open. Reads via memmap and
writes in row-chunks sized to a memory budget (default 1024 MB, override with
the SRAS_MEM_BUDGET_MB env var), so peak RAM stays bounded regardless of file
size — this is what makes the tool usable on multi-hundred-GB scans.
Averaging every N frames also coarsens the physical X spacing between output
frames (each frame is a distinct stage position — see scan_format.md's
Spatial Mapping section: x_k = x_start + k * velocity_mm_s / laser_freq_hz) —
so the output header's laser_freq_hz is divided by N to keep x_axis_mm()
correct on the averaged file. This means the GUI's "Laser freq" info label
will show that adjusted value rather than the scope's real setting for an
averaged file; v6/v7 has no separate field for effective pixel pitch.
Usage: Usage:
python sras_average.py input.sras output.sras --n 10 python sras_average.py input.sras output.sras --n 10
@@ -13,27 +29,40 @@ Options:
--n INT Number of frames to average into one (required). --n INT Number of frames to average into one (required).
--discard-remainder Drop trailing frames that don't fill a complete group. --discard-remainder Drop trailing frames that don't fill a complete group.
Default: include a partial average for the last group. Default: include a partial average for the last group.
Environment:
SRAS_MEM_BUDGET_MB Ceiling on one row-chunk's working memory, in MB
(default 1024). Lower it on a memory-constrained
machine; the tool just takes more, smaller chunks.
""" """
import sys
import struct
import argparse import argparse
import numpy as np import os
import shutil
import struct
import sys
from pathlib import Path from pathlib import Path
# --------------------------------------------------------------------------- import numpy as np
# Header format — must match sras_viewer.py exactly
# ---------------------------------------------------------------------------
HDR_FMT = ">4sBHHffffIIdBB" from sras_format import GEO_FMT_V6, HDR_FMT_V6, SrasFile
HDR_SIZE = struct.calcsize(HDR_FMT) # 43 bytes
_SUPPORTED = (6, 7)
_DEFAULT_BUDGET_MB = 1024
# Throttle per-chunk progress printing to roughly this many lines per angle,
# so a huge angle (thousands of chunks) doesn't flood stdout while a small
# one still gets to print every chunk.
_MAX_PROGRESS_LINES = 40
def parse_args(): def parse_args():
p = argparse.ArgumentParser( p = argparse.ArgumentParser(
description="Average every N waveforms in a .sras file and write a new file." description="Average every N waveforms in a .sras file and write a new file.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
) )
p.add_argument("input", help="Input .sras file") p.add_argument("input", help="Input .sras file")
p.add_argument("output", help="Output .sras file") p.add_argument("output", help="Output .sras file")
p.add_argument("--n", type=int, required=True, metavar="N", p.add_argument("--n", type=int, required=True, metavar="N",
help="Number of consecutive frames to average into one") help="Number of consecutive frames to average into one")
@@ -42,143 +71,142 @@ def parse_args():
return p.parse_args() return p.parse_args()
def read_sras(path: Path): def _memory_budget_bytes() -> int:
"""Read all sections of a .sras file and return them as a dict.""" return int(os.environ.get("SRAS_MEM_BUDGET_MB", _DEFAULT_BUDGET_MB)) * 1024 * 1024
with open(path, "rb") as f:
header_bytes = f.read(HDR_SIZE)
fields = struct.unpack(HDR_FMT, header_bytes)
(magic, ver, n_angles, n_rows, x_start, x_delta, vel, freq,
n_frames_hdr, spf, sr, bps, n_ch) = fields
if magic != b"SRAS":
raise ValueError(f"Not a .sras file (bad magic: {magic!r})")
if ver not in (2, 3, 4):
raise ValueError(f"Unsupported .sras version: {ver}")
with open(path, "rb") as f:
f.seek(HDR_SIZE)
angles = f.read(n_angles * 4) # big-endian float32 array, raw bytes
y_pos = f.read(n_rows * 4) # big-endian float32 array, raw bytes
preambles = [] # list of raw bytes (length-prefixed strings)
if ver >= 3:
for _ in range(n_ch):
(length,) = struct.unpack(">H", f.read(2))
preambles.append(f.read(length))
background = b"" # raw bytes for the v4 background block
if ver >= 4:
(n_bg,) = struct.unpack(">I", f.read(4))
background = f.read(n_bg)
raw = f.read() # all waveform data
# -----------------------------------------------------------------------
# Determine actual frame count from file size (the header value can be
# wrong — the viewer does the same correction)
# -----------------------------------------------------------------------
total_samples = len(raw) // bps
samples_per_pixel = n_ch * spf # samples in one (angle, row, frame) cell
samples_per_full = n_angles * n_rows * samples_per_pixel
actual_n_frames = total_samples // (n_angles * n_rows * samples_per_pixel)
good_bytes = actual_n_frames * samples_per_full * bps
# Decode waveform data
dtype = np.int8 if bps == 1 else ">i2"
data = np.frombuffer(raw[:good_bytes], dtype=dtype)
data = data.reshape(n_angles, n_rows, n_ch, actual_n_frames, spf)
# Work in int16 (safe intermediate for both int8 and int16 inputs)
data = data.astype(np.int16)
return {
"ver": ver, "n_angles": n_angles, "n_rows": n_rows,
"x_start": x_start, "x_delta": x_delta, "vel": vel, "freq": freq,
"n_frames_hdr": n_frames_hdr, "spf": spf, "sr": sr,
"bps": bps, "n_ch": n_ch,
"angles_raw": angles, "y_pos_raw": y_pos,
"preambles": preambles, "background": background,
"data": data, # shape: (n_angles, n_rows, n_ch, n_frames, spf), int16
}
def average_frames(data: np.ndarray, n: int, discard_remainder: bool) -> np.ndarray: def _plan_angle(n_frames_in: int, n: int, discard_remainder: bool) -> tuple[int, int, int]:
"""Average every N frames along axis 3. """(n_full, remainder, n_frames_out) for averaging one angle's frames."""
n_full = n_frames_in // n
remainder = n_frames_in % n
n_out = n_full + (1 if remainder and not discard_remainder else 0)
return n_full, remainder, n_out
data shape: (n_angles, n_rows, n_ch, n_frames, spf)
Returns array of shape (n_angles, n_rows, n_ch, n_out, spf). def _chunk_rows(n_rows: int, n_channels: int, n_frames_in: int,
samples_per_frame: int, budget: int) -> int:
"""How many rows to hold in RAM at once so one chunk's working buffers
(source block, mean's float64 accumulator, output block) fit the budget.
The 4 bytes/sample below is a deliberate middle estimate, not a sum of
the three buffers: the int16 source and int16 output are 2 each, and
only mean()'s float64 result is 8, over the reduced frame axis rather
than the full block. Raise SRAS_MEM_BUDGET_MB if a machine still runs
tight at a large --n."""
bytes_per_row = max(1, n_channels * n_frames_in * samples_per_frame * 4)
return max(1, min(n_rows, budget // bytes_per_row))
def _average_block(block: np.ndarray, n: int, discard_remainder: bool,
bps: int) -> np.ndarray:
"""Average every N frames of a (rows, n_ch, n_frames, spf) block along
the frame axis, returning the result already encoded in the on-disk
dtype: int8 (clipped) if bps == 1, else big-endian int16.
Cast to native int16 (not float32) before calling .mean(): numpy's mean()
uses a float64 accumulator by default for integer input, matching the
original implementation exactly (which kept the whole file as int16 and
called .mean() directly). A float32 cast here would use a float32
accumulator instead — for large group sizes that can round the sum
differently than float64 and, after the int16 cast below, occasionally
land on a value 1 ADC count away from the original tool's output.
""" """
n_frames = data.shape[3] n_frames = block.shape[2]
n_full = n_frames // n
n_full = n_frames // n
remainder = n_frames % n remainder = n_frames % n
# Average full groups using reshape-trick (no Python loop) parts = []
if n_full > 0: if n_full:
full = data[:, :, :, :n_full * n, :] # trim to full groups full = block[:, :, :n_full * n, :].astype(np.int16)
full = full.reshape(data.shape[0], data.shape[1], data.shape[2], full = full.reshape(block.shape[0], block.shape[1], n_full, n, block.shape[3])
n_full, n, data.shape[4]) # (..., n_out, n, spf) parts.append(full.mean(axis=3).astype(np.int16))
averaged = full.mean(axis=4).astype(np.int16) # (..., n_out, spf) if remainder and not discard_remainder:
tail = block[:, :, n_full * n:, :].astype(np.int16)
parts.append(tail.mean(axis=2, keepdims=True).astype(np.int16))
if not parts:
averaged = np.empty((*block.shape[:2], 0, block.shape[3]), dtype=np.int16)
else: else:
averaged = np.empty((*data.shape[:3], 0, data.shape[4]), dtype=np.int16) averaged = parts[0] if len(parts) == 1 else np.concatenate(parts, axis=2)
if remainder > 0 and not discard_remainder: if bps == 1:
tail = data[:, :, :, n_full * n:, :] # shape (..., remainder, spf) return np.clip(averaged, -128, 127).astype(np.int8)
tail_avg = tail.mean(axis=3, keepdims=True).astype(np.int16) return averaged.astype(">i2")
averaged = np.concatenate([averaged, tail_avg], axis=3)
return averaged
def write_sras(path: Path, src: dict, data_out: np.ndarray): def write_v6_averaged(sras: SrasFile, out_path: Path, n: int,
"""Write a new .sras file with the averaged waveform data.""" discard_remainder: bool, budget: int | None = None) -> list[int]:
ver = src["ver"] """Write *sras* averaged every N frames to a new v6 .sras file, streaming
bps = src["bps"] row-chunks per angle so peak RAM never holds more than one chunk (bounded
n_out = data_out.shape[3] by *budget* bytes, default from SRAS_MEM_BUDGET_MB).
Stages to a sibling '.part' file and os.replace()s it into position on
success, per scan_format.md's requirement that any .sras writer must
never leave a half-written file visible under its final name (a short
v6 file opens successfully with the wrong angle/frame count rather than
failing loudly).
Returns the per-angle output frame counts.
"""
budget = _memory_budget_bytes() if budget is None else max(1, budget)
n_ch = sras.n_channels
spf = sras.samples_per_frame
bps = sras.bytes_per_sample
plans = [_plan_angle(int(sras.n_frames[a]), n, discard_remainder)
for a in range(sras.n_angles)]
n_out_per_angle = [p[2] for p in plans]
# Pack header — update only n_frames_hdr; everything else stays the same
header = struct.pack( header = struct.pack(
HDR_FMT, HDR_FMT_V6, b"SRAS", 6, sras.n_angles,
b"SRAS", sras.x_start_nominal_mm, sras.y_start_nominal_mm,
ver, sras.x_delta_nominal_mm, sras.y_delta_nominal_mm,
src["n_angles"], sras.row_spacing_mm, sras.velocity_mm_s, sras.laser_freq_hz / n,
src["n_rows"], spf, sras.sample_rate_hz, bps, n_ch,
src["x_start"],
src["x_delta"],
src["vel"],
src["freq"],
n_out, # updated frame count
src["spf"],
src["sr"],
bps,
src["n_ch"],
) )
# Encode waveform data back to original dtype geo = bytearray()
if bps == 1: for a in range(sras.n_angles):
raw_out = np.clip(data_out, -128, 127).astype(np.int8).tobytes() geo += struct.pack(GEO_FMT_V6, float(sras.x_start_mm[a]),
else: float(sras.x_delta_mm_per_angle[a]),
# big-endian int16 int(n_out_per_angle[a]), int(sras.n_rows[a]))
raw_out = data_out.astype(">i2").tobytes()
with open(path, "wb") as f: part_path = out_path.with_name(out_path.name + ".part")
f.write(header) try:
f.write(src["angles_raw"]) with open(part_path, "wb") as f:
f.write(src["y_pos_raw"]) f.write(header)
f.write(sras.angles_deg.astype(">f4").tobytes())
f.write(bytes(geo))
for a in range(sras.n_angles):
f.write(sras.y_pos_per_angle[a].astype(">f4").tobytes())
f.write(sras.encoded_preambles())
f.write(sras.encoded_background())
if ver >= 3: for a in range(sras.n_angles):
for preamble_bytes in src["preambles"]: n_rows = int(sras.n_rows[a])
f.write(struct.pack(">H", len(preamble_bytes))) n_frames_in = int(sras.n_frames[a])
f.write(preamble_bytes) chunk_rows = _chunk_rows(n_rows, n_ch, n_frames_in, spf, budget)
total_chunks = max(1, -(-n_rows // chunk_rows))
print_every = max(1, total_chunks // _MAX_PROGRESS_LINES)
if ver >= 4: data = sras.data[a]
bg = src["background"] for i, r0 in enumerate(range(0, n_rows, chunk_rows)):
f.write(struct.pack(">I", len(bg))) r1 = min(r0 + chunk_rows, n_rows)
f.write(bg) block = np.asarray(data[r0:r1])
out_block = _average_block(block, n, discard_remainder, bps)
f.write(out_block.tobytes())
if total_chunks > 1 and (i % print_every == 0 or r1 == n_rows):
print(f" angle {a + 1}/{sras.n_angles}: "
f"{r1}/{n_rows} rows", flush=True)
f.write(raw_out) f.flush()
os.fsync(f.fileno())
os.replace(part_path, out_path)
except BaseException:
part_path.unlink(missing_ok=True)
raise
return n_out_per_angle
def main(): def main():
@@ -188,62 +216,59 @@ def main():
print("Error: --n must be at least 1.", file=sys.stderr) print("Error: --n must be at least 1.", file=sys.stderr)
sys.exit(1) sys.exit(1)
in_path = Path(args.input) in_path = Path(args.input)
out_path = Path(args.output) out_path = Path(args.output)
if not in_path.exists(): if not in_path.exists():
print(f"Error: input file not found: {in_path}", file=sys.stderr) print(f"Error: input file not found: {in_path}", file=sys.stderr)
sys.exit(1) sys.exit(1)
if out_path.resolve() == in_path.resolve(): if out_path.resolve() == in_path.resolve():
print("Error: output path must differ from input path.", file=sys.stderr) print("Error: output path must differ from input path.", file=sys.stderr)
sys.exit(1) sys.exit(1)
print(f"Reading {in_path} ...", flush=True) print(f"Reading {in_path} ...", flush=True)
src = read_sras(in_path) sras = SrasFile(str(in_path))
if sras.version not in _SUPPORTED:
print(f"Error: unsupported .sras version: {sras.version} "
f"(this tool handles v{'/v'.join(map(str, _SUPPORTED))} only)",
file=sys.stderr)
sys.exit(1)
n_frames_in = src["data"].shape[3] aborted_note = " (scan aborted; trailing angle(s) already excluded)" if sras.scan_aborted else ""
print(f" Version : v{src['ver']}") print(f" Version : v{sras.version}")
print(f" Angles : {src['n_angles']}") print(f" Angles : {sras.n_angles}{aborted_note}")
print(f" Rows : {src['n_rows']}") print(f" Channels : {sras.n_channels}")
print(f" Frames (actual): {n_frames_in}") print(f" Samples/frame: {sras.samples_per_frame}")
print(f" Channels : {src['n_ch']}") print(f" Bytes/sample : {sras.bytes_per_sample}")
print(f" Samples/frame : {src['spf']}")
print(f" Bytes/sample : {src['bps']}")
if args.n == 1: if args.n == 1:
print("--n 1: no averaging needed; copying file as-is.") print("--n 1: no averaging needed; copying file as-is.")
import shutil
shutil.copy2(in_path, out_path) shutil.copy2(in_path, out_path)
print(f"Wrote {out_path}") print(f"Wrote {out_path}")
return return
if args.n > n_frames_in: print(f"\n{'idx':>4} {'angle_deg':>10} {'rows':>6} {'frames_in':>10} {'frames_out':>11}")
print(f"Warning: --n ({args.n}) exceeds available frames ({n_frames_in}). " for a in range(sras.n_angles):
"The entire dataset will be averaged into a single frame.") n_frames_in = int(sras.n_frames[a])
n_full, remainder, n_out = _plan_angle(n_frames_in, args.n, args.discard_remainder)
print(f"{a:>4} {sras.angles_deg[a]:>10.4f} {int(sras.n_rows[a]):>6} "
f"{n_frames_in:>10} {n_out:>11}")
if args.n > n_frames_in:
print(f" Warning: --n ({args.n}) exceeds angle {a}'s frames "
f"({n_frames_in}); it collapses to a single frame.")
print(f"\nAveraging every {args.n} frames ...", flush=True) if sras.version == 7:
data_out = average_frames(src["data"], args.n, args.discard_remainder) print("\nNote: input has a v7 cache tail; it is indexed by frame count "
n_frames_out = data_out.shape[3] "and will be dropped. The viewer will recompute DC/FFT on next open.")
n_full = n_frames_in // args.n print(f"\nAveraging every {args.n} frames and writing {out_path} ...", flush=True)
remainder = n_frames_in % args.n n_out_per_angle = write_v6_averaged(sras, out_path, args.n, args.discard_remainder)
if remainder and not args.discard_remainder:
status = f"({n_full} full groups + 1 partial group of {remainder})"
elif remainder and args.discard_remainder:
status = f"({n_full} full groups, {remainder} trailing frames discarded)"
else:
status = f"({n_full} full groups)"
print(f" {n_frames_in} frames -> {n_frames_out} frames {status}") in_mb = in_path.stat().st_size / 1024**2
print(f"\nWriting {out_path} ...", flush=True)
write_sras(out_path, src, data_out)
in_mb = in_path.stat().st_size / 1024**2
out_mb = out_path.stat().st_size / 1024**2 out_mb = out_path.stat().st_size / 1024**2
print(f"\n Frames out: {n_out_per_angle}")
print(f" Input size : {in_mb:.1f} MB") print(f" Input size : {in_mb:.1f} MB")
print(f" Output size: {out_mb:.1f} MB ({out_mb/in_mb*100:.1f}% of input)") print(f" Output size: {out_mb:.1f} MB ({out_mb / in_mb * 100:.1f}% of input)")
print("Done.") print("Done.")
+1928
View File
File diff suppressed because it is too large Load Diff
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""
sras_edit_scans.py — Remove one or more angle scans from a .sras file.
A .sras file holds one or more "angles" (rotation positions); the viewer
cross-correlates each non-reference angle against the reference to align
them. If one angle's acquisition went wrong (stage glitch, bad trigger,
laser dropout, ...) it throws off that alignment for the whole file. This
tool drops the bad angle(s) and renumbers the rest, writing a new .sras file
with everything else — waveform samples, calibration preambles, background
waveform, row/geometry tables — carried over byte-for-byte.
Handles v2-v7. Any precomputed FFT/DC cache (v5 PREC tail, v7 CACH tail) is
dropped on write, since it's indexed by angle and would be stale/misaligned
after renumbering; the viewer just recomputes it next time the file opens.
This tool only ever *drops* angles — every kept angle's waveform bytes and
geometry are carried across verbatim. To write a file whose angles have been
resampled onto one shared aligned grid and cropped, use the viewer's
Fusion -> Alignment Wizard (sras_align_export.py) instead.
Usage:
python sras_edit_scans.py input.sras --list
python sras_edit_scans.py input.sras output.sras --drop 2,5
python sras_edit_scans.py input.sras output.sras --keep 0,1,3,4,6
"""
import argparse
import struct
import sys
from pathlib import Path
from sras_format import GEO_FMT_V6, HDR_FMT, HDR_FMT_V6, HDR_SIZE, SrasFile
_LEGACY_VERSIONS = (2, 3, 4, 5)
_V6_VERSIONS = (6, 7)
def _die(msg: str):
print(f"Error: {msg}", file=sys.stderr)
sys.exit(1)
def parse_args():
p = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
p.add_argument("input", help="Input .sras file")
p.add_argument("output", nargs="?", help="Output .sras file (omit with --list)")
p.add_argument("--list", action="store_true",
help="Print each angle's index/degrees/geometry and exit")
g = p.add_mutually_exclusive_group()
g.add_argument("--drop", metavar="I,J,...",
help="Comma-separated angle indices to remove")
g.add_argument("--keep", metavar="I,J,...",
help="Comma-separated angle indices to keep (all others dropped)")
return p.parse_args()
def _parse_index_list(s: str, n_angles: int) -> set[int]:
out = set()
for piece in s.split(","):
piece = piece.strip()
if not piece:
continue
i = int(piece)
if not (0 <= i < n_angles):
raise ValueError(f"angle index {i} out of range [0, {n_angles - 1}]")
out.add(i)
return out
def print_listing(sras: SrasFile):
print(f"\n{'idx':>4} {'angle_deg':>10} {'x_start_mm':>11} {'rows':>6} {'frames':>7}")
for a in range(sras.n_angles):
print(f"{a:>4} {sras.angles_deg[a]:>10.4f} {sras.x_start_mm[a]:>11.4f} "
f"{int(sras.n_rows[a]):>6} {int(sras.n_frames[a]):>7}")
def _copy_range(fin, fout, offset: int, nbytes: int, chunk: int = 64 * 1024 * 1024):
"""Stream *nbytes* raw bytes from *fin* at *offset* into *fout*, without
ever holding more than one chunk in memory (waveform blocks can be
hundreds of MB to low GB each)."""
fin.seek(offset)
remaining = nbytes
while remaining:
buf = fin.read(min(chunk, remaining))
if not buf:
raise IOError("unexpected EOF while copying waveform data")
fout.write(buf)
remaining -= len(buf)
# ---------------------------------------------------------------------------
# Legacy (v2-v5): uniform geometry across angles, one flat waveform block
# ---------------------------------------------------------------------------
def _write_legacy(sras: SrasFile, keep: list[int], out_path: Path):
n_rows = int(sras.n_rows[0])
n_frames = int(sras.n_frames[0]) # uniform across angles for v2-v5
n_ch = sras.n_channels
spf = sras.samples_per_frame
bps = sras.bytes_per_sample
header = struct.pack(
HDR_FMT, b"SRAS", sras.version, len(keep), n_rows,
float(sras.x_start_mm[0]), float(sras.x_delta_mm),
sras.velocity_mm_s, sras.laser_freq_hz,
n_frames, spf, sras.sample_rate_hz, bps, n_ch,
)
# Row table + preambles + background sit right after the angle table and
# don't vary per angle — copy that whole span through unmodified.
angle_table_size = sras.n_angles * 4
with open(sras.path, "rb") as f:
f.seek(HDR_SIZE + angle_table_size)
shared_mid = f.read(sras.data_offset - (HDR_SIZE + angle_table_size))
angle_bytes = n_rows * n_ch * n_frames * spf * bps
with open(sras.path, "rb") as fin, open(out_path, "wb") as fout:
fout.write(header)
fout.write(sras.angles_deg[keep].astype(">f4").tobytes())
fout.write(shared_mid)
for a in keep:
_copy_range(fin, fout, sras.data_offset + a * angle_bytes, angle_bytes)
# ---------------------------------------------------------------------------
# v6/v7: per-angle geometry, ragged waveform blocks
# ---------------------------------------------------------------------------
def _write_v6(sras: SrasFile, keep: list[int], out_path: Path):
header = struct.pack(
HDR_FMT_V6, b"SRAS", sras.version, len(keep),
sras.x_start_nominal_mm, sras.y_start_nominal_mm,
sras.x_delta_nominal_mm, sras.y_delta_nominal_mm,
sras.row_spacing_mm, sras.velocity_mm_s, sras.laser_freq_hz,
sras.samples_per_frame, sras.sample_rate_hz,
sras.bytes_per_sample, sras.n_channels,
)
blocks = {a: (offset, nbytes) for a, offset, nbytes in sras.iter_angle_blocks()}
with open(sras.path, "rb") as fin, open(out_path, "wb") as fout:
fout.write(header)
fout.write(sras.angles_deg[keep].astype(">f4").tobytes())
for i in keep:
fout.write(struct.pack(
GEO_FMT_V6, float(sras.x_start_mm[i]),
float(sras.x_delta_mm_per_angle[i]),
int(sras.n_frames[i]), int(sras.n_rows[i])))
for i in keep:
fout.write(sras.y_pos_per_angle[i].astype(">f4").tobytes())
fout.write(sras.preambles_raw)
fout.write(sras.background_raw)
for i in keep:
offset, nbytes = blocks[i]
_copy_range(fin, fout, offset, nbytes)
def main():
args = parse_args()
in_path = Path(args.input)
if not in_path.exists():
_die(f"input file not found: {in_path}")
print(f"Reading {in_path} ...", flush=True)
try:
sras = SrasFile(str(in_path))
except ValueError as e:
_die(str(e))
if sras.version not in (*_LEGACY_VERSIONS, *_V6_VERSIONS):
_die(f"unsupported .sras version: {sras.version}")
aborted_note = " (scan aborted; trailing angle(s) already excluded)" if sras.scan_aborted else ""
print(f" Version : v{sras.version}", flush=True)
print(f" Angles : {sras.n_angles}{aborted_note}", flush=True)
if args.list:
print_listing(sras)
return
if not args.output:
_die("output path required unless --list is given.")
if not (args.drop or args.keep):
_die("specify --drop or --keep (see --list for indices).")
out_path = Path(args.output)
if out_path.resolve() == in_path.resolve():
_die("output path must differ from input path.")
try:
if args.drop:
drop = _parse_index_list(args.drop, sras.n_angles)
keep = [a for a in range(sras.n_angles) if a not in drop]
else:
keep = sorted(_parse_index_list(args.keep, sras.n_angles))
except ValueError as e:
_die(str(e))
if not keep:
_die("at least one angle must remain.")
dropped = [a for a in range(sras.n_angles) if a not in keep]
print(f"\nDropping angle(s): {dropped}")
print(f"Keeping angle(s) : {keep} ({len(keep)} of {sras.n_angles})")
print(f"\nWriting {out_path} ...", flush=True)
if sras.version in _LEGACY_VERSIONS:
_write_legacy(sras, keep, out_path)
else:
_write_v6(sras, keep, out_path)
in_mb = in_path.stat().st_size / 1024**2
out_mb = out_path.stat().st_size / 1024**2
print(f" Input size : {in_mb:.1f} MB")
print(f" Output size: {out_mb:.1f} MB")
print("Done.")
print("Note: any precomputed FFT/DC cache was dropped (it's indexed by "
"angle); the viewer will recompute it next time this file opens.")
if __name__ == "__main__":
main()
+866
View File
@@ -0,0 +1,866 @@
#!/usr/bin/env python3
"""SRAS binary scan file format — parsing and writing.
Reads v2–v7 .sras files. Depends only on numpy + struct, so compute workers
(including multiprocessing children) can import it without pulling in Qt or
matplotlib. See scan_format.md for the full v6/v7 spec.
Channel semantics (fixed by sc3_aui_app.py acquisition settings):
CH1 — RF Acoustic Packet (AC-coupled, 100 mV/div): FFT → peak frequency
CH3 — Bias A (DC-coupled, 50 mV/div): waveform mean
CH4 — Bias B (DC-coupled, 50 mV/div): waveform mean
Frame-count correction: the scanner writes the *configured* frame count in the
header before acquisition, but the scope may acquire fewer frames. The actual
count is computed from the file size and used for the reshape so channels are
correctly aligned.
Scan geometry: v6/v7 files scan a different bounding box per angle (x_start,
x_delta, n_frames, n_rows all vary by angle), so geometry is exposed per-angle
via SrasFile.n_rows / n_frames / x_start_mm arrays and the x_axis_mm() /
y_positions_mm() methods. v2–v5 files have uniform geometry across angles, so
those arrays simply repeat the same value n_angles times.
v7 files are v6 files with an optional trailing cache section holding
precomputed per-angle DC and/or FFT images, so display never has to recompute
them after the "Convert" menu's batch actions have stored them once.
"""
import os
import re
import struct
from pathlib import Path
import numpy as np
# ---------------------------------------------------------------------------
# Format constants
# ---------------------------------------------------------------------------
# v2–v5: fixed header, uniform geometry across angles (43 bytes)
HDR_FMT = ">4sBHHffffIIdBB"
HDR_SIZE = struct.calcsize(HDR_FMT)
# v6/v7: fixed header, per-angle geometry in a separate table (49 bytes)
HDR_FMT_V6 = ">4sBHfffffffIdBB"
HDR_SIZE_V6 = struct.calcsize(HDR_FMT_V6)
# v6/v7: per-angle geometry record (x_start, x_delta, n_frames, n_rows)
GEO_FMT_V6 = ">ffIH"
GEO_SIZE_V6 = struct.calcsize(GEO_FMT_V6)
# v5 precomputed-image tail
PREC_MAGIC = b"PREC"
PREC_FLAG_BG_SUB = 0x01
# v7 cache tail. v7 is byte-identical to v6 (the version byte is the only
# header difference) plus this optional trailing section. Sub-block sizes are
# per-angle (n_rows[a] * n_frames[a]), taken from the Per-Angle Geometry Table
# already parsed for v6 — no new geometry fields are needed.
CACH_MAGIC = b"CACH"
CACH_HDR_FMT = ">4sBB" # magic, cach_version, block_flags
CACH_HDR_SIZE = struct.calcsize(CACH_HDR_FMT)
CACH_VERSION = 4 # written on every fresh write
CACH_VERSIONS_READABLE = (1, 2, 3, 4) # accepted on read — see
# _read_sfft_block. Each bump only
# appended a field, and every older
# tail has a well-defined reading:
# v1 predates row-averaged FFT
# caching (row_avg_n=0), v1/v2
# predate padded caching, so both
# are natural-resolution (pad 1),
# and v1-v3 predate the min peak
# frequency floor (min_freq 0 =
# no floor).
CACH_FLAG_DC = 0x01
CACH_FLAG_FFT = 0x02
SDCB_MAGIC = b"SDCB"
SDCB_HDR_FMT = ">4sBH" # magic, reserved, n_stored
SDCB_HDR_SIZE = struct.calcsize(SDCB_HDR_FMT)
SFFT_MAGIC = b"SFFT"
SFFT_HDR_FMT_V1 = ">4sBH" # magic, flags, n_stored (cach_version 1)
SFFT_HDR_FMT_V2 = ">4sBHB" # + row_avg_n (cach_version 2)
SFFT_HDR_FMT_V3 = ">4sBHBH" # + pad_factor (cach_version 3)
SFFT_HDR_FMT = ">4sBHBHI" # + min_freq_khz (cach_version 4)
SFFT_HDR_SIZE_V1 = struct.calcsize(SFFT_HDR_FMT_V1)
SFFT_HDR_SIZE_V2 = struct.calcsize(SFFT_HDR_FMT_V2)
SFFT_HDR_SIZE_V3 = struct.calcsize(SFFT_HDR_FMT_V3)
SFFT_HDR_SIZE = struct.calcsize(SFFT_HDR_FMT)
MAX_PAD_FACTOR = 0xFFFF # the H field above
# min_freq_khz is fixed-point (u32, units of 0.001 MHz), not a float32:
# the viewer's floor spinbox has 0.001 MHz granularity, and
# round(mhz * 1000) / 1000.0 reproduces the exact float64 the user typed,
# so the accept rule in sras_compute.cache_mismatch_reasons can compare
# with plain integer ordering. A ">f" float32 of e.g. 20.1 would read back
# as 20.10000038… > 20.1 and report a spurious mismatch forever.
MAX_MIN_FREQ_KHZ = 0xFFFFFFFF # the I field above; 0 = no floor
SFFT_FLAG_BG_SUB = 0x01
SFFT_FLAG_ROW_AVG = 0x02 # peak_freq_mhz came from same-row,
# distance-weighted averaged CH1
# waveforms, not raw per-pixel ones;
# row_avg_n is the neighbor half-width
# (pixels) used. Bits 2-7 reserved.
# Fixed channel indices into the .sras data array (CH1=RF, CH3/CH4=Bias DC)
CH1_IDX, CH3_IDX, CH4_IDX = 0, 1, 2
CH_NAMES = ["CH1", "CH3", "CH4", "VEL"]
# Fallback scope calibration used only when reading v2 files without embedded
# preambles. v3+ files carry the WFMOutpre string so these are not used.
# 50 mV/div, 8 div full-scale, int8 ADC, position = -2.72 div
# ymult = 50 mV × 8 / 256 = 1.5625 mV/count
# yoff = position × (256/8) = -2.72 × 32 = -87.04 (ADC count for 0 V)
_FALLBACK_YMULT_MV = 1.5625 # mV per ADC count
_FALLBACK_YOFF_ADC = -87.04 # ADC count that represents 0 V
# ---------------------------------------------------------------------------
# Calibration
# ---------------------------------------------------------------------------
def _parse_preamble(preamble: str) -> dict[str, float]:
"""Extract YMULT, YOFF, YZERO from a Tektronix WFMOutpre string.
Returns a dict with float values for whichever keys are present.
YMULT is left in V/count as the scope reports it.
"""
result = {}
for key in ("YMULT", "YOFF", "YZERO"):
m = re.search(rf'\b{key}\s+([-+]?\d*\.?\d+(?:[Ee][+-]?\d+)?)', preamble)
if m:
result[key] = float(m.group(1))
return result
def mv_to_adc(mv: float, ymult_mv: float = _FALLBACK_YMULT_MV,
yoff_adc: float = _FALLBACK_YOFF_ADC,
yzero_mv: float = 0.0) -> float:
return (mv - yzero_mv) / ymult_mv + yoff_adc
def adc_to_mv(adc, ymult_mv: float = _FALLBACK_YMULT_MV,
yoff_adc: float = _FALLBACK_YOFF_ADC,
yzero_mv: float = 0.0):
return (adc - yoff_adc) * ymult_mv + yzero_mv
def _axes_extent(x_axis, y_axis, dx: float, dy: float) -> list[float]:
"""Matplotlib imshow extent with half-pixel margins, Y flipped so row 0
renders at the top."""
return [x_axis[0] - dx / 2, x_axis[-1] + dx / 2,
y_axis[-1] + dy / 2, y_axis[0] - dy / 2]
# ---------------------------------------------------------------------------
# Binary read helpers
# ---------------------------------------------------------------------------
def _read_struct(f, fmt: str) -> tuple:
return struct.unpack(fmt, f.read(struct.calcsize(fmt)))
def _read_preambles(f, n_ch: int) -> list[str]:
"""n_ch length-prefixed UTF-8 WFMOutpre strings."""
out = []
for _ in range(n_ch):
(length,) = _read_struct(f, ">H")
out.append(f.read(length).decode("utf-8"))
return out
def _read_background(f) -> np.ndarray:
"""uint32 sample count followed by that many int8 samples."""
(n_bg,) = _read_struct(f, ">I")
return np.frombuffer(f.read(n_bg), dtype=np.int8).astype(np.float32)
def _read_f32_image(f, shape: tuple[int, int]) -> np.ndarray:
"""One big-endian float32 image, converted to native float32.
The conversion matters: np.frombuffer hands back a read-only big-endian
view, and these arrays flow straight into the display caches and every
downstream arithmetic op.
"""
n_bytes = shape[0] * shape[1] * 4
return np.frombuffer(f.read(n_bytes), dtype=">f4").reshape(shape).astype(np.float32)
# ---------------------------------------------------------------------------
# File parser
# ---------------------------------------------------------------------------
class SrasFile:
"""Parsed in-memory representation of a v2–v7 .sras file.
Scan geometry (rows, frames, x_start) is exposed per-angle via the
``n_rows`` / ``n_frames`` / ``x_start_mm`` arrays and the ``x_axis_mm()``
/ ``y_positions_mm()`` methods, since v6/v7 files scan a different
bounding box per angle. v2–v5 files have uniform geometry, so these
arrays just repeat the same value ``n_angles`` times. Waveform data is
likewise exposed as ``data[angle_idx]``, an array of shape
``(n_rows[a], n_channels, n_frames[a], samples_per_frame)``.
Precomputed images (v5's PREC tail or v7's CACH tail) are exposed as
``precomputed_dc3_mv`` / ``precomputed_dc4_mv`` / ``precomputed_freq_mhz``,
always as ragged per-angle lists (``list[np.ndarray | None]``, one entry
per angle, ``None`` where that angle was never stored) regardless of
source version. The scalars ``precomputed_bg_sub`` /
``precomputed_row_avg_n`` / ``precomputed_pad_factor`` /
``precomputed_min_freq_mhz`` record the settings the stored FFT images
were computed under, so a reader can tell whether they answer the
question it is actually asking.
"""
def __init__(self, path: str):
self.path = Path(path)
self._parse()
def _parse(self):
with open(self.path, "rb") as f:
magic = f.read(4)
if magic != b"SRAS":
raise ValueError(f"Bad magic bytes: {magic!r}")
(version,) = struct.unpack(">B", f.read(1))
self.version = version
if version in (2, 3, 4, 5):
self._parse_legacy()
elif version in (6, 7):
self._parse_v6()
else:
raise ValueError(f"Unsupported version: {version}")
# ------------------------------------------------------------------
# Calibration
# ------------------------------------------------------------------
def _set_calibration(self, preambles: list[str] | None, n_ch: int):
"""Populate the per-channel ymult/yoff/yzero lists from preamble
strings, falling back to the hardcoded scope constants for v2 files
that carry no preambles."""
if preambles is None:
self.preambles = None
self.ch_ymult_mv = [_FALLBACK_YMULT_MV] * n_ch
self.ch_yoff_adc = [_FALLBACK_YOFF_ADC] * n_ch
self.ch_yzero_mv = [0.0] * n_ch
return
self.preambles = preambles
self.ch_ymult_mv, self.ch_yoff_adc, self.ch_yzero_mv = [], [], []
for p in preambles:
cal = _parse_preamble(p)
# The scope reports YMULT and YZERO in volts; store both as mV.
self.ch_ymult_mv.append(cal.get("YMULT", _FALLBACK_YMULT_MV / 1000) * 1000)
self.ch_yoff_adc.append(cal.get("YOFF", _FALLBACK_YOFF_ADC))
self.ch_yzero_mv.append(cal.get("YZERO", 0.0) * 1000)
def cal(self, ch_idx: int) -> tuple[float, float, float]:
"""(ymult_mv, yoff_adc, yzero_mv) for one channel — splat straight
into adc_to_mv / mv_to_adc."""
return (self.ch_ymult_mv[ch_idx], self.ch_yoff_adc[ch_idx],
self.ch_yzero_mv[ch_idx])
def _init_precomputed(self, n_angles: int):
self.precomputed_freq_mhz: list[np.ndarray | None] = [None] * n_angles
self.precomputed_dc4_mv: list[np.ndarray | None] = [None] * n_angles
self.precomputed_dc3_mv: list[np.ndarray | None] = [None] * n_angles
self.precomputed_bg_sub: bool = False
self.precomputed_row_avg_n: int = 0
# Zero-padding factor the stored peak_freq_mhz images were resolved
# at: 1 = natural resolution (n_fft == samples_per_frame). A padded
# FFT resolves peaks a padded view would, and only such a view can
# be served from it — see sras_compute.cached_rf_image.
self.precomputed_pad_factor: int = 1
# Min peak frequency floor (MHz) the stored peak search excluded
# bins below; 0.0 = no floor. Unlike bg-sub/pad/row-avg it is
# tighten-only re-applicable: a *higher* floor can be re-imposed on
# a stored image by masking pixels below it, but bins below the
# stored floor were never searched, so a lower floor needs a real
# recompute — see sras_compute.cache_mismatch_reasons.
self.precomputed_min_freq_mhz: float = 0.0
def encoded_preambles(self) -> bytes:
"""This file's Preamble Blocks section, as bytes a writer can emit.
v6/v7 files kept the on-disk span verbatim, which is both cheaper and
lossless; legacy files did not keep it, and a v2 file has no preambles
at all, so those are re-encoded from the parsed strings (empty ones for
v2). Empty is not a silent downgrade: _parse_preamble("") returns {} and
_set_calibration falls back to the hardcoded scope constants, which is
exactly the calibration a v2 file already gets, so mV values round-trip
unchanged.
Lives here rather than at each writer so the version fan-out sits next
to the parser that creates it, and no writer has to probe the object
to find out which shape it got.
"""
raw = getattr(self, "preambles_raw", None)
if raw is not None:
return raw
out = bytearray()
for s in self.preambles or [""] * self.n_channels:
encoded = s.encode("utf-8")
out += struct.pack(">H", len(encoded)) + encoded
return bytes(out)
def encoded_background(self) -> bytes:
"""This file's Background Block, as bytes a writer can emit.
When there is none (v2/v3), this is samples_per_frame zeros rather than
a zero-length block. Every consumer guards on `background is not None`
and then subtracts it from a (spf,)-shaped row, so a length-0 array
would broadcast-fail at the first background-subtracted FFT; zeros make
the subtraction a correct no-op instead.
"""
raw = getattr(self, "background_raw", None)
if raw is not None:
return raw
if self.background is None:
samples = np.zeros(self.samples_per_frame, dtype=np.int8)
else:
samples = np.rint(self.background).astype(np.int8)
return struct.pack(">I", samples.size) + samples.tobytes()
def cached_dc_mv(self, angle_idx: int, ch_idx: int) -> np.ndarray | None:
"""A stored DC image (already in mV) for (angle, channel), or None."""
store = self.precomputed_dc3_mv if ch_idx == CH3_IDX else self.precomputed_dc4_mv
return store[angle_idx] if angle_idx < len(store) else None
def image_shape(self, angle_idx: int) -> tuple[int, int]:
return int(self.n_rows[angle_idx]), int(self.n_frames[angle_idx])
# ------------------------------------------------------------------
# v2–v5 parsing (uniform geometry, flat waveform block)
# ------------------------------------------------------------------
def _parse_legacy(self):
with open(self.path, "rb") as f:
(magic, ver, n_angles, n_rows, x_start, x_delta, vel, freq,
n_frames_hdr, spf, sr, bps, n_ch) = _read_struct(f, HDR_FMT)
self.n_angles = n_angles
self.velocity_mm_s = float(vel)
self.laser_freq_hz = float(freq)
self.n_frames_header = n_frames_hdr # configured count (may be wrong)
self.samples_per_frame = spf
self.sample_rate_hz = float(sr)
self.bytes_per_sample = bps
self.n_channels = n_ch
self._init_precomputed(n_angles)
self.scan_aborted = False
self.n_angles_declared = n_angles
# Pre-v6 files carry no nominal ROI. Defined as None rather than
# left absent so the object's shape does not depend on its version
# and writers can ask instead of probing with hasattr.
self.x_start_nominal_mm = None
self.y_start_nominal_mm = None
self.x_delta_nominal_mm = None
self.y_delta_nominal_mm = None
self.row_spacing_mm = None
angles = np.frombuffer(f.read(n_angles * 4), dtype=">f4").astype(np.float32)
y_pos = np.frombuffer(f.read(n_rows * 4), dtype=">f4").astype(np.float32)
self._set_calibration(_read_preambles(f, n_ch) if ver >= 3 else None, n_ch)
self.background = _read_background(f) if ver >= 4 else None
# Record where raw waveform data begins; np.memmap maps from here.
data_offset = f.tell()
# ---- Determine actual frame count from file size ---------------
# For v4 and earlier the header n_frames may be the *configured*
# count before acquisition; the actual count is derived from the
# bytes on disk. For v5 files a PREC tail follows the waveform
# data, so we must not include those extra bytes in the frame count.
file_size = self.path.stat().st_size
samples_per_row_per_ch = n_ch * spf
available_bytes = file_size - data_offset
if ver == 5:
actual_n_frames = n_frames_hdr
remainder = 0
else:
total_samples = available_bytes // bps
per_frame = n_angles * n_rows * samples_per_row_per_ch
actual_n_frames = total_samples // per_frame
remainder = total_samples % per_frame
self.frame_count_mismatch = (actual_n_frames != n_frames_hdr)
self.n_frames_remainder = remainder
# ---- Memory-map the waveform data (zero RAM cost) --------------
# Instead of f.read() → astype() (which peaks at 2× file size),
# memmap lets the OS page only the bytes that are actually touched.
data5d = np.memmap(
str(self.path),
dtype=np.int8 if bps == 1 else ">i2",
mode="r",
offset=data_offset,
shape=(n_angles, n_rows, n_ch, actual_n_frames, spf),
)
# Expose as a list of per-angle views so downstream code shares one
# indexing convention with v6: sras.data[a][row, ch, frame, sample]
self.data = [data5d[a] for a in range(n_angles)]
# Uniform per-angle geometry, repeated so callers don't need to
# special-case legacy vs. v6 files.
self.n_rows = np.full(n_angles, n_rows, dtype=np.int64)
self.n_frames = np.full(n_angles, actual_n_frames, dtype=np.int64)
self.x_start_mm = np.full(n_angles, float(x_start), dtype=np.float64)
self.x_delta_mm = float(x_delta) # reference only; kept for re-encode
self._y_pos_per_angle = [y_pos] * n_angles
self.angles_deg = angles
self._data_offset = data_offset
if ver >= 5:
waveform_bytes = actual_n_frames * n_angles * n_rows * n_ch * spf * bps
prec_offset = data_offset + waveform_bytes
if file_size > prec_offset:
self._parse_prec_section(prec_offset)
def _parse_prec_section(self, offset: int):
"""Parse the v5 PREC tail that holds precomputed images.
Stored per-angle as (freq, dc4, dc3), each a full-image float32
block prefixed by its uint16 angle index.
"""
with open(self.path, "rb") as f:
f.seek(offset)
header_raw = f.read(6) # magic(4) + fmt_ver(1) + flags(1)
if len(header_raw) < 6 or header_raw[:4] != PREC_MAGIC:
return
self.precomputed_bg_sub = bool(header_raw[5] & PREC_FLAG_BG_SUB)
(n_stored,) = _read_struct(f, ">H")
for _ in range(n_stored):
(aidx,) = _read_struct(f, ">H")
if aidx >= self.n_angles:
break
shape = self.image_shape(aidx)
self.precomputed_freq_mhz[aidx] = _read_f32_image(f, shape)
self.precomputed_dc4_mv[aidx] = _read_f32_image(f, shape)
self.precomputed_dc3_mv[aidx] = _read_f32_image(f, shape)
# ------------------------------------------------------------------
# v6/v7 parsing (per-angle geometry, ragged waveform blocks)
# ------------------------------------------------------------------
def _parse_v6(self):
with open(self.path, "rb") as f:
(magic, ver, n_angles, x_start_nom, y_start_nom, x_delta_nom,
y_delta_nom, row_spacing, vel, freq, spf, sr, bps,
n_ch) = _read_struct(f, HDR_FMT_V6)
n_angles_declared = n_angles
self.velocity_mm_s = float(vel)
self.laser_freq_hz = float(freq)
self.samples_per_frame = spf
self.sample_rate_hz = float(sr)
self.bytes_per_sample = bps
self.n_channels = n_ch
# Reference-only fields: the ROI as entered before per-angle
# bounding-box expansion. Actual per-angle geometry used for
# rendering comes from the Per-Angle Geometry Table below.
self.x_start_nominal_mm = float(x_start_nom)
self.y_start_nominal_mm = float(y_start_nom)
self.x_delta_nominal_mm = float(x_delta_nom)
self.y_delta_nominal_mm = float(y_delta_nom)
self.row_spacing_mm = float(row_spacing)
self.n_frames_header = None
self.frame_count_mismatch = False
self.n_frames_remainder = 0
angles = np.frombuffer(f.read(n_angles * 4), dtype=">f4").astype(np.float32)
x_start = np.empty(n_angles, dtype=np.float64)
x_delta = np.empty(n_angles, dtype=np.float64)
n_frames = np.empty(n_angles, dtype=np.int64)
n_rows = np.empty(n_angles, dtype=np.int64)
for a in range(n_angles):
xs, xd, nf, nr = _read_struct(f, GEO_FMT_V6)
x_start[a], x_delta[a], n_frames[a], n_rows[a] = xs, xd, nf, nr
y_pos_per_angle = [
np.frombuffer(f.read(int(n_rows[a]) * 4), dtype=">f4").astype(np.float32)
for a in range(n_angles)
]
# Verbatim on-disk spans of the preamble and background sections,
# kept so file-rewriting tools (sras_edit_scans) can carry them
# over byte-for-byte without re-parsing.
span_start = f.tell()
self._set_calibration(_read_preambles(f, n_ch), n_ch)
span_end = f.tell()
f.seek(span_start)
self.preambles_raw = f.read(span_end - span_start)
span_start = span_end
self.background = _read_background(f)
span_end = f.tell()
f.seek(span_start)
self.background_raw = f.read(span_end - span_start)
data_offset = span_end
self._data_offset = data_offset
# ---- Memory-map each angle's ragged waveform block -------------
# v6 gives each angle its own row/frame count, so waveform data is
# no longer one uniform (n_angles, n_rows, ...) block — each angle's
# block sits at a different offset with its own shape. An aborted
# scan truncates the file mid-angle; per the format spec we keep
# whatever complete angles are present rather than refusing to open
# the file.
file_size = self.path.stat().st_size
waveform_dtype = np.int8 if bps == 1 else ">i2"
data = []
offset = data_offset
for a in range(n_angles):
nr, nf = int(n_rows[a]), int(n_frames[a])
nbytes = nr * n_ch * nf * spf * bps
if offset + nbytes > file_size:
break
data.append(np.memmap(
str(self.path), dtype=waveform_dtype, mode="r",
offset=offset, shape=(nr, n_ch, nf, spf),
))
offset += nbytes
n_complete = len(data)
if n_complete == 0:
raise ValueError(
"v6 file has no complete angle blocks — scan was aborted "
"before the first angle finished.")
self.data = data
self.n_angles = n_complete
self.n_angles_declared = n_angles_declared
self.scan_aborted = n_complete < n_angles_declared
self.angles_deg = angles[:n_complete]
self.x_start_mm = x_start[:n_complete]
self.x_delta_mm_per_angle = x_delta[:n_complete]
self.n_frames = n_frames[:n_complete]
self.n_rows = n_rows[:n_complete]
self._y_pos_per_angle = y_pos_per_angle[:n_complete]
self._init_precomputed(n_complete)
if self.version == 7 and offset < file_size:
self._parse_cach_section(offset)
# ------------------------------------------------------------------
# Byte-layout accessors (public: used by file-rewriting tools)
# ------------------------------------------------------------------
@property
def data_offset(self) -> int:
"""File offset where the waveform data begins (headers end)."""
return self._data_offset
@property
def y_pos_per_angle(self) -> list[np.ndarray]:
"""Per-angle Y row positions (mm). The list and its arrays are the
live parsed state — tools that reproject may replace entries."""
return self._y_pos_per_angle
@y_pos_per_angle.setter
def y_pos_per_angle(self, value: list[np.ndarray]):
self._y_pos_per_angle = value
def iter_angle_blocks(self):
"""Yields (angle_idx, byte_offset, byte_count) for each complete
angle's waveform block. Works for every version: legacy files have
uniform per-angle geometry, so the same walk applies."""
offset = self._data_offset
for a in range(self.n_angles):
nbytes = (int(self.n_rows[a]) * self.n_channels
* int(self.n_frames[a]) * self.samples_per_frame
* self.bytes_per_sample)
yield a, offset, nbytes
offset += nbytes
# ------------------------------------------------------------------
# v7 cache tail (CACH section: precomputed DC / FFT images)
# ------------------------------------------------------------------
def _cache_tail_offset(self) -> int:
"""Deterministic file offset where the CACH tail starts (or would
start), derived purely from the header + Per-Angle Geometry Table —
independent of whether a cache tail is actually present. Used by
both the parser and the in-place writer."""
end = self._data_offset
for _, offset, nbytes in self.iter_angle_blocks():
end = offset + nbytes
return end
def _read_cache_block(self, f, hdr_fmt: str, magic: bytes,
stores: list[list]) -> int | None:
"""Read one CACH sub-block header, then its per-angle image entries
into *stores* (one list per image the block stores per angle).
Returns the header's flags byte, or None if the block is malformed.
"""
raw = f.read(struct.calcsize(hdr_fmt))
if len(raw) < struct.calcsize(hdr_fmt):
return None
block_magic, flags, n_stored = struct.unpack(hdr_fmt, raw)
if block_magic != magic:
return None
for _ in range(n_stored):
(angle_idx,) = _read_struct(f, ">H")
if angle_idx >= self.n_angles:
break
shape = self.image_shape(angle_idx)
for store in stores:
store[angle_idx] = _read_f32_image(f, shape)
return flags
def _read_sfft_block(self, f, cach_version: int) -> tuple[int, int, int, float] | None:
"""Read the SFFT block header — its layout depends on cach_version,
since each bump appended a trailing field (v2 row_avg_n, v3
pad_factor, v4 min_freq_khz) — then n_stored per-angle
peak_freq_mhz entries (unchanged across versions).
Returns (flags, row_avg_n, pad_factor, min_freq_mhz), or None if
the block is malformed. The absent fields of an older tail take the
value that describes what such a tail can only have been:
row_avg_n=0 for v1, which predates row-averaged FFT caching;
pad_factor=1 for v1/v2, which predate padded caching and so are
natural-resolution; and min_freq_mhz=0.0 for v1-v3, which predate
the min peak frequency floor and so searched every bin above DC.
"""
hdr_fmt = {1: SFFT_HDR_FMT_V1, 2: SFFT_HDR_FMT_V2,
3: SFFT_HDR_FMT_V3}.get(cach_version, SFFT_HDR_FMT)
raw = f.read(struct.calcsize(hdr_fmt))
if len(raw) < struct.calcsize(hdr_fmt):
return None
row_avg_n, pad_factor, min_freq_khz = 0, 1, 0
if cach_version == 1:
magic, flags, n_stored = struct.unpack(hdr_fmt, raw)
elif cach_version == 2:
magic, flags, n_stored, row_avg_n = struct.unpack(hdr_fmt, raw)
elif cach_version == 3:
magic, flags, n_stored, row_avg_n, pad_factor = struct.unpack(hdr_fmt, raw)
else:
(magic, flags, n_stored, row_avg_n, pad_factor,
min_freq_khz) = struct.unpack(hdr_fmt, raw)
if magic != SFFT_MAGIC:
return None
for _ in range(n_stored):
(angle_idx,) = _read_struct(f, ">H")
if angle_idx >= self.n_angles:
break
self.precomputed_freq_mhz[angle_idx] = _read_f32_image(
f, self.image_shape(angle_idx))
return flags, row_avg_n, pad_factor, min_freq_khz / 1000.0
def _parse_cach_section(self, offset: int):
"""Parse the v7 CACH tail that holds precomputed DC/FFT images."""
with open(self.path, "rb") as f:
f.seek(offset)
header_raw = f.read(CACH_HDR_SIZE)
if len(header_raw) < CACH_HDR_SIZE:
return
magic, cach_version, block_flags = struct.unpack(CACH_HDR_FMT, header_raw)
if magic != CACH_MAGIC or cach_version not in CACH_VERSIONS_READABLE:
return
if block_flags & CACH_FLAG_DC:
if self._read_cache_block(
f, SDCB_HDR_FMT, SDCB_MAGIC,
[self.precomputed_dc3_mv, self.precomputed_dc4_mv]) is None:
return
if block_flags & CACH_FLAG_FFT:
result = self._read_sfft_block(f, cach_version)
if result is None:
return
flags, row_avg_n, pad_factor, min_freq_mhz = result
self.precomputed_bg_sub = bool(flags & SFFT_FLAG_BG_SUB)
self.precomputed_row_avg_n = row_avg_n if (flags & SFFT_FLAG_ROW_AVG) else 0
self.precomputed_pad_factor = max(1, pad_factor)
# No flag bit gates the floor: 0 (= no floor) is already the
# value every pre-v4 tail reads as.
self.precomputed_min_freq_mhz = min_freq_mhz
def write_v7_cache(self, *,
new_dc3_mv: list[np.ndarray | None] | None = None,
new_dc4_mv: list[np.ndarray | None] | None = None,
new_freq_mhz: list[np.ndarray | None] | None = None,
new_bg_sub: bool | None = None,
new_row_avg_n: int | None = None,
new_pad_factor: int | None = None,
new_min_freq_mhz: float | None = None):
"""Store computed DC and/or FFT images into this file's CACH tail,
in place, converting a v6 source to v7 (or updating an existing v7
file). Only the block(s) passed in are recomputed; whichever block
isn't passed is carried forward unchanged from whatever this
``SrasFile`` already has in memory (from parsing, or a prior write
in this same session) — its bytes are never re-read from disk.
*new_row_avg_n* is the same-row neighbor half-width (pixels) the
passed *new_freq_mhz* was averaged over before its FFT, 0 for a raw
(unaveraged) compute — carried forward like *new_bg_sub* when None.
It describes the whole stored FFT block, not per-angle, mirroring
how bg-sub has never been tracked per-angle either.
*new_pad_factor* is the zero-padding factor the passed *new_freq_mhz*
was resolved at (1 = natural resolution), carried forward the same
way. Like row_avg_n it is provenance, not a hint: a view at a
different pad resolves different peaks, so recording it is what lets
a reader refuse the cache instead of showing the wrong numbers.
*new_min_freq_mhz* is the min peak frequency floor the passed
*new_freq_mhz*'s peak search excluded bins below (0.0 = no floor),
carried forward the same way. It is stored fixed-point (whole kHz),
so the value is quantized to 0.001 MHz on write and
``precomputed_min_freq_mhz`` is updated to the quantized value —
what a reload would see, never a float the header can't represent.
The waveform data itself is never touched: the cache tail always
starts at ``_cache_tail_offset()``, a fixed offset derived from the
header and geometry table alone.
"""
if self.version not in (6, 7):
raise ValueError(
f"write_v7_cache only supports v6/v7 source files, got v{self.version}")
final_dc3 = new_dc3_mv if new_dc3_mv is not None else self.precomputed_dc3_mv
final_dc4 = new_dc4_mv if new_dc4_mv is not None else self.precomputed_dc4_mv
final_freq = new_freq_mhz if new_freq_mhz is not None else self.precomputed_freq_mhz
final_bg_sub = new_bg_sub if new_bg_sub is not None else self.precomputed_bg_sub
final_row_avg_n = (new_row_avg_n if new_row_avg_n is not None
else self.precomputed_row_avg_n)
final_pad_factor = (new_pad_factor if new_pad_factor is not None
else self.precomputed_pad_factor)
final_min_freq_mhz = (new_min_freq_mhz if new_min_freq_mhz is not None
else self.precomputed_min_freq_mhz)
if not (0 <= final_row_avg_n <= 255):
raise ValueError(f"row_avg_n must fit in a byte (0-255), got {final_row_avg_n}")
if not (1 <= final_pad_factor <= MAX_PAD_FACTOR):
raise ValueError(
f"pad_factor must be 1-{MAX_PAD_FACTOR}, got {final_pad_factor}")
if not (np.isfinite(final_min_freq_mhz) and final_min_freq_mhz >= 0):
raise ValueError(
f"min_freq_mhz must be a finite value >= 0, got {final_min_freq_mhz}")
final_min_freq_khz = int(round(final_min_freq_mhz * 1000))
if final_min_freq_khz > MAX_MIN_FREQ_KHZ:
raise ValueError(
f"min_freq_mhz too large for the u32 kHz header field: "
f"{final_min_freq_mhz}")
# dc3/dc4 are always populated together by every current caller, but
# guard the per-angle pairing explicitly rather than assume it: an
# angle present in only one of the two arrays would otherwise crash
# below on final_dc4[a].astype(...) (or silently store the wrong
# dc3/dc4 pairing for that angle).
dc_entries = [a for a in range(self.n_angles)
if final_dc3[a] is not None and final_dc4[a] is not None]
fft_entries = [a for a in range(self.n_angles) if final_freq[a] is not None]
block_flags = ((CACH_FLAG_DC if dc_entries else 0)
| (CACH_FLAG_FFT if fft_entries else 0))
payload = bytearray()
payload += struct.pack(CACH_HDR_FMT, CACH_MAGIC, CACH_VERSION, block_flags)
if dc_entries:
payload += struct.pack(SDCB_HDR_FMT, SDCB_MAGIC, 0, len(dc_entries))
for a in dc_entries:
payload += struct.pack(">H", a)
payload += final_dc3[a].astype(">f4").tobytes()
payload += final_dc4[a].astype(">f4").tobytes()
if fft_entries:
fft_flags = SFFT_FLAG_BG_SUB if final_bg_sub else 0
fft_flags |= SFFT_FLAG_ROW_AVG if final_row_avg_n else 0
payload += struct.pack(SFFT_HDR_FMT, SFFT_MAGIC, fft_flags,
len(fft_entries), final_row_avg_n,
final_pad_factor, final_min_freq_khz)
for a in fft_entries:
payload += struct.pack(">H", a)
payload += final_freq[a].astype(">f4").tobytes()
with open(self.path, "r+b") as f:
f.seek(self._cache_tail_offset())
f.write(payload)
f.truncate()
f.flush()
os.fsync(f.fileno())
# Version-byte flip last: when converting a v6 source (self.version
# was 6 on entry), if the process dies before this point the file
# is still readable as plain v6 (v6 parsing only bounds-checks
# per-angle offset+nbytes <= file_size, it never asserts exactly
# how many bytes follow the last angle) — so an interrupted write
# can never corrupt the file, only leave harmless trailing bytes
# that the next successful write overwrites via this same
# deterministic cache offset.
#
# That guarantee does NOT extend to updating an already-v7 file:
# the version byte here is already 7 before this call, so a crash
# during the payload write above (before flush/fsync/truncate)
# can leave a cache tail that mixes a prefix of the new payload
# with a stale suffix of the old one, and this method has no
# protection against that case (no atomic rename — the tail is
# rewritten in place to avoid copying the, potentially huge,
# waveform data that precedes it).
f.seek(4)
f.write(struct.pack("B", 7))
f.flush()
os.fsync(f.fileno())
self.version = 7
self.precomputed_dc3_mv = final_dc3
self.precomputed_dc4_mv = final_dc4
self.precomputed_freq_mhz = final_freq
self.precomputed_bg_sub = final_bg_sub
self.precomputed_row_avg_n = final_row_avg_n
self.precomputed_pad_factor = final_pad_factor
self.precomputed_min_freq_mhz = final_min_freq_khz / 1000.0
# ------------------------------------------------------------------
# Axes helpers
# ------------------------------------------------------------------
@property
def pixel_x_mm(self) -> float:
return self.velocity_mm_s / self.laser_freq_hz
def x_axis_mm(self, angle_idx: int) -> np.ndarray:
n = int(self.n_frames[angle_idx])
return self.x_start_mm[angle_idx] + np.arange(n) * self.pixel_x_mm
def y_positions_mm(self, angle_idx: int) -> np.ndarray:
return self._y_pos_per_angle[angle_idx]
def angles_share_raw_grid(self) -> bool:
"""True iff every angle's raw (x, y) pixel grid is literally the same
array as angle 0's -- the case for a .sras file the viewer's own
Alignment Wizard exported (see scan_format.md, "Files written by the
viewer's Alignment Wizard"): the writer packs one Per-Angle Geometry
record and one Row Table span and repeats those same bytes for every
angle, so re-parsed arrays are bit-identical copies rather than
independently re-derived numbers -- a bare np.array_equal is the
correct test here, no tolerance needed.
"""
if self.n_angles <= 1:
return True
x0 = self.x_axis_mm(0)
y0 = self.y_positions_mm(0)
return all(np.array_equal(self.x_axis_mm(a), x0)
and np.array_equal(self.y_positions_mm(a), y0)
for a in range(1, self.n_angles))
def time_axis_ns(self) -> np.ndarray:
return np.arange(self.samples_per_frame) / self.sample_rate_hz * 1e9
def freq_axis_mhz(self, n_fft: int | None = None) -> np.ndarray:
n = n_fft if n_fft is not None else self.samples_per_frame
return np.fft.rfftfreq(n, d=1.0 / self.sample_rate_hz) / 1e6
+204
View File
@@ -0,0 +1,204 @@
"""Pure-matplotlib rendering of a displayed SRAS image: imshow + colorbar +
axis/title labeling, shared by the interactive Qt canvas
(sras_viewer.canvases.ImageCanvas, which supplies its own already-Qt-backed
Figure/Axes) and the headless batch image-export worker (which builds a
throwaway Agg Figure per file and never touches Qt) -- so an exported PNG
can never quietly start looking different from what the GUI actually shows.
Deliberately no PyQt6 import anywhere in this module: BatchExportImagesWorker
(sras_workers.py) may run export_view_image inside a spawned
ProcessPoolExecutor subprocess, exactly like sras_compute.cache_file, and
importing anything under the sras_viewer package would run its __init__.py
and pull in the whole Qt widget tree for no reason.
"""
from pathlib import Path
import matplotlib as mpl
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure
from sras_compute import compute_rf_image, dc_image_mv
from sras_format import CH4_IDX, CH_NAMES, SrasFile, _axes_extent
# Figure size (inches) to fall back on when a caller has no live view to
# match: the size ImageCanvas is constructed at, so a headless render with
# no canvas behind it still looks like the viewer's starting layout.
DEFAULT_FIGSIZE = (7.0, 5.0)
# The canvas size travels from the GUI as-is, so it can be degenerate (a
# collapsed splitter pane, a window minimized mid-batch) or, on a very wide
# multi-monitor window, big enough that width * dpi approaches matplotlib's
# 2**16 pixel limit. Clamp rather than fail: an export's aspect ratio is a
# presentation detail and must never be the reason a batch loses an image.
_MIN_FIG_IN = 1.0
_MAX_FIG_IN = 100.0
def sanitize_figsize(figsize) -> tuple[float, float]:
"""(width, height) in inches, clamped to something renderable.
*figsize* is None (use DEFAULT_FIGSIZE) or any 2-sequence of numbers --
including the float64 pair Figure.get_size_inches returns, which is how
the GUI hands over the live canvas's current size.
"""
try:
w, h = float(figsize[0]), float(figsize[1])
except (TypeError, ValueError, IndexError, KeyError):
return DEFAULT_FIGSIZE
if not (np.isfinite(w) and np.isfinite(h)):
return DEFAULT_FIGSIZE
return (min(max(w, _MIN_FIG_IN), _MAX_FIG_IN),
min(max(h, _MIN_FIG_IN), _MAX_FIG_IN))
def draw_view_image(ax, fig, img: np.ndarray, extent: list[float], cmap,
vmin: float, vmax: float, xlabel: str, ylabel: str,
title: str, colorbar_label: str = "", cb_ticks=None,
norm=None, bad_color=None):
"""imshow + colorbar + labels onto an already-created (ax, fig) pair.
*cmap* may be a name or a Colormap instance. *norm* (which overrides
vmin/vmax) and *cb_ticks* let a caller draw a discrete integer image
with whole-number colorbar bands instead of a continuous shade.
*bad_color*, if given, is the fill for NaN pixels -- a copy of *cmap* is
made so a shared, registered instance is never mutated.
Shared by ImageCanvas.show_image (Qt-backed ax/fig) and
export_view_image (headless Agg ax/fig) so the two can never drift into
showing different things for the same settings.
"""
if bad_color is not None:
cmap = (cmap if hasattr(cmap, "with_extremes")
else mpl.colormaps[cmap]).with_extremes(bad=bad_color)
kw = ({"norm": norm} if norm is not None
else {"vmin": vmin, "vmax": vmax})
im = ax.imshow(
img, aspect="auto", origin="upper",
extent=extent, cmap=cmap, interpolation="nearest", **kw,
)
cb = fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04, ticks=cb_ticks)
if colorbar_label:
cb.set_label(colorbar_label)
ax.set_xlabel(xlabel)
ax.set_ylabel(ylabel)
ax.set_title(title)
return im
def export_view_image(path: str, *, out_dir: str, angle_idx: int, ch_idx: int,
is_fft_mode: bool, is_velocity: bool,
dc_threshold_mv: float, apply_bg_sub: bool,
pad_factor: int, min_freq_mhz: float, grating_um: float,
cmap: str, auto_scale: bool, vmin: float, vmax: float,
highlight_masked: bool, mode_str: str,
colorbar_label: str, mask_color: str = "magenta",
max_workers: int | None = None,
figsize: tuple[float, float] | None = None,
dpi: int = 150) -> tuple[str, str]:
"""One file's contribution to Batch Export View as Images: renders
(angle_idx, ch_idx) at the given display settings to a PNG under
*out_dir*, via draw_view_image -- so a batch export is a folder of what
ImageCanvas.show_image would have put on screen for these settings, not
a raw data dump.
Module-level and picklable, like sras_compute.cache_file, so it can run
in a ProcessPoolExecutor -- see BatchExportImagesWorker. Unlike
cache_file this never writes to *path*: export is a read of the file's
own data, not a cache conversion, so any version SrasFile can open
works, with no v6/v7 precondition.
*figsize* is the on-screen ImageCanvas's current size in inches, so the
PNG carries the aspect ratio the view was actually being read at. It
matters more here than it would for a plot with fixed data aspect:
draw_view_image uses aspect="auto", so the image stretches to whatever
box it is given -- rendering a view the user has sized wide into a
hard-coded 7x5 squeezes the map into a different shape than the one
they judged it by. Passing the full size, not just the ratio, also
keeps titles, tick labels and the colorbar in the same proportion to
the map as on screen; *dpi* alone then sets the output resolution.
*pad_factor* (not n_fft) travels across files deliberately: n_fft
depends on samples_per_frame, which can differ between files in the
same batch, so n_fft is derived per file, here, from *this* file's own
value -- the same reason sras_compute.cache_file does the same thing.
row_avg_n is per file for the same reason and so is not a parameter at
all: it comes from *this* file's stored cache, mirroring
SrasViewerWindow._stored_fft_image and _start_compute, which both ask at
the window size the store was written at. Leaving it at compute_rf_image's
"raw per-pixel" default would make a row-averaged cache a mismatch, so a
file the viewer displays from its store would export as a full raw
recompute instead -- a different (and much slower) image than the one the
batch was triggered to reproduce.
Returns (error, out_name). error is "" on success. out_name is the
filename this call targeted -- set as soon as it's known, even on most
failures -- so the caller can flag same-stem collisions across the
batch without any cross-process bookkeeping.
"""
out_name = ""
try:
sras = SrasFile(path)
if angle_idx >= sras.n_angles:
return (f"angle {angle_idx} out of range "
f"(file has {sras.n_angles} angle(s))", out_name)
out_name = f"{Path(path).stem}_angle{angle_idx}_{CH_NAMES[ch_idx]}.png"
if is_fft_mode:
n_fft = (sras.samples_per_frame * pad_factor
if pad_factor > 1 else None)
freq = compute_rf_image(
sras, angle_idx, dc_threshold_mv=dc_threshold_mv,
apply_bg_sub=apply_bg_sub, n_fft=n_fft,
row_avg_n=sras.precomputed_row_avg_n,
min_freq_mhz=min_freq_mhz, max_workers=max_workers)
img = freq * grating_um if is_velocity else freq
else:
img = dc_image_mv(sras, angle_idx, ch_idx, max_workers=max_workers)
display_img, bad_color = img, None
if highlight_masked and is_fft_mode:
# Mirrors the viewer's _redraw_image rule: DC-masked pixels and
# value-0 pixels (the "no valid peak" sentinel — DC-masked,
# below the min-freq floor, or empty spectrum; the grating
# multiply above preserves zeros, so this holds for Velocity
# too) both render in the highlight color.
dc4 = dc_image_mv(sras, angle_idx, CH4_IDX, max_workers=max_workers)
valid = dc4 >= dc_threshold_mv
if valid.shape == display_img.shape:
valid &= display_img != 0.0
display_img = display_img.astype(np.float32, copy=True)
display_img[~valid] = np.nan
bad_color = mask_color
if auto_scale:
v0, v1 = float(np.nanmin(display_img)), float(np.nanmax(display_img))
if not np.isfinite(v0):
v0, v1 = 0.0, 0.0 # every pixel masked out
else:
v0, v1 = vmin, vmax
x_axis = sras.x_axis_mm(angle_idx)
y_axis = sras.y_positions_mm(angle_idx)
dx = x_axis[1] - x_axis[0] if len(x_axis) > 1 else sras.pixel_x_mm
dy = float(y_axis[1] - y_axis[0]) if len(y_axis) > 1 else 1.0
extent = _axes_extent(x_axis, y_axis, dx, dy)
title = (f"{CH_NAMES[ch_idx]} | {mode_str} | "
f"{sras.angles_deg[angle_idx]:.1f}°")
fig = Figure(figsize=sanitize_figsize(figsize), tight_layout=True)
FigureCanvasAgg(fig) # Agg-only: never registered with pyplot
ax = fig.add_subplot(111)
draw_view_image(ax, fig, display_img, extent, cmap, v0, v1,
"X (mm)", "Y (mm)", title, colorbar_label,
bad_color=bad_color)
fig.savefig(str(Path(out_dir) / out_name), dpi=dpi)
return ("", out_name)
except Exception as exc:
return (str(exc), out_name)
-2352
View File
File diff suppressed because it is too large Load Diff
+29
View File
@@ -0,0 +1,29 @@
"""
SRAS Scan File Viewer
PyQt6 application for visualizing channel data from .sras binary scan files.
Channel semantics (fixed by sc3_aui_app.py acquisition settings):
CH1 — RF Acoustic Packet (AC-coupled, 100 mV/div): FFT → peak frequency
CH3 — Bias A (DC-coupled, 50 mV/div): waveform mean
CH4 — Bias B (DC-coupled, 50 mV/div): waveform mean
RF images are masked: pixels where CH4_dc < dc_threshold show 0.
File parsing lives in sras_format, image/alignment math in sras_compute, and
background workers in sras_workers — none of which import Qt or matplotlib,
so multiprocessing children can load them cheaply.
"""
import faulthandler
faulthandler.enable() # print a native stack trace on SIGSEGV/SIGABRT/etc.
from .align_wizard import AlignmentWizard # noqa: E402,F401
from .canvases import ( # noqa: E402,F401
AlignOverlayCanvas, ImageCanvas, RoiQuad, WaveformCanvas,
)
from .common import CH_LABELS, CMAPS, VELOCITY_MODE_IDX # noqa: E402,F401
from .dialogs import ( # noqa: E402,F401
FftOptionsDialog, FusedRoiExportDialog, RowAverageFftOptionsDialog,
)
from .main_window import SrasViewerWindow, main # noqa: E402,F401
+4
View File
@@ -0,0 +1,4 @@
from .main_window import main
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
+647
View File
@@ -0,0 +1,647 @@
"""Matplotlib canvases and the ROI primitive."""
import matplotlib as mpl
import numpy as np
from matplotlib.backends.backend_qtagg import FigureCanvasQTAgg
from matplotlib.colors import BoundaryNorm, ListedColormap
from matplotlib.figure import Figure
from matplotlib.patches import Polygon
from matplotlib.path import Path as MplPath
from PyQt6.QtCore import Qt, pyqtSignal
from PyQt6.QtGui import QKeyEvent
from PyQt6.QtWidgets import QSizePolicy
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, CH_NAMES, SrasFile, adc_to_mv
from sras_render import draw_view_image
def count_colormap(n_angles: int):
"""(cmap, norm, ticks) for an integer "how many angles cover this pixel"
image, 0..n_angles.
Discrete, not continuous: the judgement the wizard's stack view exists for
is "is this a plateau at N, or a fan of partial overlaps", so a region
covered by one angle too few has to read as its own band rather than a
slightly darker shade. Count 0 is fully transparent so uncovered canvas
cannot be mistaken for a low count.
Shared by both wizard pages that draw this image — they use different canvas
classes, and the same number must not change colour between them.
"""
n = max(1, int(n_angles))
base = mpl.colormaps["viridis"].resampled(n)
colors = [(0.0, 0.0, 0.0, 0.0)] + [base(i) for i in range(n)]
return (ListedColormap(colors),
BoundaryNorm(np.arange(-0.5, n + 1), len(colors)),
np.arange(0, n + 1))
# ---------------------------------------------------------------------------
# ROI (free quadrilateral in data coordinates)
# ---------------------------------------------------------------------------
class RoiQuad:
"""Free quadrilateral defined in data coordinates (mm).
Stored as 4 corner points (shape (4, 2)) in CCW order: BL, BR, TR, TL.
Each corner can be positioned independently, allowing skewed /
non-orthogonal regions of interest. Because it lives in scan/data
coords it persists unchanged when the displayed channel/mode switches.
"""
def __init__(self, pts: np.ndarray):
"""pts : array-like, shape (4, 2)."""
self._pts = np.asarray(pts, dtype=np.float64).reshape(4, 2).copy()
@classmethod
def from_bbox(cls, x0: float, y0: float, x1: float, y1: float) -> "RoiQuad":
"""Create an axis-aligned rectangle from two opposite corners."""
lx, rx = min(x0, x1), max(x0, x1)
by, ty = min(y0, y1), max(y0, y1)
return cls(np.array([[lx, by], [rx, by], [rx, ty], [lx, ty]]))
def copy(self) -> "RoiQuad":
return RoiQuad(self._pts.copy())
def corners(self) -> np.ndarray:
"""World-coord corners, shape (4, 2), CCW: BL, BR, TR, TL."""
return self._pts.copy()
def centroid(self) -> np.ndarray:
return self._pts.mean(axis=0)
def bbox_size(self) -> np.ndarray:
"""Width and height of the axis-aligned bounding box, shape (2,)."""
return self._pts.max(axis=0) - self._pts.min(axis=0)
def contains(self, x: float, y: float) -> bool:
return bool(MplPath(self._pts).contains_point((x, y)))
def mask_for_grid(self, x_axis: np.ndarray,
y_axis: np.ndarray) -> np.ndarray:
"""Boolean mask (n_rows, n_frames) of pixels whose centres lie
inside the quadrilateral.
Only the quad's axis-aligned bounding box is tested — meshgrid and
contains_points over the *whole* grid would be tens of millions of
point-in-polygon tests (and hundreds of MB of float64 temporaries)
on a large scan, on every ROI edit.
"""
x = np.asarray(x_axis, dtype=np.float64)
y = np.asarray(y_axis, dtype=np.float64)
mask = np.zeros((y.size, x.size), dtype=bool)
(x0, y0), (x1, y1) = self._pts.min(axis=0), self._pts.max(axis=0)
cols = np.nonzero((x >= x0) & (x <= x1))[0]
rows = np.nonzero((y >= y0) & (y <= y1))[0]
if cols.size == 0 or rows.size == 0:
return mask
c0, c1 = int(cols[0]), int(cols[-1]) + 1
r0, r1 = int(rows[0]), int(rows[-1]) + 1
X, Y = np.meshgrid(x[c0:c1], y[r0:r1])
inside = MplPath(self._pts).contains_points(
np.column_stack([X.ravel(), Y.ravel()]))
mask[r0:r1, c0:c1] = inside.reshape(X.shape)
return mask
# ---------------------------------------------------------------------------
# Matplotlib canvases
# ---------------------------------------------------------------------------
class ImageCanvas(FigureCanvasQTAgg):
pixel_clicked = pyqtSignal(int, int) # row_idx, frame_idx
roi_changed = pyqtSignal() # ROI created / edited / cleared
draw_mode_changed = pyqtSignal(bool) # "draw new ROI" arm toggled
# Interaction state values
_IDLE = "idle"
_DRAW_NEW = "draw_new"
_MOVE = "move"
_DRAG_CORNER = "drag_corner"
# Hit tolerance (display pixels) for handles.
_HANDLE_PX = 12
_CLICK_THRESH_PX = 4 # releases within this of press count as a click
def __init__(self, parent=None, *, rect_only: bool = False):
"""*rect_only* constrains the ROI to an axis-aligned rectangle.
Used by the alignment wizard's crop page, where a free quadrilateral
would be actively misleading: v6 geometry can only express an
axis-aligned rectangle, so anything else the user drew would have to be
squared off behind their back. Default off, so the main window's
free-quad ROI is unaffected.
"""
fig = Figure(figsize=(7, 5), tight_layout=True)
self.ax = fig.add_subplot(111)
super().__init__(fig)
self.setParent(parent)
self.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)
self._extent = None
self._img_shape = None
self._rect_only = rect_only
# ROI state
self._roi: RoiQuad | None = None
self._roi_artists: list = []
self._state = self._IDLE
self._draw_mode = False
# Per-interaction snapshots / anchors
self._press_xy: tuple[float, float] | None = None
self._press_pixel: tuple[float, float] | None = None
self._press_button = None
self._snapshot: RoiQuad | None = None
self._drag_corner_idx: int = -1
self._move_anchor = None # press-point in world coords
self._draw_previous: RoiQuad | None = None
self.mpl_connect("button_press_event", self._on_press)
self.mpl_connect("motion_notify_event", self._on_motion)
self.mpl_connect("button_release_event", self._on_release)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def show_image(self, img: np.ndarray, extent: list[float], cmap,
vmin: float, vmax: float, xlabel: str, ylabel: str, title: str,
colorbar_label: str = "", cb_ticks=None, norm=None,
bad_color=None):
"""*cmap* may be a name or a Colormap instance. *norm* (which overrides
vmin/vmax) and *cb_ticks* let a caller draw a discrete integer image —
the wizard's overlap-count view — with whole-number colorbar bands
instead of a continuous shade. *bad_color*, if given, is the fill for
NaN pixels — a copy of *cmap* is made so the shared, registered
instance is never mutated."""
self.figure.clf()
self.ax = self.figure.add_subplot(111)
# Patches and lines are destroyed by figure.clf(); drop stale refs.
self._roi_artists = []
self._extent = extent
self._img_shape = img.shape
# Shared with the headless batch image-export worker (sras_render.py)
# so an exported PNG can never quietly drift from what this canvas
# shows on screen for the same settings.
draw_view_image(self.ax, self.figure, img, extent, cmap, vmin, vmax,
xlabel, ylabel, title, colorbar_label, cb_ticks, norm,
bad_color)
# Re-draw the ROI (if any) on top of the fresh image so it persists
# unchanged across mode / angle / channel switches.
self._draw_roi()
self.draw()
def get_roi(self) -> RoiQuad | None:
return self._roi
def set_roi(self, roi: RoiQuad | None):
self._roi = roi.copy() if roi is not None else None
self._draw_roi()
self.draw_idle()
self.roi_changed.emit()
def clear_roi(self):
self._roi = None
self._remove_roi_artists()
self.draw_idle()
self.roi_changed.emit()
def start_drawing(self):
"""Arm the next click+drag on the image to create a new ROI,
replacing any existing one."""
self._draw_mode = True
self.setCursor(Qt.CursorShape.CrossCursor)
self.draw_mode_changed.emit(True)
def cancel_drawing(self):
if self._draw_mode:
self._draw_mode = False
self.setCursor(Qt.CursorShape.ArrowCursor)
self.draw_mode_changed.emit(False)
# ------------------------------------------------------------------
# Rendering
# ------------------------------------------------------------------
def _remove_roi_artists(self):
for a in self._roi_artists:
try:
a.remove()
except (ValueError, AttributeError, NotImplementedError):
pass
self._roi_artists = []
def _draw_roi(self):
self._remove_roi_artists()
if self._roi is None or self.ax is None:
return
corners = self._roi.corners()
# Filled quad, then a sharp unfilled edge for visibility over bright
# images, then draggable corner handles.
for kwargs in (
dict(fill=True, facecolor="#ffd93a", edgecolor="#e53935",
alpha=0.22, linewidth=2.0, zorder=10),
dict(fill=False, edgecolor="#e53935", linewidth=1.8, zorder=11),
):
patch = Polygon(corners, closed=True, **kwargs)
self.ax.add_patch(patch)
self._roi_artists.append(patch)
self._roi_artists.append(self.ax.scatter(
corners[:, 0], corners[:, 1], s=60, c="white",
edgecolors="#e53935", linewidths=1.6, zorder=13))
# ------------------------------------------------------------------
# Hit testing (display pixels for handles, data coords for "inside")
# ------------------------------------------------------------------
def _hit_test(self, event) -> tuple[str, int | None] | None:
if self._roi is None or self.ax is None:
return None
if event.x is None or event.y is None:
return None
corners_disp = self.ax.transData.transform(self._roi.corners())
click = np.array([event.x, event.y])
for i in range(4):
if np.hypot(*(corners_disp[i] - click)) <= self._HANDLE_PX:
return ("corner", i)
if event.xdata is not None and event.ydata is not None:
if self._roi.contains(event.xdata, event.ydata):
return ("inside", None)
return None
# ------------------------------------------------------------------
# Mouse event handlers
# ------------------------------------------------------------------
def _on_press(self, event):
if event.inaxes is not self.ax or self._extent is None:
return
if event.button != 1: # only left mouse button
return
# If the matplotlib toolbar is in pan / zoom mode, let it handle
# the interaction instead of starting a ROI manipulation.
tb = getattr(self, "toolbar", None)
if tb is not None and getattr(tb, "mode", ""):
return
self._press_xy = (event.xdata, event.ydata)
self._press_pixel = (event.x, event.y)
self._press_button = event.button
if self._draw_mode:
self._draw_previous = self._roi.copy() if self._roi else None
self._roi = RoiQuad.from_bbox(event.xdata, event.ydata,
event.xdata, event.ydata)
self._state = self._DRAW_NEW
self._draw_roi()
self.draw_idle()
return
hit = self._hit_test(event)
if hit is None:
self._state = self._IDLE
return
kind, idx = hit
self._snapshot = self._roi.copy()
if kind == "corner":
self._state = self._DRAG_CORNER
self._drag_corner_idx = idx
else:
self._state = self._MOVE
self._move_anchor = (event.xdata, event.ydata)
def _on_motion(self, event):
if self._state == self._IDLE:
return
if event.xdata is None or event.ydata is None:
return
if event.inaxes is not self.ax:
return
if self._state == self._DRAW_NEW:
x0, y0 = self._press_xy
self._roi = RoiQuad.from_bbox(x0, y0, event.xdata, event.ydata)
elif self._state == self._MOVE:
delta = np.array([event.xdata - self._move_anchor[0],
event.ydata - self._move_anchor[1]])
self._roi._pts = self._snapshot.corners() + delta
elif self._state == self._DRAG_CORNER:
self._roi._pts[self._drag_corner_idx] = [event.xdata, event.ydata]
if self._rect_only:
self._rectify_corner(self._drag_corner_idx)
self._draw_roi()
self.draw_idle()
def _rectify_corner(self, idx: int):
"""Re-square the quad after a corner drag, anchored on the *opposite*
corner.
Anchoring on the diagonal opposite (idx ^ 2, since corners run
BL, BR, TR, TL) rather than taking the bbox of all four points is what
lets the rectangle shrink: a bbox over the three stale corners plus the
new one is the union of the old rectangle and the new point, so dragging
inward would never make it smaller.
"""
pts = self._roi.corners()
ax_, ay = pts[idx ^ 2]
bx, by = pts[idx]
self._roi._pts = RoiQuad.from_bbox(min(ax_, bx), min(ay, by),
max(ax_, bx), max(ay, by)).corners()
def _on_release(self, event):
if event.button != 1 and self._press_button != 1:
return
prev_state = self._state
self._state = self._IDLE
try:
if prev_state == self._DRAW_NEW:
self._finish_draw()
elif prev_state in (self._MOVE, self._DRAG_CORNER):
self._draw_roi()
self.draw_idle()
self.roi_changed.emit()
else:
self._maybe_emit_pixel_click(event)
finally:
self._press_xy = self._press_pixel = None
self._press_button = None
def _finish_draw(self):
"""Commit (or reject) a freshly-dragged quad."""
if self._extent is not None:
x0, x1, y_bot, y_top = self._extent
min_w = abs(x1 - x0) * 0.01 # minimum: 1% of each axis range
min_h = abs(y_bot - y_top) * 0.01
else:
min_w = min_h = 1e-6
if self._roi is None:
too_small = True
else:
bbox = self._roi.bbox_size()
too_small = bbox[0] < min_w or bbox[1] < min_h
if too_small:
self._roi = self._draw_previous
self._draw_previous = None
self.cancel_drawing()
self._draw_roi()
self.draw_idle()
self.roi_changed.emit()
def _maybe_emit_pixel_click(self, event):
"""A release close enough to its press counts as a pixel click."""
if (self._press_pixel is None or event.x is None or event.y is None
or self._extent is None or event.inaxes is not self.ax
or event.xdata is None):
return
dx_px = event.x - self._press_pixel[0]
dy_px = event.y - self._press_pixel[1]
if dx_px * dx_px + dy_px * dy_px > self._CLICK_THRESH_PX ** 2:
return
x0, x1, y_bot, y_top = self._extent
n_rows, n_frames = self._img_shape
col = int((event.xdata - x0) / (x1 - x0) * n_frames)
row = int((event.ydata - y_top) / (y_bot - y_top) * n_rows)
self.pixel_clicked.emit(max(0, min(row, n_rows - 1)),
max(0, min(col, n_frames - 1)))
class WaveformCanvas(FigureCanvasQTAgg):
def __init__(self, parent=None):
fig = Figure(figsize=(8, 3), tight_layout=True)
self.ax_wave = fig.add_subplot(121)
self.ax_right = fig.add_subplot(122)
super().__init__(fig)
self.setParent(parent)
self.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)
def show_rf_waveform(self, sras: SrasFile, angle_idx: int,
row_idx: int, frame_idx: int,
apply_bg_sub: bool = True,
min_freq_mhz: float = 0.0):
"""CH1 RF: time-domain + FFT spectrum.
If apply_bg_sub is True and sras.background is not None, the background
waveform is overlaid on the time-domain plot and the FFT is computed
on the subtracted signal. The unsubtracted FFT is also shown faintly
for comparison.
*min_freq_mhz* > 0 restricts the labeled peak to bins at or above
it — the same floor the image's peak search uses, so the label
explains the map pixel instead of contradicting it — and shades the
excluded band on the spectrum. The spectrum curves themselves stay
complete (they are the evidence for choosing the floor). The peak
can still legitimately differ from a padded or row-averaged map:
this panel is always a single waveform at natural resolution.
"""
data = sras.data[angle_idx]
waveform = data[row_idx, CH1_IDX, frame_idx, :].astype(np.float32)
t_ns = sras.time_axis_ns()
f_mhz = sras.freq_axis_mhz()
dc3_val = data[row_idx, CH3_IDX, frame_idx, :].astype(np.float32).mean()
dc4_val = data[row_idx, CH4_IDX, frame_idx, :].astype(np.float32).mean()
bg = sras.background if (apply_bg_sub and sras.background is not None) else None
waveform_plot = waveform - bg if bg is not None else waveform
self.ax_wave.cla()
self.ax_right.cla()
if bg is not None:
self.ax_wave.plot(t_ns, waveform, linewidth=0.5, color="#aaaaaa",
label="raw", zorder=1)
self.ax_wave.plot(t_ns, bg, linewidth=0.5, color="#e07030",
linestyle="--", label="background", zorder=2)
self.ax_wave.plot(t_ns, waveform_plot, linewidth=0.7, color="#4488cc",
label="subtracted", zorder=3)
self.ax_wave.legend(fontsize=7, loc="upper right")
else:
self.ax_wave.plot(t_ns, waveform, linewidth=0.7, color="#4488cc")
self.ax_wave.set_xlabel("Time (ns)")
self.ax_wave.set_ylabel("ADC counts")
bg_tag = " [bg sub]" if bg is not None else ""
dc3_mv = adc_to_mv(dc3_val, *sras.cal(CH3_IDX))
dc4_mv = adc_to_mv(dc4_val, *sras.cal(CH4_IDX))
self.ax_wave.set_title(
f"CH1 RF row={row_idx} frame={frame_idx}{bg_tag}\n"
f"CH3={dc3_val:.1f} CH4={dc4_val:.1f} "
f"({dc3_mv:.2f} / {dc4_mv:.2f} mV)",
fontsize=8,
)
# FFT of the (possibly subtracted) waveform
power_sub = np.abs(np.fft.rfft(waveform_plot)) ** 2
power_sub[0] = 0.0
# First bin at or above the floor, exactly as the image peak search
# picks it (bin 0 always excluded). If the floor excludes every bin,
# fall back to the unrestricted peak rather than indexing past the
# end — the label is informational, not a mask.
lo = max(1, int(np.searchsorted(f_mhz, min_freq_mhz)))
if lo < len(power_sub):
peak_mhz = f_mhz[lo + int(np.argmax(power_sub[lo:]))]
else:
peak_mhz = f_mhz[int(np.argmax(power_sub))]
if min_freq_mhz > 0.0:
self.ax_right.axvspan(0, min_freq_mhz, color="#888888",
alpha=0.15, zorder=0,
label=f"< {min_freq_mhz:g} MHz excluded")
if bg is not None:
# Also show the unsubtracted FFT for reference
power_raw = np.abs(np.fft.rfft(waveform)) ** 2
power_raw[0] = 0.0
self.ax_right.plot(f_mhz, power_raw, linewidth=0.5, color="#aaaaaa",
label="raw FFT", zorder=1)
self.ax_right.plot(f_mhz, power_sub, linewidth=0.7, color="#4488cc",
label="subtracted FFT" if bg is not None else None, zorder=2)
self.ax_right.axvline(peak_mhz, color="tomato", linestyle="--",
linewidth=1.2, label=f"peak = {peak_mhz:.1f} MHz")
self.ax_right.set_xlabel("Frequency (MHz)")
self.ax_right.set_ylabel("Power (arb.)")
self.ax_right.set_title("FFT Power Spectrum")
self.ax_right.set_xlim(0, 500)
self.ax_right.legend(fontsize=8)
self.draw()
def show_dc_waveform(self, sras: SrasFile, angle_idx: int, ch_idx: int,
row_idx: int, frame_idx: int):
"""CH3 or CH4 DC: time-domain + mean annotation."""
waveform = sras.data[angle_idx][row_idx, ch_idx, frame_idx, :].astype(np.float32)
mean_val = float(waveform.mean())
mean_mv = adc_to_mv(mean_val, *sras.cal(ch_idx))
self.ax_wave.cla()
self.ax_right.cla()
self.ax_wave.plot(sras.time_axis_ns(), waveform, linewidth=0.7, color="#4488cc")
self.ax_wave.axhline(mean_val, color="tomato", linestyle="--",
linewidth=1.2, label=f"mean = {mean_val:.2f} ADC")
self.ax_wave.set_xlabel("Time (ns)")
self.ax_wave.set_ylabel("ADC counts")
self.ax_wave.set_title(
f"{CH_NAMES[ch_idx]} DC row={row_idx} frame={frame_idx}")
self.ax_wave.legend(fontsize=8)
self.ax_right.text(
0.5, 0.5,
f"DC mode\n\nmean = {mean_val:.3f} ADC\n = {mean_mv:.3f} mV",
ha="center", va="center",
transform=self.ax_right.transAxes, fontsize=11,
)
self.ax_right.set_axis_off()
self.draw()
class AlignOverlayCanvas(FigureCanvasQTAgg):
"""Renders the alignment wizard's multi-angle mask views and turns keyboard
input into translate/rotate nudge requests for whichever angle is active.
Two views of the same reprojected masks, because they answer different
questions. show_counts colours each pixel by *how many* angles cover it,
which is the at-a-glance verdict on a correlation run: a good alignment is
one saturated plateau, a bad one is a fringe of low-count halos.
show_overlay gives each angle its own colour, which is what you need while
nudging a specific angle by hand.
A pure input+render widget — it holds no alignment state and never
touches SrasFile itself; the wizard page owns all of that and decides,
from these signals, whether a cheap single-layer refresh or a
full preview-canvas rebuild is needed.
FigureCanvasQTAgg is a real QWidget, so keyPressEvent works like on any
other widget, but Qt only ever delivers key events to whichever widget
currently has focus — StrongFocus, plus grabbing focus on click and once
right after the dialog is shown, are both required or arrow keys
silently do nothing.
Rotate keys are letters (Q/E), not punctuation (comma/period or
brackets): Shift+letter still reports the same Qt.Key on every platform,
whereas Shift+comma/bracket can report a different virtual key
(Key_Less / Key_BraceLeft) depending on platform and keyboard layout —
which would silently break the "Shift = coarse step" modifier for
rotation specifically. Arrow keys have no such hazard.
"""
nudge_translate = pyqtSignal(int, int, bool) # dir_x, dir_y in {-1,0,1}; coarse
nudge_rotate = pyqtSignal(int, bool) # dir in {-1,1} (CCW/CW); coarse
_TRANSLATE_KEYS = {
Qt.Key.Key_Left: (-1, 0),
Qt.Key.Key_Right: (1, 0),
Qt.Key.Key_Up: (0, -1),
Qt.Key.Key_Down: (0, 1),
}
_ROTATE_KEYS = {Qt.Key.Key_Q: 1, Qt.Key.Key_E: -1} # CCW, CW
def __init__(self, parent=None):
fig = Figure(figsize=(6, 6), tight_layout=True)
self.ax = fig.add_subplot(111)
super().__init__(fig)
self.setParent(parent)
self.setFocusPolicy(Qt.FocusPolicy.StrongFocus)
self.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)
self.mpl_connect("button_press_event", lambda _e: self.setFocus())
def show_overlay(self, rgba: np.ndarray, extent: list[float], title: str):
self.figure.clf()
self.ax = self.figure.add_subplot(111)
self.ax.imshow(rgba, extent=extent, origin="upper", aspect="auto")
self._finish(title)
def show_counts(self, counts: np.ndarray, n_angles: int,
extent: list[float], title: str):
"""The mask stack coloured by how many angles cover each pixel.
A discrete colormap with integer-ticked colorbar rather than a
continuous one: the judgement being made is "is this a plateau at N, or
a fan of partial overlaps", and a region covered by one angle too few
has to read as its own band rather than a slightly darker shade.
Uncovered pixels are transparent so they cannot be mistaken for a low
count.
"""
self.figure.clf()
self.ax = self.figure.add_subplot(111)
cmap, norm, ticks = count_colormap(n_angles)
im = self.ax.imshow(
np.asarray(counts), extent=extent, origin="upper", aspect="auto",
interpolation="nearest", cmap=cmap, norm=norm)
cb = self.figure.colorbar(im, ax=self.ax, fraction=0.046, pad=0.04,
ticks=ticks)
cb.set_label("angles overlapping")
self._finish(title)
def _finish(self, title: str):
self.ax.set_xlabel("X (mm)")
self.ax.set_ylabel("Y (mm)")
self.ax.set_title(title)
self.draw_idle() # coalesces rapid redraws — matters for key-repeat.
def keyPressEvent(self, event: QKeyEvent):
key = event.key()
coarse = bool(event.modifiers() & Qt.KeyboardModifier.ShiftModifier)
if key in self._TRANSLATE_KEYS:
dx, dy = self._TRANSLATE_KEYS[key]
self.nudge_translate.emit(dx, dy, coarse)
event.accept()
elif key in self._ROTATE_KEYS:
self.nudge_rotate.emit(self._ROTATE_KEYS[key], coarse)
event.accept()
else:
super().keyPressEvent(event)
+176
View File
@@ -0,0 +1,176 @@
"""Shared constants and small layout helpers for the viewer widgets."""
from PyQt6.QtCore import Qt
from PyQt6.QtWidgets import (
QComboBox, QDoubleSpinBox, QFormLayout, QFrame, QGroupBox, QLabel,
QScrollArea, QSizePolicy, QVBoxLayout, QWidget,
)
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, _axes_extent # noqa: F401 (re-exported)
# ---------------------------------------------------------------------------
# Display constants
# ---------------------------------------------------------------------------
CH_LABELS = [
"CH1 — RF (FFT peak freq)",
"CH3 — Bias A (DC mean)",
"CH4 — Bias B (DC mean)",
"CH1 — Velocity (SRAS)",
]
# Combo index for the derived velocity mode (uses CH1_IDX data)
VELOCITY_MODE_IDX = 3
# All modes that operate on CH1 waveforms
CH1_DERIVED_MODES = (CH1_IDX, VELOCITY_MODE_IDX)
CMAPS = ["gray", "viridis", "plasma", "inferno", "hot", "jet", "RdBu_r", "seismic"]
# Fill color for masked (below-DC-threshold) pixels when "Highlight masked
# pixels" is on, chosen to stand out against every colormap in CMAPS above.
_MASKED_HIGHLIGHT_COLOR = "magenta"
# (mode_str, status-bar unit, colorbar label) per channel index
_CHANNEL_DISPLAY = {
CH1_IDX: ("RF", "Peak frequency (MHz)", "MHz"),
CH3_IDX: ("DC", "DC mean (mV)", "mV"),
CH4_IDX: ("DC", "DC mean (mV)", "mV"),
VELOCITY_MODE_IDX: ("Velocity", "Velocity (m/s)", "m/s"),
}
_CSS_HINT = "font-size: 11px; color: #aaa;"
_CSS_INFO = "font-size: 11px;"
_CSS_MUTED = "color: #888; font-size: 11px;"
_CSS_WARN = "color: #e07000; font-size: 11px;"
_CSS_BUSY = "color: #4a90d9; font-size: 11px;"
# Side-panel column widths (the scroll areas that hold the controls).
_LEFT_PANEL_W = 288
_RIGHT_PANEL_W = 272
# Minimum width for a spin box so its value + suffix are never clipped.
_SPIN_MIN_W = 96
# ---------------------------------------------------------------------------
# Small layout helpers
# ---------------------------------------------------------------------------
class Jobs:
"""Keys for SrasViewerWindow's background-job registry (_run_worker /
_job_running) and its progress dialogs — one place instead of string
literals scattered across window and dialogs."""
LOAD = "load"
COMPUTE = "compute"
DC_PRECOMPUTE = "dc_precompute"
BATCH = "batch"
EXPORT = "export"
# The alignment wizard's three background steps: fetching each angle's CH4
# image for the mask stack, registering the angles, and writing the aligned
# export. Separate keys because a retry of one must not be blocked by
# another having run, and _run_worker's busy check is per key.
ALIGN_MASKS = "align_masks"
ALIGN_CORRELATE = "align_correlate"
ALIGN_EXPORT = "align_export"
def _make_dspin(lo: float, hi: float, decimals: int, *, suffix: str = "",
value: float | None = None, step: float | None = None) -> QDoubleSpinBox:
"""A QDoubleSpinBox with the panel-standard construction."""
spin = QDoubleSpinBox()
spin.setRange(lo, hi)
spin.setDecimals(decimals)
if suffix:
spin.setSuffix(suffix)
if step is not None:
spin.setSingleStep(step)
if value is not None:
spin.setValue(value)
spin.setMinimumWidth(_SPIN_MIN_W)
return spin
def _combo(items=(), *, min_chars: int = 10) -> QComboBox:
"""A combo box whose size hint does not depend on its longest entry.
By default a QComboBox asks for enough width to show its widest item. These
hold descriptive phrases, and the side panels are fixed-width — in a scroll
area with the horizontal scrollbar off (`_scroll_panel`) an unconstrained
hint pushes the inner widget past the panel and everything on the right,
including the hint text, is silently clipped instead of scrolling.
*items* is a sequence of (label, data) pairs, or of plain labels.
"""
combo = QComboBox()
combo.setSizeAdjustPolicy(
QComboBox.SizeAdjustPolicy.AdjustToMinimumContentsLengthWithIcon)
combo.setMinimumContentsLength(min_chars)
for item in items:
if isinstance(item, tuple):
combo.addItem(item[0], item[1])
else:
combo.addItem(item)
return combo
def _wrap_label(text: str = "", css: str | None = None) -> QLabel:
"""A word-wrapped QLabel that reports its *wrapped* height to the layout.
A plain word-wrapped QLabel advertises a single-line minimum height, so in a
fixed-width column the layout happily shrinks it and the extra lines get
clipped. Enabling height-for-width makes the box layout ask for the real
height at the column's width instead.
"""
lbl = QLabel(text)
lbl.setWordWrap(True)
sp = lbl.sizePolicy()
sp.setVerticalPolicy(QSizePolicy.Policy.Minimum)
sp.setHeightForWidth(True)
lbl.setSizePolicy(sp)
if css:
lbl.setStyleSheet(css)
return lbl
def _group(title: str) -> tuple[QGroupBox, QVBoxLayout]:
"""A group box with consistent, non-cramped internal margins."""
grp = QGroupBox(title)
lay = QVBoxLayout(grp)
lay.setContentsMargins(10, 8, 10, 10)
lay.setSpacing(6)
return grp, lay
def _form() -> QFormLayout:
"""A label/field form layout for a narrow side panel."""
form = QFormLayout()
form.setContentsMargins(0, 0, 0, 0)
form.setHorizontalSpacing(8)
form.setVerticalSpacing(6)
form.setLabelAlignment(Qt.AlignmentFlag.AlignRight
| Qt.AlignmentFlag.AlignVCenter)
form.setFormAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignTop)
form.setFieldGrowthPolicy(
QFormLayout.FieldGrowthPolicy.AllNonFixedFieldsGrow)
form.setRowWrapPolicy(QFormLayout.RowWrapPolicy.DontWrapRows)
return form
def _scroll_panel(inner: QWidget, width: int) -> QScrollArea:
"""Put a side panel in a fixed-width scroll area.
Without this the panels are sized by the window: a short window squeezes the
controls past their minimum heights, which is what makes text overlap the
widget below it. Scrolling keeps every control at its natural size.
"""
area = QScrollArea()
area.setWidget(inner)
area.setWidgetResizable(True)
area.setFrameShape(QFrame.Shape.NoFrame)
area.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
area.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAsNeeded)
area.setFixedWidth(width)
area.viewport().setAutoFillBackground(False)
inner.setAutoFillBackground(False)
return area
+523
View File
@@ -0,0 +1,523 @@
"""FFT option dialogs.
Angle alignment used to live here too, as ManualAlignmentDialog; it is now the
alignment wizard's first page (see align_wizard.py), which needed the same
mask-overlay editor plus the crop and export steps.
"""
from pathlib import Path
from PyQt6.QtWidgets import (
QButtonGroup, QCheckBox, QDialog, QDialogButtonBox, QDoubleSpinBox,
QFileDialog, QGridLayout, QGroupBox, QHBoxLayout, QLabel, QLineEdit,
QMessageBox, QPushButton, QRadioButton, QScrollArea, QSpinBox,
QVBoxLayout, QWidget,
)
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, CH_NAMES
from sras_workers import ExportChannel
from .common import (
CH_LABELS, VELOCITY_MODE_IDX, _CHANNEL_DISPLAY, _CSS_HINT, _CSS_WARN,
_SPIN_MIN_W, _form, _group, _make_dspin, _wrap_label,
)
# ---------------------------------------------------------------------------
# FFT Options dialog
# ---------------------------------------------------------------------------
class FftOptionsDialog(QDialog):
"""Configure FFT zero-padding.
Changes take effect only when the user clicks Apply. Cancel discards
all pending edits. The live 'frequency resolution' label updates as
the user adjusts the pad factor so they can see the trade-off before
committing.
"""
def __init__(self, parent=None, *,
current_pad_factor: int,
samples_per_frame: int | None,
sample_rate_hz: float | None,
grating_um: float):
super().__init__(parent)
self.setWindowTitle("FFT Options")
self.setModal(True)
self.setMinimumWidth(380)
self._samples_per_frame = samples_per_frame
self._sample_rate_hz = sample_rate_hz
self._grating_um = grating_um
layout = QVBoxLayout(self)
# ---- Zero-padding ----------------------------------------------
grp_zp = QGroupBox("Zero-Padding")
zl = QVBoxLayout(grp_zp)
pad_row = QHBoxLayout()
pad_row.addWidget(QLabel("Pad factor:"))
self._spin_pad = QSpinBox()
self._spin_pad.setRange(1, 256)
self._spin_pad.setValue(max(1, current_pad_factor))
self._spin_pad.setToolTip(
"Multiply the waveform length by this factor via zero-padding\n"
"before computing the FFT.\n"
"1 = no padding (natural length).\n"
"Powers of 2 (2, 4, 8 …) give the best performance."
)
self._spin_pad.valueChanged.connect(self._update_info)
pad_row.addWidget(self._spin_pad)
zl.addLayout(pad_row)
self._lbl_nfft = QLabel()
self._lbl_freq_res = QLabel()
self._lbl_vel_res = QLabel()
for lbl in (self._lbl_nfft, self._lbl_freq_res, self._lbl_vel_res):
lbl.setStyleSheet(_CSS_HINT)
zl.addWidget(lbl)
layout.addWidget(grp_zp)
# ---- Buttons ---------------------------------------------------
buttons = QDialogButtonBox()
buttons.addButton("Apply", QDialogButtonBox.ButtonRole.AcceptRole
).clicked.connect(self.accept)
buttons.addButton("Cancel", QDialogButtonBox.ButtonRole.RejectRole
).clicked.connect(self.reject)
layout.addWidget(buttons)
self._update_info()
def _update_info(self):
spf = self._samples_per_frame
sr = self._sample_rate_hz
pad = self._spin_pad.value()
if spf is None or sr is None:
self._lbl_nfft.setText("Load a file to preview FFT parameters.")
self._lbl_freq_res.setText("")
self._lbl_vel_res.setText("")
return
n_fft = spf * pad
freq_res_hz = sr / n_fft
freq_res_mhz = freq_res_hz / 1e6
# v (m/s) = freq (MHz) × grating (µm)
vel_res_ms = freq_res_mhz * self._grating_um
self._lbl_nfft.setText(f"FFT points: {spf} × {pad} = {n_fft:,}")
self._lbl_freq_res.setText(
f"Frequency bin: {freq_res_mhz:.4f} MHz ({freq_res_hz / 1e3:.2f} kHz)")
self._lbl_vel_res.setText(
f"Velocity bin: {vel_res_ms:.3f} m/s "
f"(at grating = {self._grating_um:.2f} µm)")
def get_pad_factor(self) -> int:
return max(1, self._spin_pad.value())
# ---------------------------------------------------------------------------
# Row-Averaged FFT Options dialog
# ---------------------------------------------------------------------------
class RowAverageFftOptionsDialog(QDialog):
"""Configure the same-row, distance-weighted neighbor averaging applied
to each pixel's CH1 waveform before 'Batch Compute Row-Averaged FFT and
Store' re-runs the FFT peak search — a same-row SNR cleanup pass, never
mixing across rows/Y (see sras_compute._row_average_waveforms).
Unlike the plain FFT batch action (which stores unmasked and defers
masking to display time), the DC threshold here is required up front:
it decides which same-row neighbors are eligible to contribute to a
pixel's average, so it can't be deferred.
Changes take effect only when the user clicks Apply. Cancel discards
all pending edits.
"""
def __init__(self, parent=None, *,
current_n: int,
current_threshold_mv: float,
pixel_x_mm: float | None):
super().__init__(parent)
self.setWindowTitle("Row-Averaged FFT Options")
self.setModal(True)
self.setMinimumWidth(380)
self._pixel_x_mm = pixel_x_mm
layout = QVBoxLayout(self)
# ---- Neighbor window ---------------------------------------------
grp_window = QGroupBox("Same-Row Neighbor Window")
wl = QVBoxLayout(grp_window)
n_row = QHBoxLayout()
n_row.addWidget(QLabel("Neighbor half-width (n):"))
self._spin_n = QSpinBox()
self._spin_n.setRange(1, 50)
self._spin_n.setValue(max(1, current_n))
self._spin_n.setToolTip(
"Each pixel's CH1 waveform is averaged with up to n same-row\n"
"neighbors on each side, distance-weighted (Gaussian) and\n"
"counting only neighbors that already pass the DC threshold\n"
"below. Never mixes across rows/Y.")
self._spin_n.valueChanged.connect(self._update_info)
n_row.addWidget(self._spin_n)
wl.addLayout(n_row)
self._lbl_width = QLabel()
self._lbl_width.setStyleSheet(_CSS_HINT)
wl.addWidget(self._lbl_width)
layout.addWidget(grp_window)
# ---- DC threshold ------------------------------------------------
grp_thr = QGroupBox("Neighbor Validity")
tl = QVBoxLayout(grp_thr)
thr_row = QHBoxLayout()
thr_row.addWidget(QLabel("DC threshold:"))
self._spin_threshold = _make_dspin(-500.0, 500.0, 3, suffix=" mV",
value=current_threshold_mv, step=0.025)
self._spin_threshold.setToolTip(
"A same-row neighbor only contributes to a pixel's average if\n"
"its own CH4 signal is at or above this threshold -- the same\n"
"test used for RF mask display. A pixel below threshold stays\n"
"masked, exactly as today; it is never rescued by its neighbors.")
thr_row.addWidget(self._spin_threshold)
tl.addLayout(thr_row)
layout.addWidget(grp_thr)
# ---- Buttons -----------------------------------------------------
buttons = QDialogButtonBox()
buttons.addButton("Apply", QDialogButtonBox.ButtonRole.AcceptRole
).clicked.connect(self.accept)
buttons.addButton("Cancel", QDialogButtonBox.ButtonRole.RejectRole
).clicked.connect(self.reject)
layout.addWidget(buttons)
self._update_info()
def _update_info(self):
n = self._spin_n.value()
if self._pixel_x_mm is None:
self._lbl_width.setText("Load a file to preview the window's physical width.")
return
width_um = 2 * n * self._pixel_x_mm * 1e3
self._lbl_width.setText(
f"Window: ±{n} px = {width_um:.2f} µm full width "
f"(pixel pitch {self._pixel_x_mm * 1e3:.3g} µm)")
def get_half_width(self) -> int:
return self._spin_n.value()
def get_threshold_mv(self) -> float:
return self._spin_threshold.value()
# ---------------------------------------------------------------------------
# Batch export dialog (Export -> Batch Export Images...)
# ---------------------------------------------------------------------------
class BatchExportDialog(QDialog):
"""Configure a batch PNG export of DC/RF/Velocity maps across every angle
of the currently open file.
Each channel's colorbar range is entered here and held fixed for every
exported angle (rather than auto-scaled per image, today's live-view
default) so the exported images are directly comparable to each other.
"""
# (ch_idx, is_velocity, label, colorbar unit, filename tag)
_ROWS = [
(CH1_IDX, False, CH_LABELS[CH1_IDX], _CHANNEL_DISPLAY[CH1_IDX][2],
CH_NAMES[CH1_IDX]),
(CH3_IDX, False, CH_LABELS[CH3_IDX], _CHANNEL_DISPLAY[CH3_IDX][2],
CH_NAMES[CH3_IDX]),
(CH4_IDX, False, CH_LABELS[CH4_IDX], _CHANNEL_DISPLAY[CH4_IDX][2],
CH_NAMES[CH4_IDX]),
(CH1_IDX, True, CH_LABELS[VELOCITY_MODE_IDX],
_CHANNEL_DISPLAY[VELOCITY_MODE_IDX][2], CH_NAMES[VELOCITY_MODE_IDX]),
]
# DC is cheap and precomputed on load; CH1/Velocity need a per-pixel FFT
# that can take minutes, so don't default to exporting them (same
# rationale as SrasViewerWindow._on_load_done's channel default).
_DEFAULT_CHECKED = {CH3_IDX, CH4_IDX}
def __init__(self, parent, *, default_dir: str, default_prefix: str,
default_ranges: dict[tuple[int, bool], tuple[float, float]]):
super().__init__(parent)
self.setWindowTitle("Batch Export Images")
self.setModal(True)
self.setMinimumWidth(460)
layout = QVBoxLayout(self)
# ---- Output ------------------------------------------------------
grp_out, ol = _group("Output")
dir_row = QHBoxLayout()
self._edit_dir = QLineEdit(default_dir)
btn_browse = QPushButton("Browse…")
btn_browse.clicked.connect(self._on_browse)
dir_row.addWidget(self._edit_dir)
dir_row.addWidget(btn_browse)
out_form = _form()
out_form.addRow("Folder:", dir_row)
self._edit_prefix = QLineEdit(default_prefix)
out_form.addRow("File prefix:", self._edit_prefix)
ol.addLayout(out_form)
layout.addWidget(grp_out)
# ---- Channels ------------------------------------------------------
grp_ch, cl = _group("Channels (fixed range, applied to every angle)")
grid = QGridLayout()
grid.setHorizontalSpacing(8)
grid.setVerticalSpacing(6)
grid.addWidget(_wrap_label("min:", _CSS_HINT), 0, 1)
grid.addWidget(_wrap_label("max:", _CSS_HINT), 0, 2)
self._rows: list[tuple[QCheckBox, QDoubleSpinBox, QDoubleSpinBox]] = []
for i, (ch_idx, is_velocity, label, unit, _tag) in enumerate(self._ROWS, 1):
chk = QCheckBox(label + (f" [{unit}]" if unit else ""))
chk.setChecked(not is_velocity and ch_idx in self._DEFAULT_CHECKED)
vmin, vmax = default_ranges.get((ch_idx, is_velocity), (0.0, 1.0))
spin_min, spin_max = QDoubleSpinBox(), QDoubleSpinBox()
for spin, val in ((spin_min, vmin), (spin_max, vmax)):
spin.setRange(-1e9, 1e9)
spin.setDecimals(4)
spin.setMinimumWidth(_SPIN_MIN_W)
spin.setValue(val)
grid.addWidget(chk, i, 0)
grid.addWidget(spin_min, i, 1)
grid.addWidget(spin_max, i, 2)
self._rows.append((chk, spin_min, spin_max))
cl.addLayout(grid)
layout.addWidget(grp_ch)
# ---- Buttons --------------------------------------------------
buttons = QDialogButtonBox()
buttons.addButton("Export", QDialogButtonBox.ButtonRole.AcceptRole
).clicked.connect(self.accept)
buttons.addButton("Cancel", QDialogButtonBox.ButtonRole.RejectRole
).clicked.connect(self.reject)
layout.addWidget(buttons)
def _on_browse(self):
d = QFileDialog.getExistingDirectory(
self, "Select Output Folder", self._edit_dir.text())
if d:
self._edit_dir.setText(d)
def accept(self):
"""Validate before closing — Cancel bypasses this entirely."""
if not self.get_prefix():
QMessageBox.warning(self, "Batch Export", "Enter a file prefix.")
return
if not self.get_output_dir():
QMessageBox.warning(self, "Batch Export", "Choose an output folder.")
return
selected = self.get_selected_channels()
if not selected:
QMessageBox.warning(
self, "Batch Export", "Select at least one channel to export.")
return
for ch in selected:
if not ch.vmin < ch.vmax:
QMessageBox.warning(
self, "Batch Export", f"{ch.label}: min must be less than max.")
return
super().accept()
def get_output_dir(self) -> str:
return self._edit_dir.text().strip()
def get_prefix(self) -> str:
return self._edit_prefix.text().strip()
def get_selected_channels(self) -> list[ExportChannel]:
result = []
for (chk, spin_min, spin_max), (ch_idx, is_velocity, label, unit, tag) in zip(
self._rows, self._ROWS):
if chk.isChecked():
result.append(ExportChannel(
ch_idx=ch_idx, is_velocity=is_velocity,
vmin=spin_min.value(), vmax=spin_max.value(),
label=label, unit=unit, tag=tag))
return result
# ---------------------------------------------------------------------------
# Export Fused ROI dialog
# ---------------------------------------------------------------------------
class FusedRoiExportDialog(QDialog):
"""Choose a value type and which angles to fuse for Export Fused ROI.
Every angle in the file is listed (a live AlignmentResult's per_angle
always covers every angle, and the raw-shared-grid path needs no per-
angle transform at all). Each checkbox is disabled — and auto-unchecked
— whenever *availability_fn(angle_idx, ch_idx)* is False for the
currently selected value type; switching the value-type radio
re-evaluates every checkbox live, since availability is per (angle,
value type) rather than just per angle — e.g. DC may be ready
everywhere while FFT is ready nowhere.
"""
_VALUE_MODES = (CH1_IDX, CH3_IDX, CH4_IDX, VELOCITY_MODE_IDX)
def __init__(self, parent=None, *,
angles: list[tuple[int, float]],
availability_fn,
default_ch_idx: int,
out_dir: str,
stem: str,
grid_note: str):
super().__init__(parent)
self.setWindowTitle("Export Fused ROI")
self.setModal(True)
self.setMinimumWidth(420)
self._availability_fn = availability_fn
self._out_dir = out_dir
self._stem = stem
self._path_user_chosen = False
layout = QVBoxLayout(self)
layout.addWidget(_wrap_label(grid_note, _CSS_HINT))
# ---- Value type --------------------------------------------------
grp_val, vl = _group("Value to Export")
self._val_group = QButtonGroup(self)
self._val_buttons: dict[int, QRadioButton] = {}
for ch_idx, label in zip(self._VALUE_MODES, CH_LABELS):
rb = QRadioButton(label)
self._val_group.addButton(rb, id=ch_idx)
self._val_buttons[ch_idx] = rb
vl.addWidget(rb)
self._val_buttons[default_ch_idx].setChecked(True)
self._val_group.idClicked.connect(self._on_value_type_changed)
layout.addWidget(grp_val)
# ---- Angles --------------------------------------------------
grp_ang, al = _group("Angles to Include")
sel_row = QHBoxLayout()
btn_all = QPushButton("Select All Available")
btn_none = QPushButton("Select None")
btn_all.clicked.connect(self._on_select_all_available)
btn_none.clicked.connect(self._on_select_none)
sel_row.addWidget(btn_all)
sel_row.addWidget(btn_none)
al.addLayout(sel_row)
scroll_inner = QWidget()
scroll_layout = QVBoxLayout(scroll_inner)
self._angle_checks: dict[int, QCheckBox] = {}
for angle_idx, angle_deg in angles:
cb = QCheckBox(f"{angle_deg:.1f}° (angle {angle_idx})")
self._angle_checks[angle_idx] = cb
cb.toggled.connect(self._update_accept_enabled)
scroll_layout.addWidget(cb)
scroll = QScrollArea()
scroll.setWidget(scroll_inner)
scroll.setWidgetResizable(True)
scroll.setMaximumHeight(220)
al.addWidget(scroll)
self._lbl_none_available = _wrap_label("", _CSS_WARN)
al.addWidget(self._lbl_none_available)
layout.addWidget(grp_ang)
# ---- Output path --------------------------------------------------
grp_out, ol = _group("Output File")
path_row = QHBoxLayout()
self._edit_path = QLineEdit()
self._edit_path.setReadOnly(True)
path_row.addWidget(self._edit_path, 1)
btn_browse = QPushButton("Browse…")
btn_browse.clicked.connect(self._on_browse)
path_row.addWidget(btn_browse)
ol.addLayout(path_row)
layout.addWidget(grp_out)
# ---- Buttons -----------------------------------------------------
buttons = QDialogButtonBox()
self._btn_export = buttons.addButton(
"Export", QDialogButtonBox.ButtonRole.AcceptRole)
self._btn_export.clicked.connect(self.accept)
buttons.addButton("Cancel", QDialogButtonBox.ButtonRole.RejectRole
).clicked.connect(self.reject)
layout.addWidget(buttons)
self._refresh_default_path()
self._apply_availability()
# ---- internals -----------------------------------------------------
def _current_ch_idx(self) -> int:
return self._val_group.checkedId()
def _apply_availability(self):
ch_idx = self._current_ch_idx()
n_ok = 0
for angle_idx, cb in self._angle_checks.items():
ok = self._availability_fn(angle_idx, ch_idx)
cb.setEnabled(ok)
if ok:
n_ok += 1
cb.setToolTip("")
else:
cb.setChecked(False)
cb.setToolTip(
f"No cached/stored {CH_LABELS[ch_idx]} data for this "
"angle yet — view it in the main window (or run "
"Batch Compute) first.")
self._lbl_none_available.setText(
"" if n_ok else "No angle has this value type ready yet.")
self._update_accept_enabled()
def _on_value_type_changed(self, _id: int):
self._apply_availability()
if not self._path_user_chosen:
self._refresh_default_path()
def _on_select_all_available(self):
for cb in self._angle_checks.values():
if cb.isEnabled():
cb.setChecked(True)
def _on_select_none(self):
for cb in self._angle_checks.values():
cb.setChecked(False)
def _refresh_default_path(self):
ch_idx = self._current_ch_idx()
name = f"{self._stem}_fused_roi_{CH_NAMES[ch_idx]}.csv"
self._edit_path.setText(str(Path(self._out_dir) / name))
self._update_accept_enabled()
def _on_browse(self):
path, _ = QFileDialog.getSaveFileName(
self, "Export Fused ROI as CSV", self._edit_path.text(),
"CSV files (*.csv);;All files (*)")
if path:
self._edit_path.setText(path)
self._path_user_chosen = True
self._update_accept_enabled()
def _update_accept_enabled(self):
any_checked = any(cb.isChecked() for cb in self._angle_checks.values())
self._btn_export.setEnabled(any_checked and bool(self._edit_path.text()))
# ---- getters ---------------------------------------------------------
def get_ch_idx(self) -> int:
return self._current_ch_idx()
def get_selected_angles(self) -> list[int]:
return sorted(a for a, cb in self._angle_checks.items() if cb.isChecked())
def get_output_path(self) -> str:
return self._edit_path.text()
File diff suppressed because it is too large Load Diff
-3
View File
@@ -1,3 +0,0 @@
PyQt6==6.10.2
numpy==2.4.1
matplotlib==3.10.8
+703
View File
@@ -0,0 +1,703 @@
#!/usr/bin/env python3
"""Background workers for the SRAS viewer.
Every worker is a plain QObject moved onto its own QThread by
SrasViewerWindow._run_worker, exposing signals only. Workers must never touch
GUI-thread-owned state (the display caches in particular) — they take
everything they need through their constructor and hand results back by signal.
"""
import os
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, as_completed
from concurrent.futures.process import BrokenProcessPool
from dataclasses import dataclass
from pathlib import Path
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure
from PyQt6.QtCore import QObject, pyqtSignal
import sras_compute as compute
from sras_align_export import write_aligned_sras
from sras_compute import cache_file, compute_rf_image, dc_image_mv
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, SrasFile
from sras_render import export_view_image, sanitize_figsize
# Concurrency caps. Batch conversion runs one process per file, and each of
# those processes threads internally, so the two must be divided rather than
# both set to the core count. Files also commonly sit on one external drive,
# where a dozen concurrent readers is slower than a few — hence the low
# default, overridable from the environment.
_BATCH_MAX_PROCS = int(os.environ.get("SRAS_BATCH_PROCS", 0)) or min(
4, os.cpu_count() or 2)
# Spawning a pool costs roughly a second of interpreter startup (each child
# re-imports the entry module). That is noise against a multi-GB scan but
# dominates a batch of small files, where it would make the job *slower* —
# so below this total size the batch just runs in the worker thread.
_BATCH_POOL_MIN_BYTES = int(os.environ.get("SRAS_BATCH_POOL_MIN_MB", 512)) * 1024 * 1024
class CancellableWorker(QObject):
"""A worker whose compute polls stop() between row chunks.
Without this a shutdown has to wait out whatever is in flight, and on a
large scan a single angle is ~40 s — far too long to block closing the
window. Chunk-level polling bounds the wait to one chunk instead.
"""
def __init__(self):
super().__init__()
self._stop = False
def stop(self):
self._stop = True
def _stopped(self) -> bool:
return self._stop
class _PooledWorker(CancellableWorker):
"""Fans a per-item computation across a thread pool, emitting each result
from this worker's own thread as it lands (never from a pool thread).
Subclasses provide _plan() -> n_workers (stashing whatever per-run
context they need), _items(), _one(item) -> result, and _emit(result).
On stop(): queued items are dropped, in-flight ones are not waited for —
that is what keeps closing the window responsive on a large scan.
"""
finished = pyqtSignal()
error = pyqtSignal(str)
def run(self):
try:
pool = ThreadPoolExecutor(max_workers=max(1, self._plan()))
try:
futures = [pool.submit(self._one, it) for it in self._items()]
for fut in as_completed(futures):
if self._stop:
break
self._emit(fut.result())
finally:
pool.shutdown(wait=not self._stop, cancel_futures=True)
self.finished.emit()
except Exception as exc:
self.error.emit(str(exc))
class LoadWorker(QObject):
finished = pyqtSignal(object) # SrasFile | None
error = pyqtSignal(str)
def __init__(self, path: str):
super().__init__()
self._path = path
def run(self):
try:
self.finished.emit(SrasFile(self._path))
except Exception as exc:
self.error.emit(str(exc))
self.finished.emit(None)
class ComputeWorker(CancellableWorker):
"""Computes one displayable image for (angle, channel).
For CH1/Velocity (FFT-derived) channels, the FFT is only run for pixels
whose DC4 (Bias B) mean is at or above dc_threshold_mv — masked pixels are
left at 0 MHz without ever being FFT'd, since that's the expensive part of
a scan. If the DC4 image for this angle is already known, pass it in as
*dc4_mv* to skip re-reading the CH4 channel from disk entirely.
Emits a plain ``np.ndarray`` already in display units.
"""
finished = pyqtSignal(object)
error = pyqtSignal(str)
def __init__(self, sras: SrasFile, angle_idx: int, ch_idx: int,
apply_bg_sub: bool = True, n_fft: int | None = None,
dc_threshold_mv: float = 0.0,
dc4_mv: np.ndarray | None = None,
is_fft_mode: bool = False,
row_avg_n: int = 0,
min_freq_mhz: float = 0.0):
super().__init__()
self._sras = sras
self._angle = angle_idx
self._ch = ch_idx
self._apply_bg_sub = apply_bg_sub
self._n_fft = n_fft
self._dc_threshold = dc_threshold_mv
self._dc4_mv = dc4_mv
self._is_fft_mode = is_fft_mode
self._row_avg_n = row_avg_n
self._min_freq_mhz = min_freq_mhz
def run(self):
try:
if self._is_fft_mode:
img = compute_rf_image(
self._sras, self._angle, dc_threshold_mv=self._dc_threshold,
apply_bg_sub=self._apply_bg_sub, n_fft=self._n_fft,
dc4_mv=self._dc4_mv, should_stop=self._stopped,
row_avg_n=self._row_avg_n,
min_freq_mhz=self._min_freq_mhz)
else:
img = dc_image_mv(self._sras, self._angle, self._ch,
should_stop=self._stopped)
# On cancellation the image is only partly filled, so hand back
# None rather than something that would be cached as real. The
# signal still fires either way — it is what quits the thread.
self.finished.emit(None if self._stop else img)
except Exception as exc:
self.error.emit(str(exc))
class DcPrecomputeWorker(_PooledWorker):
"""Computes CH3/CH4 DC images for every angle in the background.
DC images are cheap (a per-waveform mean, no FFT) compared to the
CH1/Velocity FFT, so precomputing them for the whole file right after load
makes switching angles instant while on a DC channel, and also means the
FFT masking step (which needs a DC4 image) rarely has to wait on anything.
"""
angle_done = pyqtSignal(int, np.ndarray, np.ndarray) # angle_idx, dc3_mv, dc4_mv
def __init__(self, sras: SrasFile):
super().__init__()
self._sras = sras
self._angle_budget = 0
def _plan(self) -> int:
n_workers, self._angle_budget = compute.plan_angle_level(self._sras)
return n_workers
def _items(self):
return range(self._sras.n_angles)
def _one(self, a: int) -> tuple[int, np.ndarray, np.ndarray]:
# max_workers=1 *and* a budget share: this call is one of several
# concurrent angles, and both the thread count and the buffer size
# have to be divided (see compute.plan_angle_level).
kw = dict(max_workers=1, budget=self._angle_budget,
should_stop=self._stopped)
return (a,
dc_image_mv(self._sras, a, CH3_IDX, **kw),
dc_image_mv(self._sras, a, CH4_IDX, **kw))
def _emit(self, result):
self.angle_done.emit(*result)
class BatchCacheWorker(QObject):
"""Batch-computes and stores DC or FFT images into each of *paths*'s v7
CACH tail, in place — converting v6 sources to v7 on first use, or updating
an existing v7 file's cache blocks without disturbing whatever the other
block already holds.
*mode* is ``"dc"`` (CH3/CH4 mean images), ``"fft"`` (CH1 peak-frequency
images, unmasked — masking is applied at display time, same as v5's PREC
convention), or ``"fft_rowavg"`` (same-row, distance-weighted CH1
averaging before the FFT — needs *dc_threshold_mv* and a positive
*row_avg_n*; see ``sras_compute.cache_file``).
Both FFT modes cache at *pad_factor*, which the caller sets from the
viewer's own padding — a cache stored at a pad the user is not viewing
at is one the display can never use. *min_freq_mhz* travels the same
way: the viewer's live min-peak-freq floor, recorded in the store as
provenance so a reader knows which bins the peak search considered.
Files are processed one per subprocess: they are fully independent, each
opens its own memmap and writes only its own bytes, and only path strings
and scalars cross the process boundary. Emits ``progress(int)`` (0–100 by
files completed), ``file_done(str, str)`` (path, error message or "") so
one file's failure doesn't abort the batch, and ``finished()``.
"""
progress = pyqtSignal(int)
file_done = pyqtSignal(str, str)
finished = pyqtSignal()
def __init__(self, paths: list[str], mode: str, apply_bg_sub: bool,
dc_threshold_mv: float | None = None, row_avg_n: int = 0,
pad_factor: int = 1, min_freq_mhz: float = 0.0):
super().__init__()
self._paths = paths
self._mode = mode
self._apply_bg_sub = apply_bg_sub
self._dc_threshold = dc_threshold_mv
self._row_avg_n = row_avg_n
self._pad_factor = pad_factor
self._min_freq_mhz = min_freq_mhz
def _report(self, path: str, err: str, done: int, total: int):
self.file_done.emit(path, err)
self.progress.emit(int(done / max(1, total) * 100))
def _run_pooled(self, paths: list[str], n_procs: int) -> list[str]:
"""Process the batch across *n_procs* subprocesses. Returns the paths
that never got a real answer because the pool itself died, so the
caller can retry them in-process.
Under spawn each child re-imports the entry module, so the batch must
survive that going wrong (an unguarded __main__, a frozen build, a
sandbox that forbids subprocesses) rather than reporting every file as
failed — hence the retry list instead of a per-file error.
"""
# Each child threads internally; divide the machine rather than
# letting every process claim every core.
per_proc_workers = max(1, (os.cpu_count() or 4) // n_procs)
unresolved: list[str] = []
done = 0
with ProcessPoolExecutor(max_workers=n_procs) as executor:
futures = {
executor.submit(cache_file, p, self._mode, self._apply_bg_sub,
per_proc_workers,
pad_factor=self._pad_factor,
dc_threshold_mv=self._dc_threshold,
row_avg_n=self._row_avg_n,
min_freq_mhz=self._min_freq_mhz): p
for p in paths
}
for fut in as_completed(futures):
path = futures[fut]
try:
err = fut.result()
except BrokenProcessPool:
unresolved.append(path)
continue
except Exception as exc:
err = str(exc)
done += 1
self._report(path, err, done, len(paths))
return unresolved
def _run_inline(self, paths: list[str], done: int, total: int):
"""Fallback / single-file path: compute in this thread. Still uses the
full core count internally, since nothing else is competing."""
for path in paths:
try:
err = cache_file(path, self._mode, self._apply_bg_sub,
compute.default_max_workers(),
pad_factor=self._pad_factor,
dc_threshold_mv=self._dc_threshold,
row_avg_n=self._row_avg_n,
min_freq_mhz=self._min_freq_mhz)
except Exception as exc:
err = str(exc)
done += 1
self._report(path, err, done, total)
def _worth_pooling(self, paths: list[str]) -> bool:
if len(paths) < 2:
return False
total = 0
for p in paths:
try:
total += os.path.getsize(p)
except OSError:
pass # unreadable files are reported by cache_file
return total >= _BATCH_POOL_MIN_BYTES
def run(self):
paths = self._paths
n_procs = max(1, min(_BATCH_MAX_PROCS, len(paths)))
if not self._worth_pooling(paths):
self._run_inline(paths, 0, len(paths))
self.finished.emit()
return
try:
unresolved = self._run_pooled(paths, n_procs)
except Exception:
# The pool could not be created or collapsed wholesale.
unresolved = list(paths)
if unresolved:
self._run_inline(unresolved, len(paths) - len(unresolved), len(paths))
self.finished.emit()
@dataclass
class ExportChannel:
"""One row of a batch image export: which raw channel to read, whether
to apply the Velocity post-multiply, and the fixed display range/labels
to render it with.
Kept free of sras_viewer's display constants (CH_LABELS, VELOCITY_MODE_IDX,
etc.) so BatchExportWorker has no dependency on the GUI module — the
caller resolves labels/units/filename tags once, up front.
"""
ch_idx: int # CH1_IDX, CH3_IDX, or CH4_IDX -- which raw data to read
is_velocity: bool # True only for the derived Velocity map (post-multiply of CH1 freq)
vmin: float
vmax: float
label: str # e.g. "CH3 -- Bias A (DC mean)"
unit: str # colorbar units, e.g. "mV"
tag: str # filename tag: "CH1", "CH3", "CH4", "VEL"
def _render_map_png(img: np.ndarray, extent: list[float], *, cmap: str,
vmin: float, vmax: float, title: str, colorbar_label: str,
out_path: str, figsize: tuple[float, float] | None = None):
"""Render one map image to *out_path* with a fixed vmin/vmax, using a
headless Agg canvas so this never touches the GUI thread's interactive
matplotlib backend. Layout mirrors ImageCanvas.show_image.
*figsize* is the live canvas's size in inches, so the PNG comes out at
the shape the view was being read at instead of a fixed 7x5 -- with
aspect="auto" below, the figure box is what sets the map's proportions.
None falls back to sras_render.DEFAULT_FIGSIZE."""
fig = Figure(figsize=sanitize_figsize(figsize), tight_layout=True)
FigureCanvasAgg(fig)
ax = fig.add_subplot(111)
im = ax.imshow(img, aspect="auto", origin="upper", extent=extent,
cmap=cmap, vmin=vmin, vmax=vmax, interpolation="nearest")
cb = fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04)
if colorbar_label:
cb.set_label(colorbar_label)
ax.set_xlabel("X (mm)")
ax.set_ylabel("Y (mm)")
ax.set_title(title)
fig.savefig(out_path, dpi=150)
class BatchExportWorker(CancellableWorker):
"""Renders and saves one PNG per (angle, selected channel) for an
already-open SrasFile, with each channel's vmin/vmax held fixed across
every angle so the colorbar is directly comparable image to image.
Takes the SrasFile object directly (like ComputeWorker/DcPrecomputeWorker)
rather than a path -- this runs against the file already open in the GUI,
not an arbitrary batch of files, so there is no need to reopen it in a
subprocess the way BatchCacheWorker does.
Every setting that decides what an FFT-derived pixel *is* has to arrive
here explicitly, because compute_rf_image defaults each one to "off" and
an omitted argument is therefore not a no-op -- it silently exports a
different image than the one on screen. *min_freq_mhz* is the one with
teeth: dropping it does not just skip a mask, it also makes this file's
stored cache (written by Batch Compute FFT at some earlier floor) look
like a match, so the export hands back the pre-floor peaks the user
raised the floor to get rid of. It mirrors
SrasViewerWindow._stored_fft_image, which takes the floor live for the
same reason -- re-masking a stored image against a *higher* floor is
free, so the displayed map always honors the spin box.
Emits progress(int) (0-100 over all angle x channel pairs), file_done(str,
str) (output path, error message or ""), and finished() -- same shape as
BatchCacheWorker.
"""
progress = pyqtSignal(int)
file_done = pyqtSignal(str, str)
finished = pyqtSignal()
def __init__(self, sras: SrasFile, channels: list[ExportChannel],
output_dir: str, prefix: str, *, cmap: str,
apply_bg_sub: bool, dc_threshold_mv: float,
n_fft: int | None, grating_um: float,
min_freq_mhz: float = 0.0,
figsize: tuple[float, float] | None = None):
super().__init__()
self._sras = sras
self._channels = channels
self._output_dir = Path(output_dir)
self._prefix = prefix
self._cmap = cmap
self._apply_bg_sub = apply_bg_sub
self._dc_threshold_mv = dc_threshold_mv
self._n_fft = n_fft
self._grating_um = grating_um
self._min_freq_mhz = min_freq_mhz
self._figsize = figsize
def _angle_extent(self, angle_idx: int) -> list[float]:
s = self._sras
x_axis = s.x_axis_mm(angle_idx)
y_axis = s.y_positions_mm(angle_idx)
dx = x_axis[1] - x_axis[0] if len(x_axis) > 1 else s.pixel_x_mm
dy = float(y_axis[1] - y_axis[0]) if len(y_axis) > 1 else 1.0
return [x_axis[0] - dx / 2, x_axis[-1] + dx / 2,
y_axis[-1] + dy / 2, y_axis[0] - dy / 2]
def run(self):
try:
s = self._sras
n_angles = s.n_angles
total = n_angles * len(self._channels)
done = 0
needs_freq = any(c.ch_idx == CH1_IDX for c in self._channels)
for angle_idx in range(n_angles):
if self._stop:
break
extent = self._angle_extent(angle_idx)
angle_deg = s.angles_deg[angle_idx]
freq_mhz = None
if needs_freq:
freq_mhz = compute_rf_image(
s, angle_idx, dc_threshold_mv=self._dc_threshold_mv,
apply_bg_sub=self._apply_bg_sub, n_fft=self._n_fft,
should_stop=self._stopped, row_avg_n=s.precomputed_row_avg_n,
min_freq_mhz=self._min_freq_mhz)
for channel in self._channels:
if self._stop:
break
out_path = (self._output_dir
/ f"{self._prefix}_angle{angle_idx:02d}_{channel.tag}.png")
try:
if channel.ch_idx == CH1_IDX:
img = (freq_mhz * self._grating_um if channel.is_velocity
else freq_mhz)
else:
img = dc_image_mv(s, angle_idx, channel.ch_idx,
should_stop=self._stopped)
title = f"{channel.tag} | {angle_deg:.1f}°"
colorbar_label = (f"{channel.label} ({channel.unit})"
if channel.unit else channel.label)
_render_map_png(
img, extent, cmap=self._cmap,
vmin=channel.vmin, vmax=channel.vmax,
title=title, colorbar_label=colorbar_label,
out_path=str(out_path), figsize=self._figsize)
self.file_done.emit(str(out_path), "")
except Exception as exc:
self.file_done.emit(str(out_path), str(exc))
done += 1
self.progress.emit(int(done / max(1, total) * 100))
self.finished.emit()
except Exception as exc:
self.file_done.emit("", str(exc))
self.finished.emit()
class BatchExportImagesWorker(QObject):
"""Renders the view settings captured by the caller at trigger time
(angle/channel/threshold/etc.) to one PNG per file in *paths*, via
sras_render.export_view_image.
Same process-pool-with-inline-fallback strategy as BatchCacheWorker
above (same _BATCH_MAX_PROCS / _BATCH_POOL_MIN_BYTES thresholds): an
FFT-derived (CH1/Velocity) view is exactly the same expensive per-file
compute Batch Compute FFT already parallelizes this way. Never mutates
*paths* — each file is opened read-only — so unlike BatchCacheWorker
there is no version gate.
Emits progress(int) (0-100 by files completed), file_done(str, str, str)
(path, error message or "", output filename this file targeted — set
even on most failures so the caller can flag same-stem collisions across
the batch without any cross-process bookkeeping), and finished().
"""
progress = pyqtSignal(int)
file_done = pyqtSignal(str, str, str)
finished = pyqtSignal()
def __init__(self, paths: list[str], **render_kwargs):
"""*render_kwargs* is exactly export_view_image's keyword-only
settings (out_dir, angle_idx, ch_idx, is_fft_mode, is_velocity,
dc_threshold_mv, apply_bg_sub, pad_factor, min_freq_mhz, grating_um,
cmap, auto_scale, vmin, vmax, highlight_masked, mode_str,
colorbar_label, mask_color) — bundled rather than repeated as
positional params across __init__/_run_pooled/_run_inline."""
super().__init__()
self._paths = paths
self._kw = render_kwargs
def _report(self, path: str, err: str, out_name: str, done: int, total: int):
self.file_done.emit(path, err, out_name)
self.progress.emit(int(done / max(1, total) * 100))
def _run_pooled(self, paths: list[str], n_procs: int) -> list[str]:
"""Same contract as BatchCacheWorker._run_pooled: returns the paths
that never got a real answer because the pool itself died, so the
caller can retry them in-process."""
per_proc_workers = max(1, (os.cpu_count() or 4) // n_procs)
unresolved: list[str] = []
done = 0
with ProcessPoolExecutor(max_workers=n_procs) as executor:
futures = {
executor.submit(export_view_image, p,
max_workers=per_proc_workers, **self._kw): p
for p in paths
}
for fut in as_completed(futures):
path = futures[fut]
try:
err, out_name = fut.result()
except BrokenProcessPool:
unresolved.append(path)
continue
except Exception as exc:
err, out_name = str(exc), ""
done += 1
self._report(path, err, out_name, done, len(paths))
return unresolved
def _run_inline(self, paths: list[str], done: int, total: int):
for path in paths:
try:
err, out_name = export_view_image(
path, max_workers=compute.default_max_workers(), **self._kw)
except Exception as exc:
err, out_name = str(exc), ""
done += 1
self._report(path, err, out_name, done, total)
def _worth_pooling(self, paths: list[str]) -> bool:
if len(paths) < 2:
return False
total = 0
for p in paths:
try:
total += os.path.getsize(p)
except OSError:
pass # unreadable files are reported by export_view_image
return total >= _BATCH_POOL_MIN_BYTES
def run(self):
paths = self._paths
n_procs = max(1, min(_BATCH_MAX_PROCS, len(paths)))
if not self._worth_pooling(paths):
self._run_inline(paths, 0, len(paths))
self.finished.emit()
return
try:
unresolved = self._run_pooled(paths, n_procs)
except Exception:
# The pool could not be created or collapsed wholesale.
unresolved = list(paths)
if unresolved:
self._run_inline(unresolved, len(paths) - len(unresolved), len(paths))
self.finished.emit()
class Ch4MaskWorker(_PooledWorker):
"""Fetches each requested angle's CH4 (Bias B) DC image in mV, for the
alignment wizard's initial threshold-mask stack.
Reuses dc_image_mv, which prefers a stored v5/v7 cache over recomputing
from raw waveforms, so this only does real work for a file that hasn't
gone through the v7 "Convert" batch step and for angles the main
window's own DcPrecomputeWorker (which runs automatically right after
every file load) hasn't reached yet. In the common case — the user opens
Fusion -> Manual Alignment after DC precompute has already finished —
*angle_indices* is empty and this worker is never even constructed (see
CorrelatePage._start_mask_prep).
"""
angle_done = pyqtSignal(int, np.ndarray) # angle_idx, dc4_mv
def __init__(self, sras: SrasFile, angle_indices: list[int]):
super().__init__()
self._sras = sras
self._angles = angle_indices
self._budget = 0
def _plan(self) -> int:
n_workers, self._budget = compute.plan_angle_level(self._sras)
return n_workers
def _items(self):
return self._angles
def _one(self, a: int) -> tuple[int, np.ndarray]:
return a, dc_image_mv(self._sras, a, CH4_IDX,
max_workers=1, budget=self._budget)
def _emit(self, result):
self.angle_done.emit(*result)
class CrossCorrelateWorker(_PooledWorker):
"""Rigid registration (rotation + translation, never scale) of each of
*angle_indices* against *ref_angle_idx*, for the alignment wizard's
Run/Re-run Correlation button.
Runs on a background thread — registering a real many-angle,
high-resolution scan takes long enough that doing it on the GUI thread
would visibly freeze the dialog. Rotation is *searched*, not taken from the
stage's reported angle: see compute.register_angle_to_reference, which
seeds from that angle but scores both of its signs and refines from there.
dc4_mv is the dialog's own already-in-memory per-angle CH4 image — this
worker does no fetching of its own.
"""
# angle_idx, rotation_deg, shift_x_mm, shift_y_mm, score, source
angle_done = pyqtSignal(int, float, float, float, float, str)
def __init__(self, sras: SrasFile, ref_angle_idx: int, angle_indices: list[int],
dc4_mv: dict[int, np.ndarray], *, reg_kwargs: dict | None = None):
"""*reg_kwargs* is splatted into register_angle_to_reference — every
registration setting the wizard exposes (sources, threshold, search
width, seed, signs, refine, grid sizes) travels in it, so this class
holds no opinion about which knobs exist and exposing another needs no
change here."""
super().__init__()
self._sras = sras
self._ref = ref_angle_idx
self._angles = angle_indices
self._dc4_mv = dc4_mv
self._reg_kwargs = dict(reg_kwargs or {})
def _plan(self) -> int:
return compute.registration_workers(self._sras)
def _items(self):
return self._angles
def _one(self, a: int) -> tuple[int, compute.RigidFit]:
return a, compute.register_angle_to_reference(
self._sras, a, self._ref, self._dc4_mv, **self._reg_kwargs)
def _emit(self, result):
a, fit = result
self.angle_done.emit(a, fit.rotation_deg, fit.shift_mm[0],
fit.shift_mm[1], fit.score, fit.source)
class AlignedExportWorker(CancellableWorker):
"""Writes the aligned, cropped .sras on a background thread.
Unlike every other worker here this one produces a *file*, which changes
what cancellation has to mean: write_aligned_sras stages into a ".part"
sibling and removes it when should_stop() fires, so a cancelled or crashed
export leaves nothing behind. That matters more than it sounds — a
truncated .sras is not detectably broken, since the v6 parser reads a short
file as an aborted scan and opens it happily.
Cancellation is polled per output row chunk, the same granularity
CancellableWorker's docstring justifies, so closing the window never waits
on a multi-gigabyte write.
"""
progress = pyqtSignal(int) # 0-100
finished = pyqtSignal(str, str) # written path ("" = none), error
def __init__(self, sras: SrasFile, result, out_path: str):
super().__init__()
self._sras = sras
self._result = result
self._out_path = out_path
def run(self):
try:
written = write_aligned_sras(
self._sras, self._result, self._out_path,
progress_cb=self.progress.emit, should_stop=self._stopped)
if self._stopped():
self.finished.emit("", "") # cancelled: no file, no error
else:
self.finished.emit(str(written), "")
except Exception as exc:
self.finished.emit("", str(exc))
+16
View File
@@ -0,0 +1,16 @@
"""Shared test setup: repo-root imports, the offscreen Qt platform, and
hermetic QSettings (tests must not read or write the user's real viewer
settings)."""
import os
import sys
import tempfile
from pathlib import Path
os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from PyQt6.QtCore import QSettings # noqa: E402
QSettings.setPath(QSettings.Format.IniFormat, QSettings.Scope.UserScope,
tempfile.mkdtemp(prefix="sras_qsettings_"))
+646
View File
@@ -0,0 +1,646 @@
"""Aligned/cropped .sras export: does the written file actually hold the
alignment the viewer showed?
The export is the one place an alignment stops being a transform applied on the
fly and becomes bytes on disk, so these tests care about two things above all:
the file's geometry describes what was written, and the pixels in it are the
same pixels apply_alignment would have drawn. The strongest check is the
round-trip — register the exported file against itself and demand identity,
which no amount of self-consistent-but-wrong index math can fake.
No Qt: this exercises sras_align_export and sras_compute directly.
"""
import struct
import numpy as np
import pytest
import sras_align_export as export
import sras_compute as compute
from sras_format import CH3_IDX, CH4_IDX, HDR_SIZE_V6, SrasFile, adc_to_mv, mv_to_adc
import tools.make_test_sras as gen
_THRESHOLD_MV = 80.0
# Same reasoning as tests/test_alignment.py: a quarter degree is already
# sub-pixel for this sample at the registration pitch.
_ROT_TOL_DEG = 0.5
_SHIFT_TOL_MM = 0.02
def dc_mv(sras: SrasFile, angle_idx: int, ch: int = CH4_IDX) -> np.ndarray:
return adc_to_mv(compute.compute_dc_image(sras, angle_idx, ch), *sras.cal(ch))
@pytest.fixture(scope="module")
def rig(tmp_path_factory):
"""The rotating-sample scan, its truth alignment, and its export."""
tmpdir = tmp_path_factory.mktemp("sras_export")
src_path = tmpdir / "rotating.sras"
meta = gen.write_rotating(src_path, n_angles=4)
sras = SrasFile(str(src_path))
params = {a: compute.ManualAngleParams(rot, shift)
for a, (rot, shift) in meta["truth"].items()}
result = compute.build_manual_alignment(sras, 0, _THRESHOLD_MV, params)
out_path = tmpdir / "rotating_aligned.sras"
export.write_aligned_sras(sras, result, out_path)
return type("Rig", (), dict(
tmpdir=tmpdir, src_path=src_path, sras=sras, meta=meta,
result=result, out_path=out_path, out=SrasFile(str(out_path))))
# ---------------------------------------------------------------------------
# Geometry and file structure
# ---------------------------------------------------------------------------
def test_output_is_v6_with_uniform_geometry(rig):
out, result = rig.out, rig.result
n_rows, n_cols = result.canvas_shape
assert out.version == 6
assert out.n_angles == rig.sras.n_angles
assert set(out.n_rows) == {n_rows}, "every angle must share the canvas rows"
assert set(out.n_frames) == {n_cols}, "every angle must share the canvas frames"
assert np.allclose(out.x_start_mm, result.canvas_origin_mm[0])
# x_delta must stay velocity/laser_freq or x_axis_mm() contradicts the
# geometry table; the canvas pitch is the reference angle's own pitch, so
# this is exact rather than approximate.
assert np.allclose(out.x_delta_mm_per_angle, rig.sras.pixel_x_mm)
assert out.pixel_x_mm == pytest.approx(rig.sras.pixel_x_mm)
def test_row_table_matches_the_canvas(rig):
expected = (rig.result.canvas_origin_mm[1]
+ np.arange(rig.result.canvas_shape[0]) * rig.result.canvas_dy_mm)
for a in range(rig.out.n_angles):
assert rig.out.y_positions_mm(a) == pytest.approx(expected, abs=1e-4)
def test_angle_table_and_calibration_round_trip(rig):
assert rig.out.angles_deg == pytest.approx(rig.sras.angles_deg)
for ch in range(rig.sras.n_channels):
assert rig.out.cal(ch) == pytest.approx(rig.sras.cal(ch))
assert rig.out.samples_per_frame == rig.sras.samples_per_frame
assert rig.out.bytes_per_sample == rig.sras.bytes_per_sample
assert rig.out.n_channels == rig.sras.n_channels
assert rig.out.background == pytest.approx(rig.sras.background)
def test_no_cache_tail(rig):
"""File ends exactly at the waveform data — nothing trailing.
A stale cache tail would be indexed by the *input's* grid, so the export
must not carry one; asserting on the exact file size is what proves it,
since a v7 tail would simply be ignored by a v6 parser.
"""
end = max(off + n for _, off, n in rig.out.iter_angle_blocks())
assert rig.out_path.stat().st_size == end
assert all(img is None for img in rig.out.precomputed_dc4_mv)
def test_declared_header_size_is_v6(rig):
raw = rig.out_path.read_bytes()[:HDR_SIZE_V6]
magic, version, n_angles = struct.unpack(">4sBH", raw[:7])
assert (magic, version, n_angles) == (b"SRAS", 6, rig.sras.n_angles)
# ---------------------------------------------------------------------------
# The pixels themselves
# ---------------------------------------------------------------------------
def test_export_matches_apply_alignment(rig):
"""The exported waveforms decode to the same DC image the viewer drew —
over the *whole* canvas, padding included.
Two rules have to be exactly right for this, and each fails differently:
* rounding must be floor(x + 0.5), not np.rint, or pixels on exact
half-integer boundaries pick the neighbouring source pixel;
* out-of-bounds must be tested on the fractional coordinate against
[0, n-1], not on the rounded index, or a one-pixel rim gets real data
where the preview shows padding.
Comparing every pixel rather than only the interior is what catches the
second one, since a rim discrepancy hides inside a `preview != 0` mask.
"""
for a in range(rig.sras.n_angles):
preview = compute.apply_alignment(rig.result, a, dc_mv(rig.sras, a))
actual = dc_mv(rig.out, a)
assert actual.shape == preview.shape
# Padding matches to within half an ADC step: apply_alignment pads with
# literal 0.0 mV, the export with the nearest integer ADC code to 0 mV.
tol = abs(rig.sras.cal(CH4_IDX)[0]) / 2.0 + 1e-4
# Exclude the epsilon rim the export deliberately keeps and scipy drops
# (see test_edge_tolerance_only_affects_the_epsilon_rim).
sr, sc = export._src_coords(rig.result.per_angle[a],
np.arange(preview.shape[0]), preview.shape[1])
rim = export._in_bounds(sr, sc, *rig.sras.image_shape(a)) & (preview == 0.0)
cmp = ~rim
assert actual[cmp] == pytest.approx(preview[cmp], abs=tol), \
f"angle {a}: exported pixels differ from the aligned preview"
# And exactly, wherever there is real data.
inside = (preview != 0.0)
assert inside.any(), f"angle {a}: preview is entirely padding"
assert actual[inside] == pytest.approx(preview[inside], abs=1e-6), \
f"angle {a}: exported data pixels are not bit-equal to the preview"
def test_reference_angle_is_exported_whole(rig):
"""The reference angle must survive as a complete, exact integer crop.
It is the coordinate authority — its transform is the identity with an
integer offset by construction — so every one of its source pixels has to
appear in the export. This is what _EDGE_TOL exists for: that offset comes
out of the mm-space affine chain as -20 - 7e-15, and a bare `>= 0` bounds
test silently drops the angle's entire first row and last column.
"""
ref = rig.result.ref_angle_idx
src_rows, src_frames = rig.sras.image_shape(ref)
plan = export.plan_export(rig.sras, rig.result)
assert plan.valid_px[ref] == src_rows * src_frames, \
"reference angle lost pixels to the in-bounds test"
# And the values themselves land as an exact, unrotated block.
src_img = dc_mv(rig.sras, ref)
out_img = dc_mv(rig.out, ref)
t = rig.result.per_angle[ref]
row0, col0 = (int(round(-t.offset[0])), int(round(-t.offset[1])))
assert np.array_equal(out_img[row0:row0 + src_rows, col0:col0 + src_frames],
src_img), \
"reference angle is not a verbatim block in the export"
def test_edge_tolerance_only_affects_the_epsilon_rim(rig):
"""Where the export's bounds test and scipy's disagree, the coordinate must
be within _EDGE_TOL of the boundary — i.e. only pixels whose scipy answer
was itself decided by float noise, never a real half-pixel decision."""
for a in range(rig.sras.n_angles):
t = rig.result.per_angle[a]
src_rows, src_frames = rig.sras.image_shape(a)
n_rows, n_cols = rig.result.canvas_shape
ones = np.ones((src_rows, src_frames), dtype=np.float32)
scipy_valid = compute.apply_alignment(rig.result, a, ones) > 0.5
sr, sc = export._src_coords(t, np.arange(n_rows), n_cols)
ours = export._in_bounds(sr, sc, src_rows, src_frames)
differ = ours != scipy_valid
assert not (scipy_valid & ~ours).any(), \
f"angle {a}: export drops pixels scipy keeps"
if differ.any():
# Every disagreement sits within the tolerance of an edge.
near = (np.abs(sr) <= export._EDGE_TOL)
near |= (np.abs(sr - (src_rows - 1)) <= export._EDGE_TOL)
near |= (np.abs(sc) <= export._EDGE_TOL)
near |= (np.abs(sc - (src_frames - 1)) <= export._EDGE_TOL)
assert near[differ].all(), \
f"angle {a}: bounds differ away from the epsilon rim"
def test_export_matches_apply_alignment_on_ch3(rig):
"""Channel-agnostic: the gather moves whole pixels, not per-channel images."""
for a in range(rig.sras.n_angles):
preview = compute.apply_alignment(rig.result, a,
dc_mv(rig.sras, a, CH3_IDX))
actual = dc_mv(rig.out, a, CH3_IDX)
inside = preview != 0.0
assert actual[inside] == pytest.approx(preview[inside], abs=1e-3)
def test_padding_is_zero_mv_not_zero_adc(rig):
"""Unreachable canvas pixels must read as ~0 mV on every channel.
Filling with literal zero ADC would decode to (0 - yoff) * ymult + yzero —
for this fixture's CH4 calibration that is +100 mV, well above any sensible
mask threshold, so the padding would masquerade as valid sample everywhere.
"""
a = rig.sras.n_angles - 1
t = rig.result.per_angle[a]
n_rows, n_cols = rig.result.canvas_shape
src_rows, src_frames = rig.sras.image_shape(a)
sr, sc = export._src_coords(t, np.arange(n_rows), n_cols)
outside = ~export._in_bounds(sr, sc, src_rows, src_frames)
assert outside.any(), "rotated angle should leave unreachable canvas corners"
for ch in (CH3_IDX, CH4_IDX):
img = dc_mv(rig.out, a, ch)
half_step = abs(rig.sras.cal(ch)[0]) / 2.0
assert np.abs(img[outside]).max() <= half_step + 1e-6, \
f"CH{ch} padding is not within half an ADC step of 0 mV"
# And the sanity check that makes the above meaningful: zero ADC would not
# have passed it.
assert abs(adc_to_mv(0, *rig.sras.cal(CH4_IDX))) > 10.0
def test_reregistering_the_export_is_identity(rig):
"""The export really is aligned: registering it against its own angle 0
recovers no rotation and no shift.
The end-to-end check — it fails for any index error, sign flip, wrong pivot
or origin mistake anywhere in crop/affine/gather, in a way the
self-consistency tests above cannot.
"""
dc4 = {a: dc_mv(rig.out, a) for a in range(rig.out.n_angles)}
for a in range(1, rig.out.n_angles):
fit = compute.register_angle_to_reference(
rig.out, a, 0, dc4, dc_threshold_mv=_THRESHOLD_MV,
seed_deg=0.0, seed_signs=(1,))
assert abs(fit.rotation_deg) <= _ROT_TOL_DEG, \
f"angle {a} still rotated by {fit.rotation_deg:.3f}° after export"
assert float(np.hypot(*fit.shift_mm)) <= _SHIFT_TOL_MM, \
f"angle {a} still shifted by {fit.shift_mm} mm after export"
def test_export_of_int16_input(rig, tmp_path):
"""bps=2 inputs keep their big-endian int16 dtype through the gather."""
src_path = tmp_path / "i16.sras"
gen.write(src_path, n_angles=2, samples_per_frame=16, bps=2)
sras = SrasFile(str(src_path))
result = compute.build_manual_alignment(sras, 0, 0.0, {})
out_path = tmp_path / "i16_aligned.sras"
export.write_aligned_sras(sras, result, out_path)
out = SrasFile(str(out_path))
assert out.bytes_per_sample == 2
assert out.data[0].dtype == np.dtype(">i2")
for a in range(sras.n_angles):
preview = compute.apply_alignment(result, a, dc_mv(sras, a))
actual = dc_mv(out, a)
inside = preview != 0.0
assert actual[inside] == pytest.approx(preview[inside], abs=1e-3)
# ---------------------------------------------------------------------------
# Cropping
# ---------------------------------------------------------------------------
def test_crop_is_a_window_of_the_full_canvas(rig):
"""crop_alignment_result must resample exactly the sub-rectangle it names.
Asserted as bit-exact equality, not approximately: the crop composes into
the affine's offset by an integer number of canvas pixels, so anything but
an exact match means the composition is wrong.
"""
n_rows, n_cols = rig.result.canvas_shape
row0, col0 = n_rows // 5, n_cols // 4
nr, nc = n_rows // 2, n_cols // 3
cropped = compute.crop_alignment_result(rig.result, row0, col0, nr, nc)
assert cropped.canvas_shape == (nr, nc)
assert cropped.canvas_origin_mm[0] == pytest.approx(
rig.result.canvas_origin_mm[0] + col0 * rig.result.canvas_dx_mm)
assert cropped.canvas_origin_mm[1] == pytest.approx(
rig.result.canvas_origin_mm[1] + row0 * rig.result.canvas_dy_mm)
for a in range(rig.sras.n_angles):
img = dc_mv(rig.sras, a)
full = compute.apply_alignment(rig.result, a, img)
assert np.array_equal(
compute.apply_alignment(cropped, a, img),
full[row0:row0 + nr, col0:col0 + nc]), \
f"angle {a}: cropped resample is not the same window"
# Rotation/shift are properties of the angle, not of the canvas.
assert cropped.per_angle[a].rotation_deg == rig.result.per_angle[a].rotation_deg
assert cropped.per_angle[a].shift_mm == rig.result.per_angle[a].shift_mm
def test_cropped_export_round_trips(rig, tmp_path):
n_rows, n_cols = rig.result.canvas_shape
row0, col0, nr, nc = n_rows // 4, n_cols // 4, n_rows // 2, n_cols // 2
cropped = compute.crop_alignment_result(rig.result, row0, col0, nr, nc)
out_path = tmp_path / "cropped.sras"
export.write_aligned_sras(rig.sras, cropped, out_path)
out = SrasFile(str(out_path))
assert set(out.n_rows) == {nr} and set(out.n_frames) == {nc}
assert out.x_start_mm[0] == pytest.approx(cropped.canvas_origin_mm[0], abs=1e-4)
for a in range(rig.sras.n_angles):
preview = compute.apply_alignment(cropped, a, dc_mv(rig.sras, a))
actual = dc_mv(out, a)
inside = preview != 0.0
if inside.any():
assert actual[inside] == pytest.approx(preview[inside], abs=1e-3)
def test_crop_rejects_empty_window(rig):
with pytest.raises(ValueError, match="empty crop"):
compute.crop_alignment_result(rig.result, 0, 0, 0, 10)
with pytest.raises(ValueError, match="empty crop"):
compute.crop_alignment_result(rig.result, 0, 0, 10, -1)
# ---------------------------------------------------------------------------
# plan_export and overlap_stats
# ---------------------------------------------------------------------------
def test_plan_export_matches_what_was_written(rig):
plan = export.plan_export(rig.sras, rig.result)
n_rows, n_cols = rig.result.canvas_shape
assert (plan.n_rows, plan.n_frames) == (n_rows, n_cols)
assert plan.n_angles == rig.sras.n_angles
data_bytes = sum(n for _, _, n in rig.out.iter_angle_blocks())
assert plan.total_bytes == data_bytes
assert plan.bytes_per_angle * plan.n_angles == plan.total_bytes
# Coverage must agree with the pixels that actually carry data. The
# reference angle is unrotated, so its whole footprint lands inside.
ref_px = np.prod(rig.sras.image_shape(0))
assert plan.valid_px[0] == ref_px
for a in range(1, rig.sras.n_angles):
assert 0 < plan.valid_px[a] <= n_rows * n_cols
assert 0.0 < plan.coverage_frac(a) < 1.0
def test_plan_export_flags_a_crop_that_misses_an_angle(rig):
"""A crop over a corner the rotated angles cannot reach must warn, and the
export must still succeed by writing that angle as padding."""
n_rows, n_cols = rig.result.canvas_shape
corner = compute.crop_alignment_result(rig.result, 0, 0,
max(1, n_rows // 12),
max(1, n_cols // 12))
plan = export.plan_export(rig.sras, corner)
empty = [a for a in range(rig.sras.n_angles) if plan.valid_px[a] == 0]
assert empty, "top-left canvas corner should be unreachable for some angle"
assert any("all padding" in w for w in plan.warnings)
def test_overlap_stats():
counts = np.array([[0, 1, 2], [3, 3, 0], [0, 2, 3]])
stats = compute.overlap_stats(counts, 3)
assert stats["union_px"] == 6
assert stats["full_px"] == 3
assert stats["full_frac"] == pytest.approx(0.5)
assert stats["max_count"] == 3
assert stats["mean_count"] == pytest.approx((1 + 2 + 3 + 3 + 2 + 3) / 6)
assert stats["empty"] is False
empty = compute.overlap_stats(np.zeros((4, 4), dtype=int), 3)
assert empty["empty"] is True
assert empty["full_frac"] == 0.0 and empty["mean_count"] == 0.0
def test_largest_rect_at_least():
# A 2x3 block of 3s with a notch that a bounding box would swallow.
counts = np.array([
[0, 0, 0, 0, 0],
[0, 3, 3, 3, 0],
[0, 3, 3, 3, 0],
[0, 3, 0, 3, 0],
])
row0, col0, nr, nc = compute.largest_rect_at_least(counts, 3)
assert (nr * nc) == 6 and (row0, col0, nr, nc) == (1, 1, 2, 3)
assert (counts[row0:row0 + nr, col0:col0 + nc] >= 3).all()
# A column taller than the wide block is the better rectangle.
tall = np.array([[3, 3], [3, 0], [3, 0], [3, 0]])
r0, c0, nr2, nc2 = compute.largest_rect_at_least(tall, 3)
assert (r0, c0, nr2, nc2) == (0, 0, 4, 1)
assert compute.largest_rect_at_least(np.zeros((3, 3), dtype=int), 1) is None
# Whole-array case: no notch, so the answer is the array itself.
assert compute.largest_rect_at_least(np.full((3, 4), 2), 2) == (0, 0, 3, 4)
def test_largest_rect_matches_brute_force():
"""Randomized check against an O(n^4) reference.
The histogram sweep is short and easy to get subtly wrong — an off-by-one in
the stack unwind yields rectangles that are merely large, and "large but not
maximal" is invisible by eye on real data.
"""
def brute(good):
n_rows, n_cols = good.shape
best = 0
for r0 in range(n_rows):
for r1 in range(r0 + 1, n_rows + 1):
run = 0
for g in good[r0:r1].all(axis=0):
run = run + 1 if g else 0
best = max(best, run * (r1 - r0))
return best
rng = np.random.default_rng(0)
for _ in range(200):
counts = rng.integers(0, 3, size=(int(rng.integers(1, 9)),
int(rng.integers(1, 9))))
got = compute.largest_rect_at_least(counts, 2)
expected = brute(counts >= 2)
if got is None:
assert expected == 0
continue
row0, col0, nr, nc = got
assert (counts[row0:row0 + nr, col0:col0 + nc] >= 2).all(), \
f"rectangle is not pure:\n{counts}\n{got}"
assert nr * nc == expected, \
f"not maximal ({nr * nc} < {expected}):\n{counts}\n{got}"
def test_largest_rect_is_pure_on_the_real_fixture(rig):
"""On real overlap counts the returned rectangle must contain only
full-overlap pixels — the property a bounding box would violate."""
n = rig.sras.n_angles
masks = {a: (dc_mv(rig.sras, a) >= _THRESHOLD_MV).astype(np.float32)
for a in range(n)}
counts = sum(compute.apply_alignment(rig.result, a, masks[a]) > 0.5
for a in range(n)).astype(int)
assert counts.max() == n, "fixture alignment should have a full-overlap region"
rect = compute.largest_rect_at_least(counts, n)
assert rect is not None
row0, col0, nr, nc = rect
assert (counts[row0:row0 + nr, col0:col0 + nc] == n).all(), \
"convenience crop must not include pixels some angle misses"
# And it must beat the naive bounding box, which here is impure.
rr, cc = np.nonzero(counts == n)
bbox_pure = (counts[rr.min():rr.max() + 1, cc.min():cc.max() + 1] == n).all()
assert not bbox_pure, "fixture no longer exercises the bounding-box hazard"
# ---------------------------------------------------------------------------
# Legacy inputs, validation and durability
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("version", [2, 4])
def test_legacy_input_exports_as_v6(version, tmp_path):
"""v2-v5 inputs keep no verbatim preamble/background spans, so those
sections have to be re-encoded. v2 additionally has neither."""
src_path = tmp_path / f"legacy_v{version}.sras"
gen.write_legacy(src_path, version=version, n_angles=2)
sras = SrasFile(str(src_path))
result = compute.build_manual_alignment(sras, 0, 0.0, {})
out_path = tmp_path / f"legacy_v{version}_aligned.sras"
export.write_aligned_sras(sras, result, out_path)
out = SrasFile(str(out_path))
assert out.version == 6
assert out.n_angles == sras.n_angles
# A zero background rather than a zero-length one: consumers subtract it
# from a (spf,)-shaped row, which a length-0 array cannot broadcast against.
assert out.background is not None
assert out.background.size == sras.samples_per_frame
if sras.background is None:
assert np.all(out.background == 0)
assert any("no background" in w for w in
export.plan_export(sras, result).warnings)
# Calibration must survive: v2 has no preambles and falls back to the
# hardcoded scope constants, and the re-encoded empty preambles must land on
# exactly the same fallback.
for ch in range(sras.n_channels):
assert out.cal(ch) == pytest.approx(sras.cal(ch))
for a in range(sras.n_angles):
preview = compute.apply_alignment(result, a, dc_mv(sras, a))
actual = dc_mv(out, a)
inside = preview != 0.0
assert actual[inside] == pytest.approx(preview[inside], abs=1e-3)
def test_too_many_rows_is_rejected_before_writing(rig, tmp_path):
"""The geometry table stores n_rows as a u16; silently truncating would
write a file whose header disagrees with its own waveform block."""
huge = compute.crop_alignment_result(rig.result, 0, 0, 70000, 4)
out_path = tmp_path / "huge.sras"
with pytest.raises(ValueError, match="exceeds the .sras per-angle geometry"):
export.write_aligned_sras(rig.sras, huge, out_path)
assert not out_path.exists()
assert not out_path.with_name(out_path.name + ".part").exists()
def test_missing_transform_is_rejected(rig, tmp_path):
broken = compute.crop_alignment_result(rig.result, 0, 0,
*rig.result.canvas_shape)
del broken.per_angle[1]
with pytest.raises(ValueError, match="no transform for angle"):
export.write_aligned_sras(rig.sras, broken, tmp_path / "broken.sras")
def test_cancelled_export_leaves_nothing_behind(rig, tmp_path):
out_path = tmp_path / "cancelled.sras"
written = export.write_aligned_sras(rig.sras, rig.result, out_path,
should_stop=lambda: True)
assert written == out_path
assert not out_path.exists(), "cancelled export must not leave an output file"
assert not out_path.with_name(out_path.name + ".part").exists()
def test_failed_write_leaves_nothing_behind(rig, tmp_path):
"""An exception mid-write must remove the partial file: a short .sras is
not detectably broken — the v6 parser reads it as an aborted scan."""
out_path = tmp_path / "boom.sras"
def explode(_pct):
raise RuntimeError("boom")
with pytest.raises(RuntimeError, match="boom"):
export.write_aligned_sras(rig.sras, rig.result, out_path,
progress_cb=explode)
assert not out_path.exists()
assert not out_path.with_name(out_path.name + ".part").exists()
def test_progress_is_monotonic_and_completes(rig, tmp_path):
seen: list[int] = []
export.write_aligned_sras(rig.sras, rig.result, tmp_path / "prog.sras",
progress_cb=seen.append)
assert seen and seen[-1] == 100
assert seen == sorted(seen)
assert all(0 <= p <= 100 for p in seen)
def test_band_reader_path_is_byte_identical(rig, tmp_path, monkeypatch):
"""A source block too large to hold in RAM is served from sliding bands
instead. That path only runs on multi-gigabyte scans, so force it with a
tiny budget and demand the same bytes — otherwise the one code path that
matters on real data is the one never tested."""
whole = tmp_path / "whole.sras"
export.write_aligned_sras(rig.sras, rig.result, whole)
monkeypatch.setattr(compute, "_TOTAL_BYTES_BUDGET", 4096)
banded = tmp_path / "banded.sras"
export.write_aligned_sras(rig.sras, rig.result, banded)
assert banded.read_bytes() == whole.read_bytes()
def test_row_chunking_is_invariant(rig, tmp_path, monkeypatch):
"""Output must not depend on how many rows are buffered per write."""
base = tmp_path / "base.sras"
export.write_aligned_sras(rig.sras, rig.result, base)
monkeypatch.setattr(export, "_ROW_CHUNK", 1)
one = tmp_path / "one.sras"
export.write_aligned_sras(rig.sras, rig.result, one)
assert one.read_bytes() == base.read_bytes()
def test_refuses_to_overwrite_the_source(rig):
"""The source's waveform blocks are live read-only memmaps; writing over
the file would corrupt the reads the gather is making from it."""
with pytest.raises(ValueError, match="refusing to export onto the source"):
export.write_aligned_sras(rig.sras, rig.result, rig.src_path)
assert SrasFile(str(rig.src_path)).n_angles == rig.sras.n_angles
def test_overwrites_an_existing_file(rig, tmp_path):
out_path = tmp_path / "existing.sras"
out_path.write_bytes(b"not a scan")
export.write_aligned_sras(rig.sras, rig.result, out_path)
assert SrasFile(str(out_path)).version == 6
# ---------------------------------------------------------------------------
# The registration knobs the wizard exposes
# ---------------------------------------------------------------------------
def test_locked_rotation_returns_exactly_the_seed(rig):
"""search_deg=0 + one sign + refine=False pins rotation to the seed, which
is what "lock rotation to the stage angle" means on the wizard's first
page. Only the translation may be searched."""
dc4 = {a: dc_mv(rig.sras, a) for a in range(rig.sras.n_angles)}
for a in range(1, rig.sras.n_angles):
nominal = compute.nominal_delta_deg(rig.sras, a, 0)
fit = compute.register_angle_to_reference(
rig.sras, a, 0, dc4, dc_threshold_mv=_THRESHOLD_MV,
search_deg=0.0, coarse_step_deg=2.0, seed_signs=(-1,), refine=False)
assert fit.rotation_deg == pytest.approx(-nominal)
def test_seed_deg_overrides_the_stage_angle(rig):
"""seed_deg=0.0 searches around no rotation at all, so a scan whose angles
are genuinely ~37° apart must fail to find them within a ±2° window —
proving the seed is what positions the search."""
dc4 = {a: dc_mv(rig.sras, a) for a in range(rig.sras.n_angles)}
fit = compute.register_angle_to_reference(
rig.sras, 1, 0, dc4, dc_threshold_mv=_THRESHOLD_MV,
search_deg=2.0, seed_deg=0.0, seed_signs=(1,), refine=False)
truth_rot = rig.meta["truth"][1][0]
assert abs(fit.rotation_deg) <= 2.0
assert abs(fit.rotation_deg - truth_rot) > 10.0
def test_rotation_candidates_signs():
both = compute._rotation_candidates(10.0, 2.0, 2.0)
assert both == compute._rotation_candidates(10.0, 2.0, 2.0, (-1, 1)), \
"default must stay the both-signs sweep"
assert compute._rotation_candidates(10.0, 0.0, 2.0, (1,)) == [10.0]
assert compute._rotation_candidates(10.0, 0.0, 2.0, (-1,)) == [-10.0]
# A zero seed collapses the two windows; the dedupe must keep one copy.
assert compute._rotation_candidates(0.0, 2.0, 2.0) == [-2.0, 0.0, 2.0]
def test_zero_mv_fill_code_is_clipped_to_dtype():
"""mv_to_adc is unclamped, so the fill code must be clipped or the int8
cast wraps around to a large-magnitude value."""
fake = type("S", (), dict(
n_channels=1, samples_per_frame=2,
cal=lambda self, ch: (1e-6, 0.0, 5000.0)))()
row = export._fill_row(fake, 3, np.dtype(np.int8))
assert row.shape == (1, 3, 2)
assert row.min() == row.max() == np.iinfo(np.int8).min
assert mv_to_adc(0.0, 1e-6, 0.0, 5000.0) < np.iinfo(np.int8).min
+212
View File
@@ -0,0 +1,212 @@
"""Angle-alignment tests: does registration actually stack the scans?
Builds a synthetic scan in which one sample is imaged at several *known*
rotations and offsets (tools/make_test_sras.write_rotating) and checks that the
alignment path recovers them, that the shared canvas is angle 0's own pixel
grid extended, and that nothing in the result depends on any other angle's
stage coordinates.
No Qt — this exercises sras_compute directly. See tests/test_gui.py for the
dialog and Aligned-View plumbing.
"""
from types import SimpleNamespace
import numpy as np
import pytest
import sras_compute as compute
from sras_format import CH4_IDX, SrasFile, adc_to_mv
import tools.make_test_sras as gen
# Registration is limited by how far a feature moves per degree: with this
# sample's ~1 mm radius and a ~16 µm registration pitch, a quarter degree is
# already sub-pixel, so it is the floor of what any metric can resolve here.
_ROT_TOL_DEG = 0.5
_SHIFT_TOL_MM = 0.02
_STACK_IOU_MIN = 0.90
_THRESHOLD_MV = 80.0
def dc4_images(sras: SrasFile) -> dict[int, np.ndarray]:
return {a: adc_to_mv(compute.compute_dc_image(sras, a, CH4_IDX), *sras.cal(CH4_IDX))
for a in range(sras.n_angles)}
def mm_transform(sras: SrasFile, result, angle_idx: int) -> np.ndarray:
"""Recover the pure mm-space rotation from a canvas->raw affine.
matrix == D @ R^T @ A_out, where A_out and D only carry the canvas and
per-angle pixel pitches; undoing both must leave something orthonormal, or
the transform is smuggling in a scale or a shear.
"""
dx_a, dy_a = compute.pixel_pitch_mm(sras, angle_idx)
A_out = np.array([[0.0, result.canvas_dx_mm], [result.canvas_dy_mm, 0.0]])
D = np.array([[0.0, 1.0 / dy_a], [1.0 / dx_a, 0.0]])
return np.linalg.inv(D) @ result.per_angle[angle_idx].matrix @ np.linalg.inv(A_out)
@pytest.fixture(scope="module")
def rig(tmp_path_factory):
"""The rotating-sample scan plus everything computed from it once."""
tmpdir = tmp_path_factory.mktemp("sras_align")
path = tmpdir / "rotating.sras"
meta = gen.write_rotating(path, n_angles=5)
sras = SrasFile(str(path))
dc4 = dc4_images(sras)
fits = {a: compute.register_angle_to_reference(
sras, a, 0, dc4, dc_threshold_mv=_THRESHOLD_MV)
for a in range(sras.n_angles)}
result = compute.compute_angle_alignment(sras, 0, _THRESHOLD_MV)
return SimpleNamespace(path=path, sras=sras, truth=meta["truth"],
dc4=dc4, fits=fits, result=result)
def test_registration_recovers_truth(rig):
"""Per-angle rigid registration (rotation + translation, no scale)."""
for a, fit in rig.fits.items():
t_rot, t_shift = rig.truth[a]
rot_err = abs(fit.rotation_deg - t_rot)
shift_err = float(np.hypot(fit.shift_mm[0] - t_shift[0],
fit.shift_mm[1] - t_shift[1]))
assert rot_err <= _ROT_TOL_DEG, \
(f"angle {a}: got {fit.rotation_deg:.3f}°, truth {t_rot:.3f}° "
f"(err {rot_err:.3f}°)")
assert shift_err <= _SHIFT_TOL_MM, f"angle {a}: err {shift_err:.4f} mm"
assert rig.fits[0] == compute.RigidFit(0.0, (0.0, 0.0), 1.0, "reference"), \
"reference angle registers as exact identity"
def test_stage_angle_sign_is_not_trusted(rig):
# The stage's rotational sense relative to this module's math-positive
# convention is not knowable from the file, and the old code hardcoded a
# guess. Flipping every reported angle must therefore change nothing: the
# search scores both signs and the images decide.
flipped = SrasFile(str(rig.path))
flipped.angles_deg = -flipped.angles_deg
flipped_fits = {a: compute.register_angle_to_reference(
flipped, a, 0, rig.dc4, dc_threshold_mv=_THRESHOLD_MV)
for a in range(1, flipped.n_angles)}
mismatches = {a: (flipped_fits[a].rotation_deg, rig.fits[a].rotation_deg)
for a in flipped_fits if flipped_fits[a] != rig.fits[a]}
assert not mismatches, \
f"negating every reported stage angle changed fits: {mismatches}"
def test_stage_coordinates_are_not_consulted(rig):
# Move every non-reference angle's scan window somewhere else entirely.
# Only angle 0's coordinates may matter, so every fit must be untouched.
moved = SrasFile(str(rig.path))
for a in range(1, moved.n_angles):
moved.x_start_mm[a] += 13.5 * a
moved.y_pos_per_angle[a] = moved.y_pos_per_angle[a] - 9.25 * a
moved_dc4 = dc4_images(moved)
moved_fits = {a: compute.register_angle_to_reference(
moved, a, 0, moved_dc4, dc_threshold_mv=_THRESHOLD_MV)
for a in range(1, moved.n_angles)}
mismatches = {a: (round(moved_fits[a].rotation_deg, 4), rig.fits[a].rotation_deg)
for a in moved_fits if moved_fits[a] != rig.fits[a]}
assert not mismatches, \
f"relocating every other angle's scan window changed fits: {mismatches}"
def test_canvas_is_reference_grid_extended(rig):
sras, result = rig.sras, rig.result
t0 = result.per_angle[0]
assert np.allclose(t0.matrix, np.eye(2)), \
f"angle 0's transform has rotation/scale/shear: {t0.matrix}"
assert np.allclose(t0.offset, np.round(t0.offset)), \
f"angle 0 does not land on whole canvas pixels: {t0.offset}"
assert ((result.canvas_dx_mm, result.canvas_dy_mm)
== compute.pixel_pitch_mm(sras, 0)), \
"canvas pitch is angle 0's own pitch"
n_rows, n_cols = result.canvas_shape
x_axis = result.canvas_origin_mm[0] + np.arange(n_cols) * result.canvas_dx_mm
y_axis = result.canvas_origin_mm[1] + np.arange(n_rows) * result.canvas_dy_mm
row0, col0 = int(round(-t0.offset[0])), int(round(-t0.offset[1]))
a0_rows, a0_cols = sras.image_shape(0)
assert np.allclose(x_axis[col0:col0 + a0_cols], sras.x_axis_mm(0)), \
"canvas X axis reproduces angle 0's own X coordinates"
assert np.allclose(y_axis[row0:row0 + a0_rows], sras.y_positions_mm(0)), \
"canvas Y axis reproduces angle 0's own Y coordinates"
assert (n_rows >= max(int(sras.n_rows[a]) for a in range(sras.n_angles))
and n_cols >= max(int(sras.n_frames[a]) for a in range(sras.n_angles))), \
f"canvas does not cover every angle's footprint: {result.canvas_shape}"
def test_transforms_are_pure_rotations(rig):
"""No scaling anywhere in the per-angle transforms."""
for a in range(rig.sras.n_angles):
R = mm_transform(rig.sras, rig.result, a)
assert (np.allclose(R @ R.T, np.eye(2), atol=1e-9)
and abs(abs(np.linalg.det(R)) - 1.0) < 1e-9), \
f"angle {a}: det={np.linalg.det(R):.6f}"
def test_all_angles_stack(rig):
aligned = {a: compute.apply_alignment(rig.result, a, rig.dc4[a])
for a in range(rig.sras.n_angles)}
base = aligned[0] >= _THRESHOLD_MV
for a in range(1, rig.sras.n_angles):
other = aligned[a] >= _THRESHOLD_MV
iou = float((base & other).sum()) / max(1, int((base | other).sum()))
assert iou >= _STACK_IOU_MIN, f"angle {a}: IoU {iou:.4f}"
def test_downsampled_preview_lands_with_full_res(rig):
# The wizard reprojects block-mean-downsampled masks, so the
# affine has to account for the factor. When it did not, every preview
# layer came out magnified by that factor and offset — the overlay showed a
# blown-up crop of each mask, which is not something you can align by eye.
sras, result = rig.sras, rig.result
pitch = (result.canvas_dx_mm, result.canvas_dy_mm)
a = sras.n_angles - 1
p = result.per_angle[a]
full_mask = (rig.dc4[a] >= _THRESHOLD_MV).astype(np.float32)
full = compute.reproject_mask(
sras, a, 0, full_mask, p.rotation_deg, p.shift_mm, pitch,
result.canvas_origin_mm, result.canvas_shape)
fy, fx = 4, 16
small = compute.reproject_mask(
sras, a, 0, compute.block_mean_2d(full_mask, fy, fx),
p.rotation_deg, p.shift_mm, (pitch[0] * fx, pitch[1] * fy),
result.canvas_origin_mm,
(result.canvas_shape[0] // fy, result.canvas_shape[1] // fx),
src_downsample=(fy, fx))
# Compare in mm, via each layer's own center of mass.
def com_mm(layer, px, py):
rows, cols = np.nonzero(layer > 0.5)
return np.array([cols.mean() * px, rows.mean() * py])
d = com_mm(small, pitch[0] * fx, pitch[1] * fy) - com_mm(full, *pitch)
assert (abs(d[0]) <= abs(pitch[0] * fx) and abs(d[1]) <= abs(pitch[1] * fy)), \
f"downsampled preview offset {d[0]:+.4f}, {d[1]:+.4f} mm"
def test_manual_path_reproduces_geometry(rig):
sras, result = rig.sras, rig.result
params = {a: compute.ManualAngleParams(t.rotation_deg, t.shift_mm)
for a, t in result.per_angle.items()}
manual = compute.build_manual_alignment(sras, 0, _THRESHOLD_MV, params)
assert (manual.canvas_shape == result.canvas_shape
and np.allclose(manual.canvas_origin_mm, result.canvas_origin_mm)
and all(np.allclose(manual.per_angle[a].matrix, result.per_angle[a].matrix)
and np.allclose(manual.per_angle[a].offset, result.per_angle[a].offset)
for a in range(sras.n_angles))), \
"build_manual_alignment matches compute_angle_alignment for the same params"
def test_sidecar_roundtrip(rig):
sras, result = rig.sras, rig.result
params = {a: compute.ManualAngleParams(t.rotation_deg, t.shift_mm)
for a, t in result.per_angle.items()}
compute.save_manual_alignment(sras, 0, _THRESHOLD_MV, params)
loaded = compute.load_manual_alignment(sras)
assert (loaded is not None
and all(np.isclose(loaded.per_angle[a].rotation_deg, params[a].rotation_deg)
and np.allclose(loaded.per_angle[a].shift_mm, params[a].shift_mm)
for a in range(sras.n_angles))), \
"sidecar reloads every angle's params"
assert compute.delete_manual_alignment(sras), "sidecar deletes cleanly"
+719
View File
@@ -0,0 +1,719 @@
"""Both batch image exports -- Convert -> Batch Export View as Images (one
PNG per *file*, via sras_render.export_view_image) and Export -> Batch
Export Images (one PNG per *angle x channel* of the open file, via
sras_workers.BatchExportWorker). Does the exported PNG actually match what
the live view would show, and does the batch dispatch (menu action ->
worker -> per-file render) behave like Batch Compute's proven pattern?
The recurring hazard both halves guard is a dropped view setting: every
parameter that decides what an FFT-derived pixel *is* (bg-sub, pad,
row-averaging, the min peak frequency floor) defaults to "off" in
compute_rf_image, so an argument the export forgets to forward does not
degrade gracefully -- it silently renders a different image than the screen,
and worse, makes this file's stored cache look like a match so the export
hands back peaks from an earlier compute.
sras_render.export_view_image is tested directly (no Qt) for the plumbing
that decides *what* gets rendered -- pad_factor derived per file, masked-
pixel NaN fill, per-file auto-scale, velocity scaling -- via a
draw_view_image spy rather than pixel-diffing PNGs, the same "spy on the
seam, don't inspect the rendered artifact" approach test_highlight_masked_
pixels (tests/test_gui.py) uses for the live canvas.
The GUI-dispatch half drives SrasViewerWindow._on_batch_export_images()
end-to-end with patched file dialogs, the same shape as
test_stored_cache.py's test_viewer_batch_row_average_dispatch.
"""
from pathlib import Path
from unittest.mock import patch
import numpy as np
import pytest
from PyQt6.QtCore import QEventLoop, QTimer
from PyQt6.QtWidgets import QApplication
import sras_render
import sras_workers
from sras_compute import cache_file, compute_rf_image, dc_image_mv
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, CH_NAMES, SrasFile
from sras_render import DEFAULT_FIGSIZE, export_view_image, sanitize_figsize
from sras_viewer import SrasViewerWindow, VELOCITY_MODE_IDX
from sras_workers import (
BatchExportImagesWorker, BatchExportWorker, ExportChannel,
)
import tools.make_test_sras as gen
_THRESHOLD_MV = 50.0
def pump(ms: int = 200):
loop = QEventLoop()
QTimer.singleShot(ms, loop.quit)
loop.exec()
def wait_until(pred, timeout_ms: int = 20000, step: int = 100) -> bool:
waited = 0
while waited < timeout_ms:
if pred():
return True
pump(step)
waited += step
return pred()
# ---------------------------------------------------------------------------
# sras_render.export_view_image -- pure function, no Qt
# ---------------------------------------------------------------------------
_DEFAULT_KW = dict(
is_fft_mode=False, is_velocity=False, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, pad_factor=1, min_freq_mhz=0.0, grating_um=1.0,
cmap="viridis", auto_scale=True, vmin=0.0, vmax=1.0,
highlight_masked=False, mode_str="DC", colorbar_label="mV",
)
def _kw(**overrides):
kw = dict(_DEFAULT_KW)
kw.update(overrides)
return kw
def _png_size(path) -> tuple[int, int]:
"""(width, height) in pixels, read straight out of the PNG's IHDR chunk
(two big-endian uint32s at byte 16) -- no image library needed just to
check the shape of an export."""
raw = Path(path).read_bytes()
assert raw[:8] == b"\x89PNG\r\n\x1a\n", "not a valid PNG"
return (int.from_bytes(raw[16:20], "big"),
int.from_bytes(raw[20:24], "big"))
def _spy_draw(monkeypatch):
"""Patches sras_render.draw_view_image to record the image array and
vmin/vmax/bad_color it was called with, then delegates to the real
implementation so the PNG is still written -- lets a test check *what*
export_view_image computed without depending on rendered PNG pixels."""
orig = sras_render.draw_view_image
captured = {}
def spy(ax, fig, img, extent, cmap, vmin, vmax, xlabel, ylabel, title,
colorbar_label="", cb_ticks=None, norm=None, bad_color=None):
captured["img"] = np.array(img, copy=True)
captured["vmin"] = vmin
captured["vmax"] = vmax
captured["bad_color"] = bad_color
return orig(ax, fig, img, extent, cmap, vmin, vmax, xlabel, ylabel,
title, colorbar_label, cb_ticks, norm, bad_color)
monkeypatch.setattr(sras_render, "draw_view_image", spy)
return captured
def test_export_dc_channel_writes_png(tmp_path):
path = tmp_path / "dc.sras"
gen.write(path, n_angles=2, seed=1, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
err, out_name = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH4_IDX, **_kw())
assert err == ""
assert out_name == f"dc_angle0_{CH_NAMES[CH4_IDX]}.png"
out_path = out_dir / out_name
assert out_path.exists() and out_path.stat().st_size > 0
assert out_path.read_bytes()[:8] == b"\x89PNG\r\n\x1a\n", "not a valid PNG"
def test_export_out_of_range_angle(tmp_path):
path = tmp_path / "short.sras"
gen.write(path, n_angles=2, seed=2, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
err, out_name = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=5, ch_idx=CH4_IDX, **_kw())
assert err != "" and "2" in err, "error should mention the file's actual angle count"
assert out_name == ""
assert list(out_dir.iterdir()) == [], "no file written for a failed export"
def test_export_fft_mode_matches_compute_rf_image(tmp_path, monkeypatch):
path = tmp_path / "fft.sras"
gen.write(path, n_angles=1, seed=3, samples_per_frame=128)
out_dir = tmp_path / "out"
out_dir.mkdir()
captured = _spy_draw(monkeypatch)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH1_IDX,
**_kw(is_fft_mode=True))
assert err == ""
sras = SrasFile(str(path))
expected = compute_rf_image(sras, 0, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, n_fft=None, min_freq_mhz=0.0)
assert np.array_equal(captured["img"], expected)
def test_export_velocity_scales_frequency(tmp_path, monkeypatch):
path = tmp_path / "vel.sras"
gen.write(path, n_angles=1, seed=4, samples_per_frame=128)
out_dir = tmp_path / "out"
out_dir.mkdir()
captured = _spy_draw(monkeypatch)
grating_um = 3.5
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=VELOCITY_MODE_IDX,
**_kw(is_fft_mode=True, is_velocity=True, grating_um=grating_um))
assert err == ""
sras = SrasFile(str(path))
freq = compute_rf_image(sras, 0, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, n_fft=None, min_freq_mhz=0.0)
assert np.array_equal(captured["img"], freq * grating_um)
def test_pad_factor_uses_each_files_own_samples_per_frame(tmp_path, monkeypatch):
"""n_fft must be derived per file from that file's own samples_per_frame,
never a value carried over from whichever file the caller had open --
otherwise every file but one in a batch gets silently mis-padded."""
orig_draw = sras_render.draw_view_image
out_dir = tmp_path / "out"
out_dir.mkdir()
for i, spf in enumerate((64, 256)):
path = tmp_path / f"pad_{spf}.sras"
gen.write(path, n_angles=1, seed=5 + i, samples_per_frame=spf)
captured = {}
def spy(ax, fig, img, *a, __c=captured, **kw):
__c["img"] = np.array(img, copy=True)
return orig_draw(ax, fig, img, *a, **kw)
monkeypatch.setattr(sras_render, "draw_view_image", spy)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH1_IDX,
**_kw(is_fft_mode=True, pad_factor=4))
assert err == ""
sras = SrasFile(str(path))
expected = compute_rf_image(sras, 0, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, n_fft=spf * 4,
min_freq_mhz=0.0)
assert np.array_equal(captured["img"], expected), \
f"samples_per_frame={spf}: n_fft must use this file's own value"
def test_highlight_masked_sets_nan_and_bad_color(tmp_path, monkeypatch):
path = tmp_path / "mask.sras"
gen.write(path, n_angles=1, seed=7, samples_per_frame=128)
out_dir = tmp_path / "out"
out_dir.mkdir()
sras = SrasFile(str(path))
dc4 = dc_image_mv(sras, 0, CH4_IDX)
threshold = float(np.median(dc4))
expect_masked = dc4 < threshold
assert expect_masked.any() and not expect_masked.all(), \
"fixture threshold should mask some but not all pixels"
captured = _spy_draw(monkeypatch)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH1_IDX,
**_kw(is_fft_mode=True, dc_threshold_mv=threshold,
highlight_masked=True, mask_color="magenta"))
assert err == ""
assert captured["bad_color"] == "magenta"
# The export masks by value too (0 == the "no valid peak" sentinel, same
# rule as the viewer's _redraw_image); on this fixture every
# above-threshold pixel has a nonzero peak, so the value mask coincides
# with the DC mask and the NaN set is exactly expect_masked.
assert np.array_equal(np.isnan(captured["img"]), expect_masked)
valid_vals = captured["img"][~expect_masked]
assert not np.isnan(valid_vals).any() and (valid_vals != 0).all(), \
"fixture precondition: every valid pixel has a nonzero peak"
captured2 = _spy_draw(monkeypatch)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH1_IDX,
**_kw(is_fft_mode=True, dc_threshold_mv=threshold, highlight_masked=False))
assert err == ""
assert captured2["bad_color"] is None
assert not np.isnan(captured2["img"]).any()
def test_auto_scale_uses_per_file_min_max(tmp_path, monkeypatch):
path = tmp_path / "scale.sras"
gen.write(path, n_angles=1, seed=8, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
captured = _spy_draw(monkeypatch)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH4_IDX,
**_kw(auto_scale=True))
assert err == ""
img = captured["img"]
assert captured["vmin"] == pytest.approx(float(np.nanmin(img)))
assert captured["vmax"] == pytest.approx(float(np.nanmax(img)))
captured2 = _spy_draw(monkeypatch)
err, _ = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH4_IDX,
**_kw(auto_scale=False, vmin=-5.0, vmax=5.0))
assert err == ""
assert captured2["vmin"] == -5.0
assert captured2["vmax"] == 5.0
@pytest.mark.parametrize("figsize", [(12.0, 4.0), (4.0, 9.0)])
def test_export_matches_requested_figsize(tmp_path, figsize):
"""The PNG comes out at the caller's figure size, so a view the user has
sized wide (or tall) doesn't get squeezed into a fixed 7x5 -- the image
is drawn with aspect="auto", so the figure box *is* the map's shape."""
path = tmp_path / "shape.sras"
gen.write(path, n_angles=1, seed=70, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
dpi = 100
err, out_name = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH4_IDX,
figsize=figsize, dpi=dpi, **_kw())
assert err == ""
w_px, h_px = _png_size(out_dir / out_name)
# Agg truncates inches*dpi to whole pixels; a pixel of slack, not an
# aspect-ratio tolerance, is what's being allowed for here.
assert abs(w_px - figsize[0] * dpi) <= 1
assert abs(h_px - figsize[1] * dpi) <= 1
def test_export_default_figsize_when_unspecified(tmp_path):
"""No figsize (a caller with no live canvas to match) still renders at
the viewer's starting size rather than failing or guessing."""
path = tmp_path / "default_shape.sras"
gen.write(path, n_angles=1, seed=71, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
err, out_name = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH4_IDX,
dpi=100, **_kw())
assert err == ""
w_px, h_px = _png_size(out_dir / out_name)
assert abs(w_px - DEFAULT_FIGSIZE[0] * 100) <= 1
assert abs(h_px - DEFAULT_FIGSIZE[1] * 100) <= 1
@pytest.mark.parametrize("bad", [None, (0.0, 0.0), (-3.0, 5.0), (np.nan, 5.0),
(float("inf"), 5.0), (7.0,), "7x5"])
def test_sanitize_figsize_never_yields_an_unrenderable_size(bad):
"""A degenerate canvas size (collapsed pane, minimized window) must cost
at most a wrong-looking image, never a failed export."""
w, h = sanitize_figsize(bad)
assert np.isfinite(w) and np.isfinite(h)
assert w >= 1.0 and h >= 1.0
def test_sanitize_figsize_passes_through_a_normal_canvas_size():
assert sanitize_figsize(np.array([12.8, 6.4])) == pytest.approx((12.8, 6.4))
# ---------------------------------------------------------------------------
# GUI dispatch: SrasViewerWindow._on_batch_export_images end-to-end
# ---------------------------------------------------------------------------
def _make_window(path) -> SrasViewerWindow:
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
win._load_file(str(path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(win._sras.n_angles))), \
"DC precompute finished"
return win
def test_batch_export_images_writes_one_png_per_file(tmp_path):
path = tmp_path / "src.sras"
gen.write(path, n_angles=2, seed=10, samples_per_frame=64)
paths = [str(path)]
for i in range(2):
p2 = tmp_path / f"other{i}.sras"
gen.write(p2, n_angles=2, seed=20 + i, samples_per_frame=64)
paths.append(str(p2))
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(path)
try:
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=(paths, "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
angle = win.spin_angle.value()
ch_name = CH_NAMES[win.combo_channel.currentIndex()]
expected_names = {f"{Path(p).stem}_angle{angle}_{ch_name}.png" for p in paths}
actual_names = {p.name for p in out_dir.iterdir()}
assert actual_names == expected_names
assert "Batch export: 3/3 image(s)" in win.statusBar().currentMessage()
finally:
win.close()
pump(200)
def test_batch_export_out_of_range_angle_reports_error_continues(tmp_path):
good_path = tmp_path / "good.sras"
short_path = tmp_path / "short.sras"
gen.write(good_path, n_angles=3, seed=30, samples_per_frame=64)
gen.write(short_path, n_angles=1, seed=31, samples_per_frame=64)
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(good_path)
try:
win.spin_angle.setValue(2) # valid for good_path, out of range for short_path
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(good_path), str(short_path)], "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
msg = win.statusBar().currentMessage()
assert "Batch export: 1/2 image(s)" in msg, msg
assert "1 failed" in msg, msg
assert len(list(out_dir.iterdir())) == 1, \
"the batch must not abort — the good file still exports"
finally:
win.close()
pump(200)
def test_batch_export_busy_guard_skips_dialogs(tmp_path, monkeypatch):
path = tmp_path / "busy.sras"
gen.write(path, n_angles=1, seed=40, samples_per_frame=64)
win = _make_window(path)
try:
monkeypatch.setattr(win, "_job_running", lambda key: True)
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames") as mock_dlg:
win._on_batch_export_images()
assert mock_dlg.call_count == 0, \
"the Jobs.BATCH busy guard must return before opening any dialog"
finally:
win.close()
pump(200)
def test_batch_export_ignores_aligned_view_toggle(tmp_path):
"""Aligned View is geometry specific to whichever single file the
Alignment Wizard last ran against and cannot be meaningfully applied
across a batch of different files -- _on_batch_export_images must not
read chk_aligned_view / self._alignment_result at all, regardless of
what's checked in the live view."""
path = tmp_path / "aligned.sras"
gen.write(path, n_angles=1, seed=41, samples_per_frame=64)
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(path)
try:
captured_kwargs = []
orig_init = BatchExportImagesWorker.__init__
def spy_init(self, paths, **kw):
captured_kwargs.append(kw)
return orig_init(self, paths, **kw)
win.chk_aligned_view.setChecked(True)
with patch.object(BatchExportImagesWorker, "__init__", spy_init), \
patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
assert len(captured_kwargs) == 1
assert not any("align" in k.lower() for k in captured_kwargs[0]), \
captured_kwargs[0].keys()
finally:
win.close()
pump(200)
def test_batch_export_uses_the_live_canvas_aspect_ratio(tmp_path):
"""The exported PNG has the shape of the view on screen, not a fixed
7x5 -- resize the window and the export follows it."""
path = tmp_path / "aspect.sras"
gen.write(path, n_angles=1, seed=45, samples_per_frame=64)
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(path)
try:
win.resize(1400, 700)
pump(300) # let the canvas' resizeEvent reach the figure
canvas_w, canvas_h = win.image_canvas.figure.get_size_inches()
captured_kwargs = []
orig_init = BatchExportImagesWorker.__init__
def spy_init(self, paths, **kw):
captured_kwargs.append(kw)
return orig_init(self, paths, **kw)
with patch.object(BatchExportImagesWorker, "__init__", spy_init), \
patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
assert captured_kwargs[0]["figsize"] == pytest.approx(
(canvas_w, canvas_h)), "the live canvas size must travel to the worker"
out_files = list(out_dir.iterdir())
assert len(out_files) == 1
w_px, h_px = _png_size(out_files[0])
assert w_px / h_px == pytest.approx(canvas_w / canvas_h, rel=0.01)
finally:
win.close()
pump(200)
def test_batch_export_filename_collision_note(tmp_path):
dir_a, dir_b = tmp_path / "dir_a", tmp_path / "dir_b"
dir_a.mkdir()
dir_b.mkdir()
path_a, path_b = dir_a / "dup.sras", dir_b / "dup.sras"
gen.write(path_a, n_angles=1, seed=50, samples_per_frame=64)
gen.write(path_b, n_angles=1, seed=51, samples_per_frame=64)
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(path_a)
try:
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path_a), str(path_b)], "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
msg = win.statusBar().currentMessage()
assert "Batch export: 2/2 image(s)" in msg, msg
assert "collision" in msg, msg
assert len(list(out_dir.iterdir())) == 1, \
"same-stem inputs silently overwrite to one output file"
finally:
win.close()
pump(200)
def test_batch_export_does_not_modify_source_files(tmp_path):
path = tmp_path / "untouched.sras"
gen.write(path, n_angles=1, seed=60, samples_per_frame=64)
before = path.read_bytes()
out_dir = tmp_path / "images"
out_dir.mkdir()
win = _make_window(path)
try:
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")), \
patch("sras_viewer.main_window.QFileDialog.getExistingDirectory",
return_value=str(out_dir)):
win._on_batch_export_images()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
finally:
win.close()
pump(200)
assert path.read_bytes() == before, "export must never write to the source file"
# ---------------------------------------------------------------------------
# Export menu: BatchExportWorker (every angle of the open file)
# ---------------------------------------------------------------------------
# High enough to sit above several of the generator's synthetic peak bins
# (bin spacing is 6.25 GS/s / 64 = 97.66 MHz, peaks land at bins 3-19), so a
# floor at this value genuinely changes which peak each pixel resolves to --
# without that, a test that the floor is honored would pass either way.
_FLOOR_MHZ = 1000.0
_GRATING_UM = 2.0
def _velocity_channel() -> ExportChannel:
return ExportChannel(ch_idx=CH1_IDX, is_velocity=True, vmin=0.0, vmax=1e5,
label="Velocity", unit="m/s", tag="VEL")
def _run_export_worker(monkeypatch, sras, out_dir, **overrides) -> list[np.ndarray]:
"""Runs BatchExportWorker to completion on the calling thread (its run()
is a plain loop; the QThread in _run_worker is a GUI concern) and returns
the image arrays it rendered, captured at the _render_map_png seam."""
captured = []
orig = sras_workers._render_map_png
def spy(img, extent, **kw):
captured.append(np.array(img, copy=True))
return orig(img, extent, **kw)
monkeypatch.setattr(sras_workers, "_render_map_png", spy)
kw = dict(cmap="viridis", apply_bg_sub=True, dc_threshold_mv=_THRESHOLD_MV,
n_fft=None, grating_um=_GRATING_UM, min_freq_mhz=_FLOOR_MHZ)
kw.update(overrides)
BatchExportWorker(sras, [_velocity_channel()], str(out_dir), "vel", **kw).run()
return captured
def test_batch_export_applies_the_min_peak_freq_floor(tmp_path, monkeypatch):
"""The exported Velocity map is the floored one the viewer shows, not the
unfloored peaks the floor was raised to reject."""
path = tmp_path / "floor.sras"
gen.write(path, n_angles=1, seed=80, samples_per_frame=64, geometry=[(6, 9)])
out_dir = tmp_path / "out"
out_dir.mkdir()
sras = SrasFile(str(path))
def expected(floor):
return compute_rf_image(sras, 0, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, n_fft=None,
min_freq_mhz=floor) * _GRATING_UM
floored, unfloored = expected(_FLOOR_MHZ), expected(0.0)
assert not np.allclose(floored, unfloored), \
"fixture must be one where the floor changes the image"
exported = _run_export_worker(monkeypatch, sras, out_dir)
assert len(exported) == 1
assert np.allclose(exported[0], floored)
def test_batch_export_does_not_serve_an_unfloored_stored_cache(tmp_path,
monkeypatch):
"""A file batch-computed before the floor existed has a stored cache with
no floor recorded. Asking for that cache at floor 0 (rather than the live
floor) makes it a match, so the export would render the stored pre-floor
peaks -- the map the user raised the floor to get rid of."""
path = tmp_path / "stored_floor.sras"
gen.write(path, n_angles=1, seed=81, samples_per_frame=64, geometry=[(6, 9)])
assert cache_file(str(path), "fft", apply_bg_sub=True) == ""
sras = SrasFile(str(path))
assert sras.precomputed_freq_mhz[0] is not None, "stored cache written"
assert sras.precomputed_min_freq_mhz == 0.0
out_dir = tmp_path / "out"
out_dir.mkdir()
exported = _run_export_worker(monkeypatch, sras, out_dir)[0]
# 0.0 is the "no valid peak" sentinel; anything else below the floor is a
# peak that only an unfloored search could have reported.
below = (exported > 0) & (exported < _FLOOR_MHZ * _GRATING_UM)
assert not below.any(), \
f"{below.sum()} sub-floor pixel(s) survived the export"
def test_batch_export_dispatch_passes_the_live_view_settings(tmp_path):
"""The Export menu hands the worker what the panel currently says --
the floor in particular, which is a display control and so persists
across file loads while the window's own FFT cache does not."""
path = tmp_path / "dispatch.sras"
gen.write(path, n_angles=1, seed=82, samples_per_frame=64)
out_dir = tmp_path / "out"
out_dir.mkdir()
win = _make_window(path)
try:
win.spin_min_freq_mhz.setValue(_FLOOR_MHZ)
win.spin_grating_um.setValue(_GRATING_UM)
win.spin_threshold_mv.setValue(_THRESHOLD_MV)
class StubDialog:
def __init__(self, parent, **kw):
pass
def exec(self):
from PyQt6.QtWidgets import QDialog
return QDialog.DialogCode.Accepted
def get_output_dir(self):
return str(out_dir)
def get_prefix(self):
return "vel"
def get_selected_channels(self):
return [_velocity_channel()]
captured_kwargs = []
orig_init = BatchExportWorker.__init__
def spy_init(self, sras, channels, output_dir, prefix, **kw):
captured_kwargs.append(kw)
return orig_init(self, sras, channels, output_dir, prefix, **kw)
with patch.object(BatchExportWorker, "__init__", spy_init), \
patch("sras_viewer.main_window.BatchExportDialog", StubDialog):
win._on_batch_export()
assert wait_until(lambda: not win._job_running("export"), 60000), "export ran"
kw = captured_kwargs[0]
assert kw["min_freq_mhz"] == _FLOOR_MHZ
assert kw["grating_um"] == _GRATING_UM
assert kw["dc_threshold_mv"] == _THRESHOLD_MV
finally:
win.close()
pump(200)
def test_export_view_image_serves_a_row_averaged_stored_cache(tmp_path,
monkeypatch):
"""Batch Export View as Images must ask this file's cache the question the
viewer asks it. _stored_fft_image reads at the file's own
precomputed_row_avg_n; requesting raw per-pixel instead makes a
row-averaged cache a mismatch, and the export silently renders a full raw
recompute where the screen shows the smoothed stored image."""
path = tmp_path / "rowavg_cache.sras"
gen.write(path, n_angles=1, seed=83, samples_per_frame=64, geometry=[(6, 9)])
# A planted stored image rather than a real row-averaged compute: the
# question here is *which* source the export reads, and a distinctive
# array answers it without depending on the synthetic waveforms being
# smooth enough for averaging to move the numbers.
sras = SrasFile(str(path))
shape = sras.image_shape(0)
stored = (100.0 + 10.0 * np.arange(shape[0] * shape[1], dtype=np.float32)
).reshape(shape)
sras.write_v7_cache(new_freq_mhz=[stored], new_row_avg_n=5,
new_bg_sub=True, new_pad_factor=1)
reread = SrasFile(str(path))
raw = compute_rf_image(reread, 0, dc_threshold_mv=None, apply_bg_sub=True,
n_fft=None, row_avg_n=0, use_stored=False)
assert not np.allclose(stored, raw), "planted cache must be distinguishable"
out_dir = tmp_path / "out"
out_dir.mkdir()
captured = _spy_draw(monkeypatch)
err, _out_name = export_view_image(
str(path), out_dir=str(out_dir), angle_idx=0, ch_idx=CH1_IDX,
**_kw(is_fft_mode=True, dc_threshold_mv=-1e9, colorbar_label="MHz",
mode_str="RF"))
assert err == ""
assert np.allclose(captured["img"], stored), \
"export rendered a recompute instead of the stored image on screen"
+478
View File
@@ -0,0 +1,478 @@
"""Behavioural tests for the compute/format layer.
Covers what the golden-hash harness can't: the v6->v7 cache round-trip
(including block carry-forward), parallel-vs-serial identity, the no-mask
fast path, and the ROI bounding-box mask optimisation.
"""
import subprocess
import sys
from pathlib import Path
import numpy as np
import pytest
import sras_average
import sras_compute as compute
from sras_compute import (
cache_file, compute_dc_image, compute_rf_image, dc_image_mv,
)
from sras_format import CH3_IDX, CH4_IDX, SrasFile, adc_to_mv
import tools.make_test_sras as gen
REPO = Path(__file__).resolve().parent.parent
def test_cache_roundtrip(tmp_path):
"""v6 -> v7 for DC, then FFT, asserting the first block survives the
second write (the carry-forward path in write_v7_cache)."""
path = tmp_path / "roundtrip.sras"
gen.write(path, n_angles=3, seed=1, samples_per_frame=64)
src = SrasFile(str(path))
assert src.version == 6, f"got v{src.version}"
expect_dc3 = [dc_image_mv(src, a, CH3_IDX) for a in range(src.n_angles)]
expect_dc4 = [dc_image_mv(src, a, CH4_IDX) for a in range(src.n_angles)]
expect_fft = [compute_rf_image(src, a, dc_threshold_mv=None, apply_bg_sub=True)
for a in range(src.n_angles)]
err = cache_file(str(path), "dc", True)
assert err == "", err
after_dc = SrasFile(str(path))
assert after_dc.version == 7, f"got v{after_dc.version}"
assert all(x is not None for x in after_dc.precomputed_dc3_mv)
assert all(np.allclose(after_dc.precomputed_dc3_mv[a], expect_dc3[a], atol=1e-4)
for a in range(after_dc.n_angles))
assert all(np.allclose(after_dc.precomputed_dc4_mv[a], expect_dc4[a], atol=1e-4)
for a in range(after_dc.n_angles))
assert all(x is None for x in after_dc.precomputed_freq_mhz), "no fft block yet"
assert (after_dc.precomputed_dc3_mv[0].dtype == np.float32
and after_dc.precomputed_dc3_mv[0].dtype.byteorder in ("=", "|")), \
"cached images are native float32"
assert after_dc.precomputed_dc3_mv[0].flags.writeable
err = cache_file(str(path), "fft", True)
assert err == "", err
both = SrasFile(str(path))
assert all(x is not None for x in both.precomputed_freq_mhz)
assert all(np.allclose(both.precomputed_freq_mhz[a], expect_fft[a], atol=1e-3)
for a in range(both.n_angles))
assert all(np.allclose(both.precomputed_dc3_mv[a], expect_dc3[a], atol=1e-4)
for a in range(both.n_angles)), \
"DC block carried forward through the FFT write"
assert both.precomputed_bg_sub is True
# The fast path must reproduce a fresh compute, and masking must still
# apply on top of a cached (unmasked) image.
fresh = SrasFile(str(path))
fresh.precomputed_freq_mhz = [None] * fresh.n_angles
dc4 = dc_image_mv(both, 0, CH4_IDX)
thr = float(np.median(dc4))
assert np.allclose(
compute_rf_image(both, 0, dc_threshold_mv=None, apply_bg_sub=True),
compute_rf_image(fresh, 0, dc_threshold_mv=None, apply_bg_sub=True),
atol=1e-3), "cached fast path == fresh compute (unmasked)"
assert np.allclose(
compute_rf_image(both, 0, dc_threshold_mv=thr, apply_bg_sub=True),
compute_rf_image(fresh, 0, dc_threshold_mv=thr, apply_bg_sub=True),
atol=1e-3), "cached fast path == fresh compute (masked)"
# Waveform data must be byte-identical to the pre-cache file.
orig = tmp_path / "roundtrip_orig.sras"
gen.write(orig, n_angles=3, seed=1, samples_per_frame=64)
o, n = SrasFile(str(orig)), SrasFile(str(path))
assert all(np.array_equal(np.asarray(o.data[a]), np.asarray(n.data[a]))
for a in range(o.n_angles)), \
"waveform data untouched by the cache write"
def test_partial_v7_cache(tmp_path):
"""Only some angles cached: uncached angles must compute, not read zeros.
This is the v5 bug the ragged normalisation fixed, checked via v7."""
path = tmp_path / "partial.sras"
gen.write(path, n_angles=3, seed=2, samples_per_frame=64)
src = SrasFile(str(path))
expected = [compute_rf_image(src, a, dc_threshold_mv=None, apply_bg_sub=True)
for a in range(src.n_angles)]
partial = [expected[0], None, expected[2]] # angle 1 deliberately absent
src.write_v7_cache(new_freq_mhz=partial, new_bg_sub=True)
reread = SrasFile(str(path))
assert reread.precomputed_freq_mhz[1] is None
assert (reread.precomputed_freq_mhz[0] is not None
and reread.precomputed_freq_mhz[2] is not None)
img1 = compute_rf_image(reread, 1, dc_threshold_mv=None, apply_bg_sub=True)
assert np.any(img1 != 0) and np.allclose(img1, expected[1], atol=1e-3), \
"uncached angle computes rather than returning zeros"
def test_parallel_identity(tmp_path, monkeypatch):
"""Forcing 1 worker vs many must give identical output — catches
chunk-boundary and race bugs."""
path = tmp_path / "parallel.sras"
# Many rows, so the row loop actually splits into several chunks.
n_rows, n_frames, spf = 48, 9, 256
gen.write(path, n_angles=1, seed=3, samples_per_frame=spf,
geometry=[(n_rows, n_frames)])
sras = SrasFile(str(path))
# Shrink the budget so the outer row loop splits into many chunks, and
# the block size so every chunk splits into many FFT tasks — the worst
# case for boundary bugs.
monkeypatch.setattr(compute, "_TOTAL_BYTES_BUDGET", 8 * n_frames * spf * 4)
monkeypatch.setattr(compute, "_FFT_BLOCK_MAX", 4)
fft_rows = compute._plan_fft_rows(n_frames, spf, compute._TOTAL_BYTES_BUDGET)
assert fft_rows < n_rows, \
f"FFT work actually splits into multiple chunks ({fft_rows} of {n_rows})"
dc_rows = compute._chunk_rows_for(n_frames, spf, compute._TOTAL_BYTES_BUDGET)
assert dc_rows < n_rows, \
f"DC work actually splits into multiple chunks ({dc_rows} of {n_rows})"
monkeypatch.setattr(compute, "_MAX_WORKERS", 1)
dc_serial = compute_dc_image(sras, 0, CH4_IDX)
rf_serial = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True)
dc4 = adc_to_mv(dc_serial, *sras.cal(CH4_IDX))
thr = float(np.median(dc4))
rf_masked_serial = compute_rf_image(sras, 0, dc_threshold_mv=thr,
apply_bg_sub=True)
rf_pad_serial = compute_rf_image(sras, 0, dc_threshold_mv=thr,
apply_bg_sub=True, n_fft=spf * 8)
monkeypatch.setattr(compute, "_MAX_WORKERS", 8)
dc_par = compute_dc_image(sras, 0, CH4_IDX)
rf_par = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True)
rf_masked_par = compute_rf_image(sras, 0, dc_threshold_mv=thr, apply_bg_sub=True)
rf_pad_par = compute_rf_image(sras, 0, dc_threshold_mv=thr,
apply_bg_sub=True, n_fft=spf * 8)
assert np.array_equal(dc_serial, dc_par), "dc image identical"
assert np.array_equal(rf_serial, rf_par), "rf image identical (unmasked)"
assert np.array_equal(rf_masked_serial, rf_masked_par), \
"rf image identical (masked)"
assert np.array_equal(rf_pad_serial, rf_pad_par), \
"rf image identical (masked, padded/zoom)"
def test_peak_bins_fuzz():
"""Hammer _peak_bins directly with adversarial spectra: noise,
un-subtracted DC offsets, on-bin and off-bin tones, near-tie tone pairs,
and all-zero rows — against an independent scipy.fft reference."""
import scipy.fft as scipy_fft
rng = np.random.default_rng(42)
for _ in range(25):
spf = int(rng.integers(16, 220))
pad = int(rng.choice([4, 5, 8, 16, 40]))
n_fft = spf * pad
n_wf = 24
w = rng.normal(scale=20.0, size=(n_wf, spf))
t = np.arange(spf)
# rows 0-5: pure/noisy tones (some off-bin), row 6-7: near-tie pair,
# row 8: big DC offset, row 9: all zeros, rest: plain noise.
for r in range(6):
f = rng.uniform(1.0, spf / 2 - 1)
w[r] = 60 * np.sin(2 * np.pi * f * t / spf) + w[r] * (r % 2)
f1, f2 = rng.uniform(2.0, spf / 2 - 2, size=2)
w[6] = 50 * np.sin(2 * np.pi * f1 * t / spf) \
+ 49.9 * np.sin(2 * np.pi * f2 * t / spf)
w[7] = 50 * np.sin(2 * np.pi * f1 * t / spf) \
+ 50 * np.cos(2 * np.pi * f2 * t / spf)
w[8] = 90 + rng.normal(scale=5.0, size=spf)
w[9] = 0.0
w = w.astype(np.float32)
S = scipy_fft.rfft(w, n=n_fft, axis=-1, workers=1)
P = S.real ** 2
P += S.imag ** 2
P[:, 0] = 0.0
ref = np.argmax(P, axis=1)
got = compute._peak_bins(w, n_fft)
bad = np.nonzero(ref != got)[0]
assert not len(bad), \
(f"spf={spf} pad={pad}: rows {bad.tolist()} picked "
f"{got[bad].tolist()} instead of {ref[bad].tolist()}")
def test_peak_bins_fuzz_min_freq():
"""Same adversarial-spectra fuzz as test_peak_bins_fuzz, but with a swept
min_bin floor: bins below the floor must be excluded from the argmax
exactly as the independent scipy.fft reference is, when zeroed the same
way before argmax."""
import scipy.fft as scipy_fft
rng = np.random.default_rng(43)
for _ in range(25):
spf = int(rng.integers(16, 220))
pad = int(rng.choice([4, 5, 8, 16, 40]))
n_fft = spf * pad
n_wf = 24
w = rng.normal(scale=20.0, size=(n_wf, spf))
t = np.arange(spf)
for r in range(6):
f = rng.uniform(1.0, spf / 2 - 1)
w[r] = 60 * np.sin(2 * np.pi * f * t / spf) + w[r] * (r % 2)
f1, f2 = rng.uniform(2.0, spf / 2 - 2, size=2)
w[6] = 50 * np.sin(2 * np.pi * f1 * t / spf) \
+ 49.9 * np.sin(2 * np.pi * f2 * t / spf)
w[7] = 50 * np.sin(2 * np.pi * f1 * t / spf) \
+ 50 * np.cos(2 * np.pi * f2 * t / spf)
w[8] = 90 + rng.normal(scale=5.0, size=spf)
w[9] = 0.0
w = w.astype(np.float32)
S = scipy_fft.rfft(w, n=n_fft, axis=-1, workers=1)
P = S.real ** 2
P += S.imag ** 2
n_bins_fine = n_fft // 2 + 1
min_bin = int(rng.integers(1, max(2, n_bins_fine // 3)))
P[:, :min_bin] = 0.0
ref = np.argmax(P, axis=1)
got = compute._peak_bins(w, n_fft, min_bin)
bad = np.nonzero(ref != got)[0]
assert not len(bad), \
(f"spf={spf} pad={pad} min_bin={min_bin}: rows {bad.tolist()} picked "
f"{got[bad].tolist()} instead of {ref[bad].tolist()}")
@pytest.mark.parametrize("spf", [64, 500, 2500])
def test_fft_block_for(spf):
"""Block size == _FFT_BLOCK_MAX at natural resolution, shrinks and stays
>= _FFT_BLOCK_MIN as n_len grows, and the implied per-thread byte
estimate respects _FFT_PLAN_BYTES_BUDGET except when the floor is
engaged."""
block_natural = compute._fft_block_for(spf, spf)
assert block_natural == compute._FFT_BLOCK_MAX
prev = compute._FFT_BLOCK_MAX
for pad in (2, 4, 8, 40, 500):
n_len = spf * pad
block = compute._fft_block_for(spf, n_len)
assert compute._FFT_BLOCK_MIN <= block <= prev
bytes_per_wf = 4 * spf + 8 * (n_len // 2 + 1)
if block > compute._FFT_BLOCK_MIN:
assert block * bytes_per_wf <= compute._FFT_PLAN_BYTES_BUDGET
prev = block
def test_compute_rf_image_min_freq_mhz(tmp_path):
"""min_freq_mhz threads through compute_rf_image end-to-end, for both
the natural-resolution and padded paths: 0.0 (default) must reproduce
the pre-existing image exactly, and a floor above every real peak must
collapse the image to bin 0 (0 MHz) — the same fallback the low-level
search uses when nothing survives the floor."""
path = tmp_path / "floor_e2e.sras"
gen.write(path, n_angles=1, seed=12, samples_per_frame=64)
sras = SrasFile(str(path))
huge_floor = float(sras.freq_axis_mhz(None)[-1]) + 1.0 # above Nyquist
for n_fft in (None, 64 * 8):
unfiltered = compute_rf_image(sras, 0, dc_threshold_mv=None,
apply_bg_sub=False, n_fft=n_fft)
same = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=False,
n_fft=n_fft, min_freq_mhz=0.0)
assert np.array_equal(unfiltered, same), \
f"n_fft={n_fft}: min_freq_mhz=0.0 changed the output"
assert unfiltered.any(), \
f"n_fft={n_fft}: fixture should have real signal"
collapsed = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=False,
n_fft=n_fft, min_freq_mhz=huge_floor)
assert not collapsed.any(), \
f"n_fft={n_fft}: floor above Nyquist should collapse to 0 MHz"
def test_nomask_equals_low_threshold(tmp_path):
"""dc_threshold_mv=None must equal a threshold below every pixel, while
skipping the CH4 read."""
path = tmp_path / "nomask.sras"
gen.write(path, n_angles=2, seed=4, samples_per_frame=128)
sras = SrasFile(str(path))
for a in range(sras.n_angles):
none_img = compute_rf_image(sras, a, dc_threshold_mv=None, apply_bg_sub=True)
low_img = compute_rf_image(sras, a, dc_threshold_mv=-1e9, apply_bg_sub=True)
assert np.array_equal(none_img, low_img), \
f"angle {a}: None == -1e9 threshold"
assert len(np.unique(none_img)) > 1, \
f"angle {a}: image is degenerate ({len(np.unique(none_img))} unique)"
def test_roi_mask():
"""The bbox-restricted mask must equal a full-grid point-in-polygon test."""
from matplotlib.path import Path as MplPath
from sras_viewer import RoiQuad
rng = np.random.default_rng(0)
x = np.linspace(-2.0, 3.0, 137)
y = np.linspace(1.0, 4.0, 91)
cases = {
"axis-aligned rect": np.array([[0.0, 1.5], [1.0, 1.5], [1.0, 3.0], [0.0, 3.0]]),
"skewed quad": np.array([[-0.5, 1.2], [1.7, 1.9], [1.2, 3.4], [-1.0, 2.6]]),
"entirely outside": np.array([[8.0, 8.0], [9.0, 8.0], [9.0, 9.0], [8.0, 9.0]]),
"covers whole grid": np.array([[-9.0, -9.0], [9.0, -9.0], [9.0, 9.0], [-9.0, 9.0]]),
"straddles left edge": np.array([[-4.0, 2.0], [0.5, 2.0], [0.5, 3.0], [-4.0, 3.0]]),
}
for _ in range(5):
cases[f"random {_}"] = rng.uniform([-2.5, 0.5], [3.5, 4.5], size=(4, 2))
for name, pts in cases.items():
roi = RoiQuad(pts)
fast = roi.mask_for_grid(x, y)
X, Y = np.meshgrid(x.astype(np.float64), y.astype(np.float64))
slow = MplPath(pts).contains_points(
np.column_stack([X.ravel(), Y.ravel()])).reshape(X.shape)
assert np.array_equal(fast, slow), f"{name} ({int(slow.sum())} px inside)"
# Descending y axis (images are stored top-down in some scans).
roi = RoiQuad(cases["skewed quad"])
y_desc = y[::-1]
fast = roi.mask_for_grid(x, y_desc)
X, Y = np.meshgrid(x.astype(np.float64), y_desc.astype(np.float64))
slow = MplPath(cases["skewed quad"]).contains_points(
np.column_stack([X.ravel(), Y.ravel()])).reshape(X.shape)
assert np.array_equal(fast, slow), "descending y axis"
def test_legacy_parse(tmp_path):
"""v2-v4 parsing against known written data."""
for version in (2, 3, 4):
path = tmp_path / f"legacy_v{version}.sras"
meta = gen.write_legacy(path, version=version, n_angles=2, n_rows=4,
n_frames=12, samples_per_frame=32, seed=version)
s = SrasFile(str(path))
assert s.version == version, f"got v{s.version}"
assert list(s.n_rows) == [4, 4] and list(s.n_frames) == [12, 12], \
f"rows={list(s.n_rows)} frames={list(s.n_frames)}"
assert all(np.array_equal(np.asarray(s.data[a]), meta["data"][a])
for a in range(s.n_angles)), \
f"v{version} waveform data matches what was written"
assert (s.background is not None) == (version >= 4), \
f"v{version} background {'present' if version >= 4 else 'absent'}"
assert (isinstance(s.precomputed_freq_mhz, list)
and len(s.precomputed_freq_mhz) == s.n_angles), \
f"v{version} precomputed stores are ragged lists"
# DC image must equal a direct mean of the known input.
expect = meta["data"][0][:, CH3_IDX, :, :].astype(np.float64).mean(axis=-1)
assert np.allclose(compute_dc_image(s, 0, CH3_IDX), expect, atol=1e-3), \
f"v{version} DC image equals a direct mean"
def test_sras_average(tmp_path):
"""The sras_average.py CLI: v6 frame averaging with remainder handling,
per-angle ragged geometry, and the laser_freq_hz X-axis correction."""
src = tmp_path / "v6.sras"
meta = gen.write(src, n_angles=2, seed=4, samples_per_frame=32,
geometry=[(4, 12)])
src_sras = SrasFile(str(src))
dst = tmp_path / "v6_avg.sras"
proc = subprocess.run(
[sys.executable, str(REPO / "sras_average.py"), str(src), str(dst), "--n", "4"],
capture_output=True, text=True, cwd=REPO)
assert proc.returncode == 0, (proc.stderr or proc.stdout).strip()[-200:]
avg = SrasFile(str(dst))
assert avg.version == 6
assert list(avg.n_frames) == [3, 3], f"{list(avg.n_frames)}"
assert (avg.n_angles == 2 and list(avg.n_rows) == [4, 4]
and avg.n_channels == src_sras.n_channels == 3)
assert np.allclose(avg.ch_ymult_mv, src_sras.ch_ymult_mv), \
"calibration preserved"
assert np.array_equal(avg.background, src_sras.background), \
"background preserved"
assert avg.laser_freq_hz == pytest.approx(src_sras.laser_freq_hz / 4), \
"laser_freq_hz divided by N keeps pixel_x_mm correct after binning"
assert np.array_equal(avg.x_start_mm, src_sras.x_start_mm), \
"per-angle x_start unchanged"
src_waves = meta["waveforms"]
# int16 (not float32) before .mean(): matches _average_block's own
# float64-accumulator behavior for integer input, so this doesn't
# drift from what _average_block actually guarantees.
expect0 = src_waves[0][:, :, 0:4, :].astype(np.int16).mean(axis=2).astype(np.int16)
assert np.array_equal(np.asarray(avg.data[0])[:, :, 0, :], expect0), \
"first averaged group equals the mean of its 4 source frames"
# Remainder handling: 12 frames / 5 -> 2 full groups + 1 partial.
dst2 = tmp_path / "v6_avg5.sras"
proc2 = subprocess.run([sys.executable, str(REPO / "sras_average.py"),
str(src), str(dst2), "--n", "5"],
capture_output=True, text=True, cwd=REPO)
assert proc2.returncode == 0, (proc2.stderr or proc2.stdout).strip()[-200:]
assert list(SrasFile(str(dst2)).n_frames) == [3, 3], \
"partial trailing group kept by default"
dst3 = tmp_path / "v6_avg5d.sras"
proc3 = subprocess.run([sys.executable, str(REPO / "sras_average.py"),
str(src), str(dst3), "--n", "5", "--discard-remainder"],
capture_output=True, text=True, cwd=REPO)
assert proc3.returncode == 0, (proc3.stderr or proc3.stdout).strip()[-200:]
assert list(SrasFile(str(dst3)).n_frames) == [2, 2], \
"--discard-remainder drops the partial group"
def test_sras_average_v7_cache_dropped(tmp_path):
"""A v7 input's cache tail is indexed by frame count, so it's invalid
after averaging changes that count -- the output must always be plain
v6, never a v7 carrying a stale cache."""
src = tmp_path / "v7.sras"
gen.write(src, n_angles=2, seed=1, samples_per_frame=32, geometry=[(3, 8)])
src_sras = SrasFile(str(src))
src_sras.write_v7_cache(
new_dc3_mv=[compute_dc_image(src_sras, a, CH3_IDX) for a in range(src_sras.n_angles)],
new_dc4_mv=[compute_dc_image(src_sras, a, CH4_IDX) for a in range(src_sras.n_angles)])
assert SrasFile(str(src)).version == 7
dst = tmp_path / "v7_avg.sras"
proc = subprocess.run(
[sys.executable, str(REPO / "sras_average.py"), str(src), str(dst), "--n", "2"],
capture_output=True, text=True, cwd=REPO)
assert proc.returncode == 0, (proc.stderr or proc.stdout).strip()[-200:]
assert SrasFile(str(dst)).version == 6, "cache-bearing input still writes plain v6"
def test_sras_average_rejects_legacy(tmp_path):
"""This tool only speaks v6/v7 now; a legacy file must fail clearly
rather than being silently misparsed."""
src = tmp_path / "legacy_v4.sras"
gen.write_legacy(src, version=4, n_angles=1, n_rows=2, n_frames=8,
samples_per_frame=16, seed=0)
dst = tmp_path / "legacy_v4_avg.sras"
proc = subprocess.run(
[sys.executable, str(REPO / "sras_average.py"), str(src), str(dst), "--n", "2"],
capture_output=True, text=True, cwd=REPO)
assert proc.returncode != 0
assert "v6" in proc.stderr and "v7" in proc.stderr
def test_sras_average_chunking_matches_unchunked(tmp_path):
"""A tiny memory budget (forcing one row per chunk) must produce
byte-identical output to a huge budget (everything in one chunk) -- the
load-bearing correctness claim of the memory-bounded rewrite: chunk
boundaries must never affect the averaged result."""
src = tmp_path / "v6.sras"
gen.write(src, n_angles=2, seed=7, samples_per_frame=48,
geometry=[(6, 10), (5, 13)])
sras = SrasFile(str(src))
dst_tiny = tmp_path / "avg_tiny.sras"
dst_big = tmp_path / "avg_big.sras"
sras_average.write_v6_averaged(sras, dst_tiny, 3, False, budget=1)
sras_average.write_v6_averaged(sras, dst_big, 3, False, budget=1 << 30)
assert dst_tiny.read_bytes() == dst_big.read_bytes(), \
"chunk size must not affect the averaged output"
def test_unsupported_version_reported(tmp_path):
"""cache_file must report, not raise, for a file it can't handle."""
bogus = tmp_path / "bogus.sras"
bogus.write_bytes(b"SRAS" + bytes([99]) + b"\x00" * 200)
err = cache_file(str(bogus), "dc", True)
assert err, "bad version returns an error string"
missing = cache_file(str(tmp_path / "does_not_exist.sras"), "dc", True)
assert missing, "missing file returns an error string"
+75
View File
@@ -0,0 +1,75 @@
"""SrasFile.angles_share_raw_grid(): the no-alignment-needed gating path for
Export Fused ROI.
A plain multi-angle scan gives each angle its own bounding box and stage
x_start (scan_format.md's whole reason v6 geometry is per-angle), so it must
read as "not shareable" without a live alignment. A file the viewer's own
Alignment Wizard exported repeats one Per-Angle Geometry record and one Row
Table span for every angle (scan_format.md, "Files written by the viewer's
Alignment Wizard"), so it must read as "shareable" with no alignment needed
at all.
No Qt: this exercises sras_format/sras_compute/sras_align_export directly,
mirroring tests/test_align_export.py.
"""
import sras_align_export as export
import sras_compute as compute
from sras_format import SrasFile
import tools.make_test_sras as gen
_THRESHOLD_MV = 80.0
def test_single_angle_file_always_shares_its_grid(tmp_path):
path = tmp_path / "one_angle.sras"
gen.write(path, n_angles=1)
sras = SrasFile(str(path))
assert sras.angles_share_raw_grid()
def test_plain_multi_angle_file_does_not_share_its_grid(tmp_path):
"""tools.make_test_sras.write gives every angle its own geometry and
stage x_start (build()'s `x_start = -0.5 + 0.1 * a`), matching how real
v6 scans vary per angle — so this must read as "not shareable"."""
path = tmp_path / "plain.sras"
gen.write(path, n_angles=3)
sras = SrasFile(str(path))
assert not sras.angles_share_raw_grid()
def test_wizard_exported_file_shares_its_grid(tmp_path):
src_path = tmp_path / "rotating.sras"
meta = gen.write_rotating(src_path, n_angles=4)
sras = SrasFile(str(src_path))
params = {a: compute.ManualAngleParams(rot, shift)
for a, (rot, shift) in meta["truth"].items()}
result = compute.build_manual_alignment(sras, 0, _THRESHOLD_MV, params)
out_path = tmp_path / "rotating_aligned.sras"
export.write_aligned_sras(sras, result, out_path)
out = SrasFile(str(out_path))
assert not sras.angles_share_raw_grid(), (
"sanity check: the *source* rotating scan must NOT already share a "
"grid, or this test would not actually exercise the wizard export")
assert out.angles_share_raw_grid()
def test_mutated_angle_breaks_the_shared_grid(tmp_path):
"""A loaded SrasFile's x_start_mm is a public per-angle array (as
tests/test_gui.py::test_alignment_geometry_is_stage_independent also
relies on) -- mutating one angle's start must be visible here too."""
src_path = tmp_path / "rotating.sras"
meta = gen.write_rotating(src_path, n_angles=3)
sras = SrasFile(str(src_path))
params = {a: compute.ManualAngleParams(rot, shift)
for a, (rot, shift) in meta["truth"].items()}
result = compute.build_manual_alignment(sras, 0, _THRESHOLD_MV, params)
out_path = tmp_path / "rotating_aligned.sras"
export.write_aligned_sras(sras, result, out_path)
out = SrasFile(str(out_path))
assert out.angles_share_raw_grid()
out.x_start_mm[1] += 1.0
assert not out.angles_share_raw_grid()
+929
View File
@@ -0,0 +1,929 @@
"""Headless GUI test: drives SrasViewerWindow through the real Qt widgets,
signals and worker threads under the offscreen platform plugin.
Covers the interactions a manual smoke test would: load, switch angles and
channels, background DC precompute, lazy FFT compute, threshold and bg-sub
changes, the alignment wizard end to end (pre-rotation, correlation, manual
nudging, crop, export), aligned view, ROI draw/move, and CSV export.
NOTE: this module is one ordered integration sequence over a single shared
window — the tests build on each other's state and must run in definition
order (pytest's default within a module). Run the whole module, not single
tests.
"""
import json
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
import pytest
from PyQt6.QtCore import QEventLoop, Qt, QTimer
from PyQt6.QtTest import QTest
from PyQt6.QtWidgets import QApplication, QDialog, QMessageBox
import sras_compute as compute
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, SrasFile
from sras_viewer import (
FusedRoiExportDialog, RoiQuad, SrasViewerWindow, VELOCITY_MODE_IDX,
)
import tools.make_test_sras as gen
def pump(ms: int = 250):
"""Run the event loop for a while so queued signals and worker threads
make progress."""
loop = QEventLoop()
QTimer.singleShot(ms, loop.quit)
loop.exec()
def wait_until(pred, timeout_ms: int = 20000, step: int = 100) -> bool:
waited = 0
while waited < timeout_ms:
if pred():
return True
pump(step)
waited += step
return pred()
@pytest.fixture(scope="module")
def ctx(tmp_path_factory):
"""The shared window, test file, and cross-test state for the sequence."""
app = QApplication.instance() or QApplication([])
tmpdir = tmp_path_factory.mktemp("sras_gui")
path = tmpdir / "gui.sras"
gen.write(path, n_angles=4, seed=11, samples_per_frame=256)
win = SrasViewerWindow()
win.show()
errors: list[str] = []
# Capture anything the app reports as an error via the status bar.
win.statusBar().messageChanged.connect(
lambda m: errors.append(m) if m and "error" in m.lower() else None)
c = SimpleNamespace(app=app, win=win, path=path, tmpdir=tmpdir,
errors=errors, s=None)
yield c
if win.isVisible():
win.close()
pump(400)
def test_load(ctx):
win = ctx.win
win._load_file(str(ctx.path))
assert wait_until(lambda: win._sras is not None), "file loaded"
ctx.s = s = win._sras
assert s.version == 6, f"v{s.version}"
assert win.combo_channel.currentIndex() == CH4_IDX, "defaults to CH4"
assert win._current_image is not None, "image displayed"
assert win.spin_angle.maximum() == s.n_angles - 1, \
"angle spinbox ranges over all angles"
assert win._info["Angles"].text() == f"Angles: {s.n_angles}", \
win._info["Angles"].text()
def test_dc_precompute_all_angles(ctx):
win, s = ctx.win, ctx.s
ok = wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
and (a, CH3_IDX) in win._dc_cache
for a in range(s.n_angles)))
assert ok, f"every angle cached for CH3 and CH4 ({len(win._dc_cache)} entries)"
assert "ready for all angles" in win.lbl_dc_precompute.text(), \
win.lbl_dc_precompute.text()
def test_angle_switching_from_cache(ctx):
win, s = ctx.win, ctx.s
for a in range(s.n_angles):
win.spin_angle.setValue(a)
win._on_view_changed()
pump(60)
expected = win._sras.image_shape(a)
assert win._current_image.shape == expected, \
f"angle {a} shows its own geometry {expected}, got {win._current_image.shape}"
assert not win._job_running("compute"), \
"no compute job needed for cached DC angles"
def test_stepping_the_angle_spinbox_redraws(ctx):
"""Clicking the angle spinbox's arrows (or pressing Up/Down in it) must
move the display, not just the number.
This is the ordinary way to walk a scan, and it used to do nothing: the
spinbox was wired on editingFinished, which QAbstractSpinBox emits only
on Return or focus-out — never on a step. Every other test in this module
called _on_view_changed() by hand and so could not have caught it.
"""
win, s = ctx.win, ctx.s
assert s.n_angles >= 3, "need room to step in both directions"
win.spin_angle.setValue(0)
assert wait_until(lambda: win._current_angle == 0), "settled on angle 0"
for expected in range(1, s.n_angles):
win.spin_angle.stepUp()
assert wait_until(lambda e=expected: win._current_angle == e), \
f"stepping up to angle {expected} redrew the display"
win.spin_angle.stepDown()
assert wait_until(lambda: win._current_angle == s.n_angles - 2), \
"stepping down redraws too"
# Keyboard stepping goes through the same signal, so it must work as well.
QTest.keyClick(win.spin_angle, Qt.Key.Key_Down)
assert wait_until(lambda: win._current_angle == s.n_angles - 3), \
"Key_Down redraws"
win.spin_angle.setValue(0)
assert wait_until(lambda: win._current_angle == 0), "back to angle 0"
def test_typing_an_angle_does_not_compute_intermediate_angles(ctx):
"""Keyboard tracking must stay off: with it on, valueChanged fires per
keystroke, so typing "12" would dispatch a compute for angle 1 first —
on a real scan, a whole wasted FFT for an angle the user never asked for.
"""
win, s = ctx.win, ctx.s
assert not win.spin_angle.keyboardTracking(), \
"keyboard tracking off is what makes valueChanged safe to connect"
target = s.n_angles - 1
assert target >= 2, "need a multi-digit-ish range to make the point"
win.spin_angle.setValue(0)
wait_until(lambda: win._current_angle == 0)
seen = []
win.spin_angle.valueChanged.connect(seen.append)
try:
win.spin_angle.lineEdit().selectAll()
QTest.keyClicks(win.spin_angle, str(target))
pump(60)
assert seen == [], f"no signal while typing, got {seen}"
QTest.keyClick(win.spin_angle, Qt.Key.Key_Return)
pump(60)
assert seen == [target], f"one signal on commit, got {seen}"
finally:
win.spin_angle.valueChanged.disconnect(seen.append)
assert wait_until(lambda: win._current_angle == target), "committed angle shown"
win.spin_angle.setValue(0)
assert wait_until(lambda: win._current_angle == 0), "back to angle 0"
def test_channel_switching(ctx):
win = ctx.win
win.spin_angle.setValue(0)
win._on_view_changed()
pump(60)
win.combo_channel.setCurrentIndex(CH3_IDX)
assert wait_until(lambda: win._current_ch == CH3_IDX), "CH3 displayed"
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(
lambda: win._current_ch == CH1_IDX and not win._job_running("compute")), \
"CH1 (FFT) computed"
assert len(win._fft_cache) > 0, "FFT result cached"
ctx.rf_img = win._current_image
assert len(np.unique(ctx.rf_img)) > 1, \
f"FFT image is degenerate ({len(np.unique(ctx.rf_img))} unique values)"
def test_velocity_mode(ctx):
"""Velocity mode is a pure post-multiply, no recompute."""
win = ctx.win
ctx.n_fft_before = len(win._fft_cache)
win.combo_channel.setCurrentIndex(VELOCITY_MODE_IDX)
assert wait_until(
lambda: win._current_ch == VELOCITY_MODE_IDX
and not win._job_running("compute")), "velocity displayed"
grating = win.spin_grating_um.value()
assert np.allclose(win._current_image, ctx.rf_img * grating, atol=1e-3), \
"velocity == freq x grating"
assert len(win._fft_cache) == ctx.n_fft_before, \
f"velocity reused the cached FFT ({ctx.n_fft_before} -> {len(win._fft_cache)})"
assert win.grp_velocity.isVisible(), "grating spinbox visible in velocity mode"
def test_threshold_change_recomputes(ctx):
"""A threshold change is a genuine cache-key change."""
win = ctx.win
win.combo_channel.setCurrentIndex(CH1_IDX)
wait_until(lambda: not win._job_running("compute"))
dc4 = win._dc_cache[(0, CH4_IDX)]
win.spin_threshold_mv.setValue(float(np.median(dc4)))
win._on_threshold_changed()
assert wait_until(
lambda: not win._job_running("compute")
and len(win._fft_cache) > ctx.n_fft_before), "recomputed at new threshold"
n_zero = int((win._current_image == 0).sum())
assert n_zero > 0, \
f"masking zeroed some pixels ({n_zero} of {win._current_image.size})"
def test_highlight_masked_pixels(ctx):
"""Masked (below-threshold) pixels are drawn as NaN - filled with a
highlight color, separate from the normal colormap - so they can't be
mistaken for a real, possibly-low-frequency pixel; unchecking restores
the old behavior where both blend into the same plain 0."""
win = ctx.win
assert win.chk_highlight_masked.isChecked(), "on by default"
dc4 = win._dc_cache[(0, CH4_IDX)]
expect_masked = dc4 < win.spin_threshold_mv.value()
assert expect_masked.any() and not expect_masked.all(), \
"fixture threshold should mask some but not all pixels"
calls = []
orig = win.image_canvas.show_image
def spy(img, *a, **kw):
calls.append((np.array(img, copy=True), kw.get("bad_color")))
return orig(img, *a, **kw)
with patch.object(win.image_canvas, "show_image", side_effect=spy):
win._redraw_image(win._current_image)
shown, bad_color = calls[-1]
assert bad_color is not None, "highlight color set while checkbox is on"
# The highlight masks by value too: in an FFT mode, exactly 0 is the
# "no valid peak" sentinel (DC-masked, below the min-freq floor, or an
# empty spectrum), so the NaN set is the union of the DC mask and the
# zero-valued pixels. On this fixture every above-threshold pixel has a
# nonzero peak, so the union equals the DC mask alone.
expect_nan = expect_masked | (win._current_image == 0)
assert np.array_equal(np.isnan(shown), expect_nan), \
"NaN where DC4 is below threshold or the value-0 sentinel, nowhere else"
assert np.array_equal(expect_nan, expect_masked), \
"fixture precondition: every valid pixel has a nonzero peak"
win.chk_highlight_masked.setChecked(False)
calls.clear()
with patch.object(win.image_canvas, "show_image", side_effect=spy):
win._redraw_image(win._current_image)
shown2, bad_color2 = calls[-1]
assert bad_color2 is None, "no highlight color once unchecked"
assert not np.isnan(shown2).any(), "unchecked: no pixel pulled out to NaN"
assert np.array_equal(shown2, win._current_image), \
"unchecked: displayed array is the raw, unmodified image"
win.chk_highlight_masked.setChecked(True)
pump(60)
def test_bg_sub_toggle(ctx):
"""bg-sub no longer gates the display: it only affects a future live
compute for an angle with nothing cached yet, or an explicit batch
recompute. Toggling it on an angle that already has an FFT image must
leave that image on screen, untouched."""
win = ctx.win
n_before = len(win._fft_cache)
img_before = win._current_image
win.chk_bg_sub.setChecked(False)
pump(200)
assert not win._job_running("compute"), \
"toggling bg-sub alone must not dispatch a recompute"
assert len(win._fft_cache) == n_before, "no new cache entry from the toggle"
assert np.array_equal(win._current_image, img_before), \
"displayed image unchanged by the bg-sub toggle"
win.chk_bg_sub.setChecked(True)
pump(200)
assert not win._job_running("compute")
assert len(win._fft_cache) == n_before
assert np.array_equal(win._current_image, img_before)
def test_roi_and_csv_export(ctx):
win, s = ctx.win, ctx.s
x = s.x_axis_mm(0)
y = s.y_positions_mm(0)
roi = RoiQuad.from_bbox(float(x[1]), float(y[1]),
float(x[-2]), float(y[-2]))
win.image_canvas.set_roi(roi)
pump(120)
assert win.image_canvas.get_roi() is not None, "ROI registered"
assert ("pixels inside" in win.lbl_roi_npix.text()
and win.lbl_roi_npix.text() != "pixels inside: —"), \
win.lbl_roi_npix.text()
npix = int(win.lbl_roi_npix.text().split(":")[1])
assert 0 < npix <= win._current_image.size, f"{npix}"
assert win.btn_export_roi.isEnabled(), "Export ROI enabled"
csv_path = ctx.tmpdir / "roi.csv"
with patch("sras_viewer.main_window.QFileDialog.getSaveFileName",
return_value=(str(csv_path), "")):
win._on_export_roi_csv()
assert csv_path.exists(), "ROI CSV written"
body = [l for l in csv_path.read_text().splitlines() if not l.startswith("#")]
assert len(body) == npix + 1, \
f"ROI CSV has {len(body)} lines for {npix} pixels (want header + one per pixel)"
img_csv = ctx.tmpdir / "img.csv"
with patch("sras_viewer.main_window.QFileDialog.getSaveFileName",
return_value=(str(img_csv), "")):
win._on_export_csv()
assert img_csv.exists(), "image CSV written"
arr = np.loadtxt(img_csv, delimiter=",")
assert (arr.shape == win._current_image.shape
and np.allclose(arr, win._current_image, rtol=1e-5, atol=1e-4)), \
"image CSV round-trips the displayed image"
def test_roi_survives_switches(ctx):
win = ctx.win
win.spin_angle.setValue(1)
win._on_view_changed()
wait_until(lambda: not win._job_running("compute"))
assert win.image_canvas.get_roi() is not None, \
"ROI still present after angle switch"
win.combo_channel.setCurrentIndex(CH4_IDX)
wait_until(lambda: win._current_ch == CH4_IDX)
assert win.image_canvas.get_roi() is not None, \
"ROI still present after channel switch"
def test_alignment_geometry_is_stage_independent(ctx):
"""Local mm is anchored on each angle's array center, not its stage
position: that is what makes a scan's placement independent of where its
window happened to sit. (Registration accuracy itself is covered by
tests/test_alignment.py, which has a synthetic sample to register.)"""
win, s = ctx.win, ctx.s
win.spin_angle.setValue(0)
win._on_view_changed()
wait_until(lambda: not win._job_running("compute"))
assert win._wizard_act.isEnabled(), "alignment wizard action enabled"
n_rows, n_frames = s.image_shape(0)
assert np.allclose(compute._center_idx(s, 0),
[(n_rows - 1) / 2, (n_frames - 1) / 2]), \
"array center is the geometric center of the pixel grid"
dx0, dy0 = compute.pixel_pitch_mm(s, 0)
assert np.allclose(compute._local_half_extent_mm(s, 0),
[(n_frames - 1) / 2 * abs(dx0), (n_rows - 1) / 2 * abs(dy0)]), \
"local half-extent is derived from shape and pitch alone"
identity = {a: compute.ManualAngleParams() for a in range(s.n_angles)}
origin_a, shape_a = compute.canvas_for_params(s, 0, (dx0, dy0), identity)
moved = SrasFile(str(ctx.path))
for a in range(1, moved.n_angles):
moved.x_start_mm[a] += 7.5
moved.y_pos_per_angle[a] = moved.y_pos_per_angle[a] + 3.25
origin_b, shape_b = compute.canvas_for_params(moved, 0, (dx0, dy0), identity)
assert shape_a == shape_b and np.allclose(origin_a, origin_b), \
("moving every non-reference angle's scan window must leave the canvas "
f"unchanged: {origin_a} {shape_a} vs {origin_b} {shape_b}")
# Both signs of the stage's reported angle are searched by default.
cands = compute._rotation_candidates(30.0, 6.0, 2.0)
assert min(cands) < -29.0 and max(cands) > 29.0, f"{min(cands)}..{max(cands)}"
# Whole-pixel translation must not wrap content around the edge.
arr = np.zeros((6, 6), dtype=np.float32)
arr[0, 0] = 1.0
assert compute._shift_into(arr, -1, -1).sum() == 0.0, \
"_shift_into zero-fills rather than wrapping"
assert compute._shift_into(arr, 2, 3)[2, 3] == 1.0, \
"_shift_into moves content by exactly the requested offset"
def test_wizard_opens_prerotated(ctx):
"""The wizard shows a mask stack before any correlation has run, built from
the stage angles in the file — the "pre-rotate" step. Nothing may seed from
the still-live automatic result; only a saved sidecar."""
win, s = ctx.win, ctx.s
win._on_alignment_wizard()
assert win._align_wizard is not None, "wizard opened"
ctx.wiz = wiz = win._align_wizard
ctx.p1 = p1 = wiz.page(wiz.PAGE_CORRELATE)
assert not win._job_running("align_masks"), \
"mask prep needed no background worker (already DC-cached)"
assert wait_until(lambda: p1.isComplete()), "masks ready, Next enabled"
assert not win._wizard_act.isEnabled(), \
"wizard action disabled while a wizard is open"
st = wiz.state
assert st.result is not None, "an AlignmentResult exists from pre-rotation alone"
assert st.counts is not None and st.counts.shape == st.preview_shape
assert 0 <= st.counts.max() <= s.n_angles
assert not st.fits, "no fits before a correlation has run"
for a in range(s.n_angles):
nominal = compute.nominal_delta_deg(s, a, st.ref_angle_idx)
expected = 0.0 if a == st.ref_angle_idx else nominal
assert abs(st.params[a].rotation_deg - expected) < 1e-9, \
f"angle {a} not pre-rotated to its stage angle"
assert st.params[a].shift_mm == (0.0, 0.0), \
"pre-rotation must not invent a translation"
def test_wizard_reference_angle_is_locked(ctx):
p1, wiz = ctx.p1, ctx.wiz
p1.combo_active.setCurrentIndex(wiz.state.ref_angle_idx)
pump(30)
before = wiz.state.params[wiz.state.ref_angle_idx]
p1._on_nudge_translate(1, 0, False)
p1._on_nudge_rotate(1, False)
assert wiz.state.params[wiz.state.ref_angle_idx] == before, \
"reference angle untouched by nudge attempts"
def test_wizard_nudges(ctx):
"""Manual correction, which the wizard absorbed from the old dialog."""
p1, wiz, s = ctx.p1, ctx.wiz, ctx.s
ctx.active = active = 1 if s.n_angles > 1 else 0
p1.combo_active.setCurrentIndex(active)
pump(30)
before = wiz.state.params[active].shift_mm
p1._on_nudge_translate(1, 0, False)
fine = p1.spin_step_translate.value()
assert abs(wiz.state.params[active].shift_mm[0] - (before[0] + fine)) < 1e-9, \
"fine translate nudge moved shift_x by exactly one fine step"
before = wiz.state.params[active].shift_mm
p1._on_nudge_translate(0, -1, True)
coarse = fine * p1.spin_step_mult.value()
assert abs(wiz.state.params[active].shift_mm[1] - (before[1] - coarse)) < 1e-9, \
"coarse translate nudge uses the multiplier"
before_rot = wiz.state.params[active].rotation_deg
p1._on_nudge_rotate(1, False)
assert wiz.state.params[active].rotation_deg != before_rot
assert len(wiz.state.layers) == s.n_angles, \
"stack rebuilt for every angle after a rotation nudge"
# A nudge only reprojects the angle that moved, patching the overlap counts
# in place. That shortcut is only sound if it lands on exactly what a full
# rebuild would have produced.
incremental = wiz.state.counts.copy()
wiz.rebuild_stack()
assert np.array_equal(wiz.state.counts, incremental), \
"incremental nudge update matches a full stack rebuild"
# Real key-event wiring (keyPressEvent -> signal -> slot).
before = wiz.state.params[active].shift_mm
QTest.keyClick(p1.canvas, Qt.Key.Key_Right)
assert wiz.state.params[active].shift_mm[0] > before[0], \
"a real Right-arrow key event nudged shift_x"
# Both views render from the same reprojected layers.
p1.combo_view.setCurrentIndex(1)
pump(50)
p1.combo_view.setCurrentIndex(0)
pump(50)
def test_wizard_correlate(ctx):
"""Cross-correlation, for every source option, retryable."""
win, p1, wiz, s = ctx.win, ctx.p1, ctx.wiz, ctx.s
from sras_viewer.align_wizard import _CORRELATE_SOURCES
for idx, (label, _sources) in enumerate(_CORRELATE_SOURCES):
p1.combo_source.setCurrentIndex(idx)
p1.btn_correlate.click()
assert not p1.isComplete(), \
f"Next must be disabled while correlating ({label})"
assert wait_until(lambda: not win._job_running("align_correlate"),
timeout_ms=60000), f"correlation finished ({label})"
assert p1.isComplete(), f"Next re-enabled ({label})"
assert all(a in wiz.state.fits for a in range(s.n_angles)
if a != wiz.state.ref_angle_idx), \
f"every non-reference angle got a fit ({label})"
assert wiz.state.params[wiz.state.ref_angle_idx] == compute.ManualAngleParams(), \
"reference angle stays identity after correlation"
assert p1.btn_correlate.isEnabled(), "controls re-enabled when done"
assert p1.table.rowCount() == s.n_angles and p1.table.item(0, 0) is not None, \
"per-angle fit table populated"
assert p1.lbl_overlap.text(), "overlap summary reported"
r = wiz.state.result
assert len(r.per_angle) == s.n_angles, "transform for every angle"
assert r.per_angle[r.ref_angle_idx].shift_mm == (0.0, 0.0), \
"reference angle has zero shift"
def test_wizard_retry_changes_geometry(ctx):
"""Editing a parameter and re-running is the retry path, and it must
invalidate anything indexed against the old canvas."""
p1, wiz = ctx.p1, ctx.wiz
gen_before = wiz.state.geometry_generation
p1.spin_threshold.setValue(p1.spin_threshold.value() + 5.0)
p1.spin_threshold.editingFinished.emit()
pump(60)
assert wiz.state.geometry_generation > gen_before, \
"a threshold change rebuilt the geometry"
# Reset drops the fits and returns to pre-rotation only.
p1.btn_reset.click()
pump(60)
assert not wiz.state.fits, "reset cleared the fits"
nominal = compute.nominal_delta_deg(ctx.s, ctx.active, wiz.state.ref_angle_idx)
assert abs(wiz.state.params[ctx.active].rotation_deg - nominal) < 1e-9
assert wiz.state.params[ctx.active].shift_mm == (0.0, 0.0), \
"reset also drops nudged translation"
# Put a real correlation back for the pages that follow.
p1.btn_correlate.click()
assert wait_until(lambda: not ctx.win._job_running("align_correlate"),
timeout_ms=60000)
def test_wizard_roi_page(ctx):
"""The crop page: presets, and two-way sync between the drawn rectangle and
the numeric canvas-pixel boxes."""
wiz = ctx.wiz
wiz.next()
pump(150)
assert wiz.currentId() == wiz.PAGE_ROI, "advanced to the ROI page"
ctx.p2 = p2 = wiz.page(wiz.PAGE_ROI)
st = wiz.state
assert st.crop is not None and p2.isComplete(), \
"a default crop is offered on entry"
n_rows, n_cols = st.result.canvas_shape
assert st.crop[2] > 1 or n_rows == 1, \
f"default crop must not collapse to a single row: {st.crop}"
p2.btn_whole.click()
pump(50)
assert st.crop == (0, 0, n_rows, n_cols), "whole-canvas preset"
p2.btn_fit_union.click()
pump(50)
assert st.counts[st.crop[0]:st.crop[0] + st.crop[2],
st.crop[1]:st.crop[1] + st.crop[3]].sum() == st.counts.sum(), \
"fit-to-union must keep every covered pixel"
if p2.btn_fit_overlap.isEnabled():
p2.btn_fit_overlap.click()
pump(50)
row0, col0, nr, nc = st.crop
assert (st.counts[row0:row0 + nr, col0:col0 + nc] >= 1).all(), \
"full-overlap crop must not include uncovered pixels"
# Numeric -> drawn rectangle.
p2.btn_whole.click()
pump(50)
target = (0, 0, max(1, n_rows // 2), max(1, n_cols // 2))
p2.spin_rows.setValue(target[2])
p2.spin_cols.setValue(target[3])
pump(50)
assert st.crop == target, f"spin boxes drive the crop: {st.crop} vs {target}"
# Drawn rectangle -> numeric, round-tripping exactly.
x0, y0 = wiz.canvas_to_mm(target[1] - 0.5, target[0] - 0.5)
x1, y1 = wiz.canvas_to_mm(target[1] + target[3] - 0.5,
target[0] + target[2] - 0.5)
p2.canvas.set_roi(RoiQuad.from_bbox(min(x0, x1), min(y0, y1),
max(x0, x1), max(y0, y1)))
pump(80)
assert st.crop == target, \
f"drawn rectangle round-trips to the same crop: {st.crop} vs {target}"
# A degenerate crop blocks Next.
st.crop = None
p2.completeChanged.emit()
assert not p2.isComplete(), "an absent crop blocks Next"
p2._set_crop(*target)
assert p2.isComplete()
ctx.crop = target
def test_wizard_crop_dropped_when_going_back(ctx):
"""A crop is canvas-pixel indexed, so it cannot survive a re-correlation."""
wiz, p2 = ctx.wiz, ctx.p2
wiz.back()
pump(120)
assert wiz.currentId() == wiz.PAGE_CORRELATE
assert wiz.state.crop is None, "cleanupPage discarded the stale crop"
assert wiz.cropped_plan() == (None, None), \
"nothing derived from the dropped crop survives either"
wiz.next()
pump(150)
assert wiz.state.crop is not None, "a fresh default crop is offered again"
p2._set_crop(*ctx.crop)
def test_wizard_export(ctx):
"""Writing the file: Finish stays unavailable until a write succeeds."""
win, wiz = ctx.win, ctx.wiz
out = ctx.tmpdir / "wizard_aligned.sras"
with patch("sras_viewer.align_wizard.QMessageBox.question",
return_value=QMessageBox.StandardButton.Yes):
wiz.next()
pump(150)
assert wiz.currentId() == wiz.PAGE_SAVE, "advanced to the save page"
ctx.p3 = p3 = wiz.page(wiz.PAGE_SAVE)
cropped, _ = wiz.cropped_plan()
assert cropped is not None, "crop applied on leaving page 2"
assert cropped.canvas_shape == ctx.crop[2:], \
"cropped result carries the chosen shape"
assert not p3.isComplete(), "Finish unavailable before anything is written"
assert p3.lbl_summary.text(), "a summary of what will be written is shown"
with patch("sras_viewer.align_wizard.QFileDialog.getSaveFileName",
return_value=(str(out), "")):
p3.btn_browse.click()
assert wiz.state.out_path == str(out)
p3.btn_export.click()
assert wait_until(lambda: not win._job_running("align_export"),
timeout_ms=60000), "export finished"
assert wiz.state.exported_path == str(out), p3.lbl_status.text()
assert p3.isComplete(), "Finish available once the file exists"
assert out.exists()
written = SrasFile(str(out))
ctx.written = written
assert written.version == 6, "export is a v6 file"
assert written.n_angles == ctx.s.n_angles
assert all(written.image_shape(a) == ctx.crop[2:]
for a in range(written.n_angles)), \
"every angle shares the cropped grid"
assert not out.with_name(out.name + ".part").exists(), \
"no staging file left behind"
def test_wizard_finish_applies_and_persists(ctx):
"""Finish makes the session match the file: Aligned View shows the exported
extent, and the sidecar records it for the input scan."""
win, wiz = ctx.win, ctx.wiz
wiz.accept()
pump(250)
assert win._align_wizard is None, "wizard reference released"
assert win._wizard_act.isEnabled(), "wizard action available again"
assert win._alignment_result is not None
assert win._alignment_result.canvas_shape == ctx.crop[2:], \
"the *cropped* result is what the view now uses"
assert win.chk_aligned_view.isEnabled() and win.chk_aligned_view.isChecked()
pump(200)
assert win.image_canvas._img_shape == ctx.crop[2:], \
f"canvas shows the cropped extent: {win.image_canvas._img_shape}"
sidecar = compute.sidecar_path(ctx.s.path)
assert sidecar.exists(), "sidecar written for the input scan"
ctx.sidecar = sidecar
ctx.sidecar_raw = raw = json.loads(sidecar.read_text())
assert raw.get("schema_version") == compute._SIDECAR_SCHEMA_VERSION
assert all(raw["per_angle"][str(a)]["rotation_deg"]
== win._alignment_result.per_angle[a].rotation_deg
for a in range(ctx.s.n_angles)), \
"sidecar round-trips the applied rotations"
win.chk_aligned_view.setChecked(False)
pump(200)
assert win.image_canvas._img_shape == ctx.s.image_shape(0), \
f"unchecking returns to the raw per-angle grid: {win.image_canvas._img_shape}"
def test_stale_schema_sidecar_ignored(ctx):
"""An old-schema sidecar (pre-pivot/sign fix) is treated as absent."""
s, raw, sidecar = ctx.s, ctx.sidecar_raw, ctx.sidecar
stale = dict(raw)
stale["schema_version"] = compute._SIDECAR_SCHEMA_VERSION - 1
sidecar.write_text(json.dumps(stale))
assert compute.load_manual_alignment(s) is None, \
"a sidecar with an old schema_version is not loaded"
sidecar.write_text(json.dumps(raw)) # restore for the rest of the sequence
def test_sidecar_restored_on_reload(ctx):
win, active = ctx.win, ctx.active
saved = json.loads(ctx.sidecar.read_text())["per_angle"][str(active)]
old_sras_id = id(win._sras)
win._load_file(str(ctx.path)) # reload the same file fresh
assert wait_until(
lambda: win._sras is not None and id(win._sras) != old_sras_id), \
"file reloaded"
ctx.s = win._sras
assert win._align_wizard is None, "no wizard left open across a reload"
assert win._alignment_result is not None, \
"reload restores the saved alignment automatically"
assert abs(win._alignment_result.per_angle[active].rotation_deg
- saved["rotation_deg"]) < 1e-9, \
"restored rotation matches what was saved"
assert win.chk_aligned_view.isChecked(), \
"Aligned View auto-checked after restoring a saved alignment"
def test_wizard_closes_with_a_reload(ctx):
"""An open wizard belongs to the file it was opened on."""
win = ctx.win
win._on_alignment_wizard()
assert win._align_wizard is not None
assert wait_until(
lambda: win._align_wizard.page(win._align_wizard.PAGE_CORRELATE).isComplete())
win._load_file(str(ctx.path))
assert wait_until(lambda: not win._job_running("load"))
pump(200)
assert win._align_wizard is None, "wizard force-closed by a reload"
ctx.s = win._sras
# ---------------------------------------------------------------------------
# Export Fused ROI
# ---------------------------------------------------------------------------
#
# Two independent ways angles can end up sharing one (x, y) grid to fuse
# onto: a live alignment result (case a, exercised on ctx.win -- the reload
# above restored one from the sidecar), or a file that is itself a previous
# Alignment Wizard export, whose angles already share a grid on disk with no
# alignment result needed at all (case b, exercised on a second window
# opened on ctx.written from test_wizard_export).
def test_fused_export_gating_case_a(ctx):
"""A live alignment result bridges the raw scan's per-angle grids --
angles_share_raw_grid() alone would be False here."""
win, s = ctx.win, ctx.s
assert win._alignment_result is not None, "alignment restored from sidecar"
assert not s.angles_share_raw_grid(), \
"sanity check: the raw (un-aligned) scan must not already share a grid"
x, y = win._aligned_canvas_axes()
roi = RoiQuad.from_bbox(float(x[1]), float(y[1]), float(x[-2]), float(y[-2]))
win.image_canvas.set_roi(roi)
pump(120)
assert win._fused_grid_ready()
assert win.btn_export_fused_roi.isEnabled()
def test_write_fused_roi_csv_case_a_content(ctx):
win, s = ctx.win, ctx.s
roi = win.image_canvas.get_roi()
assert roi is not None, "ROI drawn by test_fused_export_gating_case_a"
csv_path = ctx.tmpdir / "fused_roi_case_a.csv"
win._write_fused_roi_csv(roi, CH4_IDX, [0, 1], str(csv_path))
assert csv_path.exists()
lines = csv_path.read_text().splitlines()
body = [l for l in lines if not l.startswith("#")]
header, *data_lines = body
assert header == (
f"x_mm,y_mm,v_{s.angles_deg[0]:.4g}deg,v_{s.angles_deg[1]:.4g}deg")
x, y = win._fused_export_axes()
mask = roi.mask_for_grid(x, y)
assert len(data_lines) == int(mask.sum())
data = np.array([[float(v) for v in line.split(",")] for line in data_lines])
expect0 = win._fused_value_image(0, CH4_IDX)[mask]
expect1 = win._fused_value_image(1, CH4_IDX)[mask]
assert np.allclose(data[:, 2], expect0, rtol=1e-5, atol=1e-4)
assert np.allclose(data[:, 3], expect1, rtol=1e-5, atol=1e-4)
def test_fused_roi_dialog_availability_live_updates(ctx):
"""Switching the value-type radio re-evaluates every angle checkbox,
disabling/auto-unchecking whichever ones are no longer available --
independent of what actually backs availability_fn, so a synthetic
stand-in keeps this a fast, deterministic test of the dialog itself."""
win, s = ctx.win, ctx.s
angles = [(a, float(s.angles_deg[a])) for a in range(s.n_angles)]
only_angle0_has_ch1 = lambda a, c: (a == 0) if c == CH1_IDX else True
dlg = FusedRoiExportDialog(
win, angles=angles, availability_fn=only_angle0_has_ch1,
default_ch_idx=CH1_IDX, out_dir=str(s.path.parent), stem=s.path.stem,
grid_note="test")
try:
assert dlg._angle_checks[0].isEnabled()
assert all(not dlg._angle_checks[a].isEnabled()
for a in range(1, s.n_angles))
dlg._angle_checks[0].setChecked(True)
dlg._val_buttons[CH4_IDX].click()
assert all(dlg._angle_checks[a].isEnabled() for a in range(s.n_angles)), \
"CH4 is available for every angle"
assert dlg._angle_checks[0].isChecked(), \
"stays checked -- still available under CH4"
if s.n_angles > 1:
dlg._angle_checks[1].setChecked(True)
dlg._val_buttons[CH1_IDX].click()
assert dlg._angle_checks[0].isChecked()
if s.n_angles > 1:
assert not dlg._angle_checks[1].isEnabled()
assert not dlg._angle_checks[1].isChecked(), \
"auto-unchecked: angle 1 has no data under CH1"
finally:
dlg.close()
def test_fused_roi_dialog_select_all_none(ctx):
win, s = ctx.win, ctx.s
angles = [(a, float(s.angles_deg[a])) for a in range(s.n_angles)]
dlg = FusedRoiExportDialog(
win, angles=angles, availability_fn=lambda a, c: c == CH4_IDX,
default_ch_idx=CH4_IDX, out_dir=str(s.path.parent), stem=s.path.stem,
grid_note="test")
try:
assert not dlg._btn_export.isEnabled(), "nothing checked yet"
dlg._on_select_all_available()
assert all(cb.isChecked() for cb in dlg._angle_checks.values())
assert dlg._btn_export.isEnabled()
dlg._on_select_none()
assert not any(cb.isChecked() for cb in dlg._angle_checks.values())
assert not dlg._btn_export.isEnabled()
finally:
dlg.close()
def test_on_export_fused_roi_csv_end_to_end(ctx):
win, s = ctx.win, ctx.s
roi = win.image_canvas.get_roi()
assert roi is not None, "ROI from the earlier fused-export tests is still set"
csv_path = ctx.tmpdir / "fused_roi_e2e.csv"
with patch("sras_viewer.main_window.FusedRoiExportDialog") as MockDlg:
inst = MockDlg.return_value
inst.exec.return_value = QDialog.DialogCode.Accepted
inst.get_ch_idx.return_value = CH4_IDX
inst.get_selected_angles.return_value = [0, 1]
inst.get_output_path.return_value = str(csv_path)
win.btn_export_fused_roi.click()
assert csv_path.exists()
kwargs = MockDlg.call_args.kwargs
assert kwargs["default_ch_idx"] == win.combo_channel.currentIndex()
assert kwargs["out_dir"] == str(s.path.parent)
assert kwargs["stem"] == s.path.stem
def test_fused_export_no_alignment_shared_grid_path(ctx):
"""ctx.written (from test_wizard_export) is itself a previous Alignment
Wizard export: opened fresh with no sidecar for its own path, so no
alignment result is ever restored -- but its angles already share one
grid on disk, so the export must work through the no-resample path."""
win2 = SrasViewerWindow()
try:
win2._load_file(str(ctx.written.path))
assert wait_until(lambda: win2._sras is not None)
s2 = win2._sras
assert win2._alignment_result is None, \
"no sidecar exists for this path -- nothing auto-restored"
assert s2.angles_share_raw_grid(), \
"a wizard export already shares one grid across angles"
assert wait_until(lambda: all((a, CH4_IDX) in win2._dc_cache
for a in range(s2.n_angles))), \
"DC precomputed for every angle"
x, y = s2.x_axis_mm(0), s2.y_positions_mm(0)
roi = RoiQuad.from_bbox(float(x[1]), float(y[1]), float(x[-2]), float(y[-2]))
win2.image_canvas.set_roi(roi)
pump(120)
assert win2._fused_grid_ready()
assert win2.btn_export_fused_roi.isEnabled()
aligned_cache_before = len(win2._aligned_cache)
angle_idxs = list(range(min(2, s2.n_angles)))
csv_path = ctx.tmpdir / "fused_roi_case_b.csv"
win2._write_fused_roi_csv(roi, CH4_IDX, angle_idxs, str(csv_path))
assert csv_path.exists()
assert len(win2._aligned_cache) == aligned_cache_before, \
"no-resample path must never touch apply_alignment"
header = next(l for l in csv_path.read_text().splitlines()
if not l.startswith("#"))
expected_header = "x_mm,y_mm," + ",".join(
f"v_{s2.angles_deg[a]:.4g}deg" for a in angle_idxs)
assert header == expected_header
# Break the shared grid and confirm gating flips off.
mutate_idx = 1 if s2.n_angles > 1 else 0
s2.x_start_mm[mutate_idx] += 1.0
assert not s2.angles_share_raw_grid()
win2._update_fused_export_enabled()
assert not win2._fused_grid_ready()
assert not win2.btn_export_fused_roi.isEnabled()
assert "Alignment Wizard" in win2.btn_export_fused_roi.toolTip()
finally:
win2.close()
pump(200)
def test_pixel_inspector(ctx):
win = ctx.win
win.chk_aligned_view.setChecked(False)
pump(100)
win._on_pixel_clicked(0, 0)
pump(150)
assert win.lbl_wave_hint.isHidden(), "waveform hint hidden after a click"
win.combo_channel.setCurrentIndex(CH1_IDX)
wait_until(lambda: not win._job_running("compute"))
win._on_pixel_clicked(1, 1)
pump(150)
assert len(win.wave_canvas.ax_wave.lines) > 0, \
f"RF waveform panel rendered ({len(win.wave_canvas.ax_wave.lines)} lines)"
def test_shutdown(ctx):
win = ctx.win
win.close()
pump(400)
assert len(win._jobs) == 0, f"all background jobs released: {list(win._jobs)}"
def test_no_status_bar_errors(ctx):
unexpected = [e for e in ctx.errors if e]
assert not unexpected, f"status-bar errors seen: {unexpected}"
+297
View File
@@ -0,0 +1,297 @@
"""Row-averaged FFT: same-row, distance-weighted CH1 waveform smoothing.
Covers the properties the design depends on: the kernel is symmetric and
n=0 is a true no-op; the masked/renormalized convolution matches an
independent brute-force reference and gives masked neighbors exactly zero
weight regardless of their content; background subtraction after averaging
is algebraically identical to subtracting before; chunking/worker count
never changes the result; and a pixel that's itself masked is never
"rescued" by averaging.
"""
import numpy as np
import pytest
import sras_compute as compute
from sras_compute import compute_rf_image, dc_image_mv
from sras_format import CH1_IDX, CH4_IDX, SrasFile
import tools.make_test_sras as gen
def _reference_row_average(masked_waves: np.ndarray, valid: np.ndarray,
weights: np.ndarray) -> np.ndarray:
"""Independent, unvectorized reference for _row_average_waveforms: for
each row position, sum weighted valid neighbors within the kernel's
radius and normalize by the actual included weight sum. Same
definition, computed by brute-force nested loops instead of
correlate1d, so it can't share a bug with the implementation."""
n_frames, spf = masked_waves.shape
n = len(weights) // 2
out = np.zeros_like(masked_waves)
for i in range(n_frames):
num = np.zeros(spf, dtype=np.float64)
den = 0.0
for d in range(-n, n + 1):
j = i + d
if 0 <= j < n_frames and valid[j]:
w = float(weights[d + n])
num += w * masked_waves[j].astype(np.float64)
den += w
out[i] = num / den if den > 0 else 0.0
return out
# ---------------------------------------------------------------------------
# _row_average_weights
# ---------------------------------------------------------------------------
def test_row_average_weights_shape_and_symmetry():
w0 = compute._row_average_weights(0)
assert w0.shape == (1,) and w0[0] == 1.0
for n in (1, 2, 5):
w = compute._row_average_weights(n)
assert w.shape == (2 * n + 1,)
assert w[n] == pytest.approx(1.0), "center tap is the peak weight"
assert np.allclose(w, w[::-1]), "symmetric about the center"
half = w[n:]
assert np.all(np.diff(half) < 0), "strictly decreasing away from center"
# ---------------------------------------------------------------------------
# _row_average_waveforms
# ---------------------------------------------------------------------------
def test_row_average_matches_hand_rolled_reference():
rng = np.random.default_rng(0)
n_frames, spf = 15, 6
raw = rng.integers(-50, 51, size=(n_frames, spf)).astype(np.float32)
valid = np.ones(n_frames, dtype=bool)
valid[[2, 3, 9]] = False # a run of two invalid, plus a lone invalid
masked = raw.copy()
masked[~valid] = 0.0
weights = compute._row_average_weights(3)
got = compute._row_average_waveforms(masked, valid, weights)
ref = _reference_row_average(masked, valid, weights)
assert np.allclose(got[valid], ref[valid], atol=1e-4)
def test_row_average_edge_of_row():
"""A window wider than the row itself must still renormalize correctly
at both ends -- mode='constant', cval=0.0 zero-pads both the numerator
and denominator, so this is not a special case, but it's the one most
likely to break if that padding were ever mismatched between the two."""
n_frames, spf = 6, 3
raw = np.arange(n_frames * spf, dtype=np.float32).reshape(n_frames, spf)
valid = np.ones(n_frames, dtype=bool)
weights = compute._row_average_weights(4) # window (9 taps) > n_frames (6)
got = compute._row_average_waveforms(raw, valid, weights)
ref = _reference_row_average(raw, valid, weights)
assert np.allclose(got, ref, atol=1e-4)
def test_row_average_excludes_masked_neighbor_from_normalization():
"""A masked neighbor must contribute zero *weight* to the normalization,
not participate as a legitimate zero-valued sample at full weight --
the two give different answers, and only the former is correct. (Note:
masked_waves must already be 0 at invalid positions per
_row_average_waveforms's contract -- that's what read_row's zero-filled
scratch buffer guarantees in production -- so the only way to vary
"what a masked position looks like" while respecting that contract is
whether its weight is excluded from the denominator at all.)"""
n_frames, spf = 9, 4
weights = compute._row_average_weights(2)
# A: position 4 is masked -- excluded from the weight sum entirely.
valid_a = np.ones(n_frames, dtype=bool)
valid_a[4] = False
masked_a = np.zeros((n_frames, spf), dtype=np.float32)
masked_a[valid_a] = 1.0
got_a = compute._row_average_waveforms(masked_a, valid_a, weights)
# B: position 4 is valid but genuinely zero-valued -- included in the
# weight sum, diluting neighbors' averages.
valid_b = np.ones(n_frames, dtype=bool)
masked_b = np.ones((n_frames, spf), dtype=np.float32)
masked_b[4] = 0.0
got_b = compute._row_average_waveforms(masked_b, valid_b, weights)
# Every position whose window reaches index 4 must average *higher* in
# A (excluded from the denominator) than in B (included as a real zero).
affected = [2, 3, 5, 6]
assert np.all(got_a[affected] > got_b[affected]), \
"masking must exclude a neighbor from normalization, not just zero its value"
# Positions outside the window (radius 2) are unaffected either way.
assert np.allclose(got_a[[0, 1, 7, 8]], got_b[[0, 1, 7, 8]])
def test_background_subtracted_once_equals_subtract_then_average():
"""Algebraic identity the implementation relies on: subtracting a fixed
background from the already-averaged waveform equals subtracting it
from every valid neighbor first, because the denominator is always the
*actual* included weight sum (never a fixed total)."""
rng = np.random.default_rng(1)
n_frames, spf = 11, 8
raw = rng.integers(-40, 41, size=(n_frames, spf)).astype(np.float32)
valid = np.ones(n_frames, dtype=bool)
valid[[1, 7]] = False
masked = raw.copy()
masked[~valid] = 0.0
background = rng.integers(-5, 6, size=spf).astype(np.float32)
weights = compute._row_average_weights(3)
# Order A (what the code does): average first, subtract background once.
order_a = compute._row_average_waveforms(masked, valid, weights) - background
# Order B: subtract background from every valid neighbor first (restoring
# the "0 at invalid positions" contract afterward), then average.
bg_subbed = masked - background
bg_subbed[~valid] = 0.0
order_b = compute._row_average_waveforms(bg_subbed, valid, weights)
assert np.allclose(order_a[valid], order_b[valid], atol=1e-3)
# ---------------------------------------------------------------------------
# compute_rf_image(row_avg_n=...) integration
# ---------------------------------------------------------------------------
def test_row_average_zero_is_identity(tmp_path):
"""row_avg_n=0 must take the exact same code path as before this
feature existed (row_avg_weights stays None), not a single-tap kernel
that merely computes to the same answer."""
path = tmp_path / "zero.sras"
gen.write(path, n_angles=1, seed=10, samples_per_frame=64)
sras = SrasFile(str(path))
plain = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True)
explicit_zero = compute_rf_image(sras, 0, dc_threshold_mv=None,
apply_bg_sub=True, row_avg_n=0)
assert np.array_equal(plain, explicit_zero)
def test_row_average_respects_own_center_mask(tmp_path):
"""A pixel that's itself below threshold stays masked (0) after row
averaging -- averaging never rescues a masked pixel, matching the
'valid neighbors only' design (masked pixels are excluded from other
pixels' averages, and are never themselves smoothed)."""
path = tmp_path / "center_mask.sras"
gen.write(path, n_angles=1, seed=13, samples_per_frame=64)
sras = SrasFile(str(path))
dc4 = dc_image_mv(sras, 0, CH4_IDX)
thr = float(np.percentile(dc4, 50))
mask = dc4 >= thr
assert mask.any() and not mask.all(), "threshold actually splits the image"
img = compute_rf_image(sras, 0, dc_threshold_mv=thr, apply_bg_sub=True,
row_avg_n=4)
assert np.array_equal(img == 0, ~mask), \
"masked pixels stay exactly 0 after row averaging; valid ones don't"
def test_row_average_composes_with_padding(tmp_path):
"""row_avg_n and n_fft (zero-padding) are independent knobs: using them
together must not raise, and must agree bit-for-bit with an independent
reference that row-averages the raw waveforms and background-subtracts
them, then runs a plain scipy rfft + argmax at the same pad factor --
i.e. row-averaging composes correctly with the padded peak search."""
import scipy.fft as scipy_fft
path = tmp_path / "padded_rowavg.sras"
gen.write(path, n_angles=1, seed=12, samples_per_frame=64)
sras = SrasFile(str(path))
spf = sras.samples_per_frame
n_rows, n_frames = sras.image_shape(0)
n_fft = spf * 40
raw_natural = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True)
avg_natural = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True,
row_avg_n=3)
avg_padded = compute_rf_image(sras, 0, dc_threshold_mv=None, apply_bg_sub=True,
row_avg_n=3, n_fft=n_fft)
assert avg_natural.shape == raw_natural.shape == avg_padded.shape
assert np.all(np.isfinite(avg_padded))
weights = compute._row_average_weights(3)
freq32 = sras.freq_axis_mhz(n_fft).astype(np.float32)
data = sras.data[0]
expected = np.zeros((n_rows, n_frames), dtype=np.float32)
for r in range(n_rows):
v = np.ones(n_frames, dtype=bool)
avg = compute._row_average_waveforms(
data[r, CH1_IDX].astype(np.float32), v, weights)
avg = avg - sras.background
S = scipy_fft.rfft(avg, n=n_fft, axis=-1, workers=1)
power = S.real ** 2 + S.imag ** 2
power[:, 0] = 0.0
expected[r] = freq32[np.argmax(power, axis=1)]
assert np.array_equal(avg_padded, expected), \
"row-averaged waveforms feed the padded peak search identically " \
"to an independent reference"
def test_row_average_improves_snr_recovery():
"""The actual point of the feature: averaging same-row waveforms that
share a true underlying tone but carry independent noise recovers that
tone far more reliably than any single raw (unaveraged) waveform does."""
rng = np.random.default_rng(42)
n_frames, spf = 21, 128
true_bin = 9
t = np.arange(spf)
tone = 15.0 * np.sin(2 * np.pi * true_bin * t / spf) # same true signal
# at every position
noise_sigma = 40.0 # much larger than the tone -- deliberately poor SNR
raw = (tone[None, :] + rng.normal(scale=noise_sigma, size=(n_frames, spf))
).astype(np.float32)
valid = np.ones(n_frames, dtype=bool)
weights = compute._row_average_weights(8) # wide window: lots of averaging
averaged = compute._row_average_waveforms(raw, valid, weights)
raw_bins = compute._peak_bins(raw, spf)
avg_bins = compute._peak_bins(averaged, spf)
raw_hits = int(np.sum(raw_bins == true_bin))
avg_hits = int(np.sum(avg_bins == true_bin))
assert avg_hits > raw_hits, (
f"row averaging should recover the true bin ({true_bin}) more often "
f"than raw per-pixel estimates: raw {raw_hits}/{n_frames}, "
f"averaged {avg_hits}/{n_frames}")
assert avg_hits >= n_frames * 0.7, \
f"averaged recovery should be reliable, not just barely better: {avg_hits}/{n_frames}"
def test_row_average_parallel_identity(tmp_path, monkeypatch):
"""Forcing 1 worker vs many must give an identical row-averaged image --
catches chunk-boundary bugs (there should be none, since averaging never
crosses rows, but this is the empirical proof, not just inspection)."""
path = tmp_path / "parallel_rowavg.sras"
n_rows, n_frames, spf = 40, 13, 128
gen.write(path, n_angles=1, seed=11, samples_per_frame=spf,
geometry=[(n_rows, n_frames)])
sras = SrasFile(str(path))
monkeypatch.setattr(compute, "_TOTAL_BYTES_BUDGET", 8 * n_frames * spf * 4)
monkeypatch.setattr(compute, "_FFT_BLOCK_MAX", 4)
# row_avg_n > 0 halves the effective budget before chunk planning.
fft_rows = compute._plan_fft_rows(n_frames, spf, compute._TOTAL_BYTES_BUDGET // 2)
assert fft_rows < n_rows, \
f"row-averaged FFT work actually splits into multiple chunks ({fft_rows} of {n_rows})"
dc4 = dc_image_mv(sras, 0, CH4_IDX)
thr = float(np.median(dc4))
monkeypatch.setattr(compute, "_MAX_WORKERS", 1)
serial = compute_rf_image(sras, 0, dc_threshold_mv=thr, apply_bg_sub=True,
row_avg_n=4)
monkeypatch.setattr(compute, "_MAX_WORKERS", 8)
parallel = compute_rf_image(sras, 0, dc_threshold_mv=thr, apply_bg_sub=True,
row_avg_n=4)
assert np.array_equal(serial, parallel), \
"row-averaged rf image identical regardless of chunking/worker count"
+951
View File
@@ -0,0 +1,951 @@
"""Does a batch-computed FFT cache actually spare the viewer the FFT?
Storing a peak-frequency image per angle in the file is only worth doing if
displaying it is then free. The regression this module pins down is the
viewer's *dispatch* decision: it used to find the stored image only inside
ComputeWorker, so after a batch every angle change still queued a background
job behind a "Computing FFT…" popup for an image already on disk.
Both layers are covered — cached_rf_image's accept/reject rules, and the
window never reaching _start_compute for a batch-cached angle.
"""
import struct
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
import pytest
from PyQt6.QtCore import QEventLoop, QTimer
from PyQt6.QtWidgets import QApplication, QDialog
import sras_compute as compute
import sras_format as fmt
from sras_compute import cache_file, cached_rf_image, compute_rf_image, dc_image_mv
from sras_format import CH1_IDX, CH3_IDX, CH4_IDX, SrasFile
from sras_viewer import SrasViewerWindow, VELOCITY_MODE_IDX
import tools.make_test_sras as gen
_THRESHOLD_MV = 50.0 # the viewer's own default
def pump(ms: int = 200):
loop = QEventLoop()
QTimer.singleShot(ms, loop.quit)
loop.exec()
def wait_until(pred, timeout_ms: int = 20000, step: int = 100) -> bool:
waited = 0
while waited < timeout_ms:
if pred():
return True
pump(step)
waited += step
return pred()
@pytest.fixture(scope="module")
def rig(tmp_path_factory):
"""A v6 file, the FFT images a from-scratch compute gives for it, and the
same file after Batch Compute FFT has written them into its v7 cache."""
path = tmp_path_factory.mktemp("stored_cache") / "cached.sras"
gen.write(path, n_angles=4, seed=7, samples_per_frame=256)
src = SrasFile(str(path))
fresh = {a: compute_rf_image(src, a, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True)
for a in range(src.n_angles)}
assert src.background is not None, "the fixture file must have a background"
err = cache_file(str(path), "fft", True)
assert err == "", err
cached = SrasFile(str(path))
assert all(x is not None for x in cached.precomputed_freq_mhz)
assert all(x is None for x in cached.precomputed_dc4_mv), \
"FFT-only batch: the mask has to come from the viewer, not the file"
return SimpleNamespace(path=path, fresh=fresh, sras=cached,
n_angles=cached.n_angles)
@pytest.fixture(scope="module")
def dc_rig(tmp_path_factory):
"""A file that has been through Batch Compute DC and Store."""
path = tmp_path_factory.mktemp("stored_dc") / "dc_cached.sras"
gen.write(path, n_angles=4, seed=9, samples_per_frame=128)
err = cache_file(str(path), "dc", True)
assert err == "", err
sras = SrasFile(str(path))
assert all(x is not None for x in sras.precomputed_dc3_mv)
assert all(x is not None for x in sras.precomputed_dc4_mv)
assert len(np.unique(sras.precomputed_dc4_mv[0])) > 1, \
"a degenerate DC image would make the comparisons below vacuous"
return SimpleNamespace(path=path, sras=sras, n_angles=sras.n_angles)
@pytest.fixture
def no_fft(monkeypatch):
"""Make any real FFT work loud: returns a list that stays empty unless a
peak search actually runs."""
calls = []
for name in ("_peak_bins",):
original = getattr(compute, name)
def spy(*args, _f=original, **kwargs):
calls.append(_f.__name__)
return _f(*args, **kwargs)
monkeypatch.setattr(compute, name, spy)
return calls
# ---------------------------------------------------------------------------
# cached_rf_image: when may the stored image stand in for a compute?
# ---------------------------------------------------------------------------
def test_stored_image_matches_a_fresh_compute(rig, no_fft):
for a in range(rig.n_angles):
dc4 = dc_image_mv(SrasFile(str(rig.path)), a, CH4_IDX)
img = cached_rf_image(rig.sras, a, dc_threshold_mv=_THRESHOLD_MV,
apply_bg_sub=True, dc4_mv=dc4)
assert img is not None, f"angle {a} is cached in the file"
assert np.allclose(img, rig.fresh[a], atol=1e-3), \
f"angle {a} differs from a from-scratch compute"
assert not no_fft, f"the stored image was used, no FFT ran: {no_fft}"
def test_unmasked_when_no_threshold(rig):
img = cached_rf_image(rig.sras, 0, dc_threshold_mv=None, apply_bg_sub=True)
assert img is not None and np.array_equal(img, rig.sras.precomputed_freq_mhz[0])
img[:] = -1.0
assert not np.any(rig.sras.precomputed_freq_mhz[0] == -1.0), \
"callers get a copy, never the file's own array"
def test_settings_the_stored_image_cannot_serve(rig):
"""A stored image carries one bg-sub state and one padding, so anything
else must fall through to a real compute rather than lie."""
spf = rig.sras.samples_per_frame
assert cached_rf_image(rig.sras, 0, _THRESHOLD_MV, apply_bg_sub=True,
n_fft=spf * 4) is None, \
"cache was written at pad 1, a pad-4 view resolves different peaks"
assert cached_rf_image(rig.sras, 0, _THRESHOLD_MV, apply_bg_sub=False) is None, \
"cache was written with bg-sub on"
uncached = SrasFile(str(rig.path))
uncached.precomputed_freq_mhz[1] = None
assert cached_rf_image(uncached, 1, _THRESHOLD_MV, apply_bg_sub=True) is None
@pytest.mark.parametrize("pad", [2, 10])
def test_cache_is_stored_at_the_configured_pad(tmp_path, pad):
"""Batching at a pad factor must produce a cache that view can read back —
a pad-1-only cache is one the padded viewer can never use."""
path = tmp_path / f"pad{pad}.sras"
gen.write(path, n_angles=2, seed=13, samples_per_frame=256)
spf = SrasFile(str(path)).samples_per_frame
assert cache_file(str(path), "fft", True, 0, pad) == ""
sras = SrasFile(str(path))
assert sras.precomputed_pad_factor == pad, "pad factor survives the round trip"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True,
n_fft=spf * pad) is not None, f"usable at pad {pad}"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True) is None, \
"not usable unpadded"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True,
n_fft=spf * (pad + 1)) is None, "not usable at another pad"
# The stored numbers must be the padded ones, not pad-1 relabelled.
fresh = SrasFile(str(path))
fresh.precomputed_freq_mhz = [None] * fresh.n_angles
for a in range(sras.n_angles):
assert np.allclose(
compute_rf_image(sras, a, dc_threshold_mv=None, apply_bg_sub=True,
n_fft=spf * pad),
compute_rf_image(fresh, a, dc_threshold_mv=None, apply_bg_sub=True,
n_fft=spf * pad), atol=1e-3), \
f"angle {a}: stored image is the pad-{pad} answer"
def test_cach_v1_reads_as_natural_resolution(tmp_path):
"""Files cached before the pad factor existed must keep working: a v1 tail
has no pad field and is pad 1 by construction."""
path = tmp_path / "v1.sras"
gen.write(path, n_angles=2, seed=14, samples_per_frame=256)
assert cache_file(str(path), "fft", True) == ""
# Rewrite the tail as a genuine CACH v1 block (old header, no pad field).
v2 = SrasFile(str(path))
freq, entries = v2.precomputed_freq_mhz, list(range(v2.n_angles))
tail_offset = v2._cache_tail_offset()
payload = struct.pack(fmt.CACH_HDR_FMT, fmt.CACH_MAGIC, 1, fmt.CACH_FLAG_FFT)
payload += struct.pack(fmt.SFFT_HDR_FMT_V1, fmt.SFFT_MAGIC,
fmt.SFFT_FLAG_BG_SUB, len(entries))
for a in entries:
payload += struct.pack(">H", a) + freq[a].astype(">f4").tobytes()
# Windows refuses to truncate a file with a live mapping (write_bytes
# opens 'wb'), and every SrasFile holds its waveform memmaps for life —
# drop the instance first. The parsed freq arrays are plain copies and
# stay usable.
del v2
head = path.read_bytes()[:tail_offset]
path.write_bytes(head + payload)
v1 = SrasFile(str(path))
assert v1.precomputed_pad_factor == 1
assert v1.precomputed_bg_sub is True
assert all(np.array_equal(v1.precomputed_freq_mhz[a], freq[a]) for a in entries), \
"v1 images read back unchanged"
assert cached_rf_image(v1, 0, None, apply_bg_sub=True) is not None
assert cached_rf_image(v1, 0, None, apply_bg_sub=True,
n_fft=v1.samples_per_frame * 10) is None
def test_mask_read_can_be_refused(rig, no_fft):
"""With no DC4 in hand, building the mask means reading a whole channel —
the GUI thread asks for None instead."""
assert cached_rf_image(rig.sras, 0, _THRESHOLD_MV, apply_bg_sub=True,
allow_dc_recompute=False) is None
dc4 = dc_image_mv(SrasFile(str(rig.path)), 0, CH4_IDX)
img = cached_rf_image(rig.sras, 0, _THRESHOLD_MV, apply_bg_sub=True,
dc4_mv=dc4, allow_dc_recompute=False)
assert img is not None and np.allclose(img, rig.fresh[0], atol=1e-3)
assert not no_fft, f"no FFT on either branch: {no_fft}"
# ---------------------------------------------------------------------------
# The viewer: no compute job at all for a batch-cached angle
# ---------------------------------------------------------------------------
def test_viewer_shows_stored_angles_without_computing(rig, no_fft, monkeypatch):
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._load_file(str(rig.path))
assert wait_until(lambda: win._sras is not None), "file loaded"
# The file carries no DC block, so the mask comes from the window's own
# background precompute — the state a user is in by the time they click.
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(rig.n_angles))), \
"DC precompute finished"
dispatched.clear()
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX), "CH1 displayed"
for a in list(range(rig.n_angles)) + [1, 0]:
win.spin_angle.setValue(a)
win._on_view_changed()
pump(60)
assert win._current_angle == a, f"angle {a} displayed"
assert np.allclose(win._current_image, rig.fresh[a], atol=1e-3), \
f"angle {a} shows the stored image"
assert dispatched == [], \
f"stored angles need no compute job, dispatched for {dispatched}"
assert not no_fft, f"no FFT ran for any stored angle: {no_fft}"
# Velocity is still a post-multiply of the same stored image.
win.combo_channel.setCurrentIndex(VELOCITY_MODE_IDX)
pump(120)
assert np.allclose(win._current_image,
rig.fresh[0] * win.spin_grating_um.value(), atol=1e-3)
assert dispatched == [] and not no_fft
# ...and a live control that no longer matches the stored image's own
# provenance must NOT force a recompute either — the stored image is
# shown as-is; only an explicit batch recompute changes what's shown.
win.combo_channel.setCurrentIndex(CH1_IDX)
pump(60)
win.chk_bg_sub.setChecked(False)
pump(200)
assert not win._job_running("compute") and not no_fft, \
"bg-sub off still shows the stored image, no real FFT"
assert np.allclose(win._current_image, rig.fresh[0], atol=1e-3)
finally:
win.close()
pump(300)
def test_batch_caches_at_the_viewers_pad_factor(tmp_path, no_fft, monkeypatch):
"""The bug a pad-10 user hits: Batch Compute FFT used to store pad-1
images regardless, so the padded view recomputed every angle forever.
"""
path = tmp_path / "padded_gui.sras"
gen.write(path, n_angles=3, seed=15, samples_per_frame=256)
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._fft_pad_factor = 10
win._load_file(str(path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(win._sras.n_angles))), \
"DC precompute finished"
# Convert -> Batch Compute FFT, on the open file, through the real slot.
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")):
win._on_batch_compute("fft")
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
assert wait_until(lambda: win._sras is not None
and win._sras.version == 7), "file reloaded as v7"
pump(200)
assert win._sras.precomputed_pad_factor == 10, \
f"cached at the viewer's pad, got {win._sras.precomputed_pad_factor}"
expected = {a: compute.cached_rf_image(
win._sras, a, dc_threshold_mv=win.spin_threshold_mv.value(),
apply_bg_sub=win.chk_bg_sub.isChecked(),
n_fft=win._current_n_fft(),
dc4_mv=win._dc_cache.get((a, CH4_IDX)))
for a in range(win._sras.n_angles)}
assert all(v is not None for v in expected.values()), "cache is readable at pad 10"
no_fft.clear()
dispatched.clear()
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX), "CH1 displayed"
for a in range(win._sras.n_angles):
win.spin_angle.setValue(a)
pump(60)
assert np.allclose(win._current_image, expected[a], atol=1e-3), \
f"angle {a} served from the pad-10 cache"
assert dispatched == [] and not no_fft, \
f"no recompute at pad 10 (jobs={dispatched}, fft={no_fft})"
assert "unusable" not in win.lbl_frame_warn.text()
# Change the live pad control so it no longer matches the stored
# image's own provenance — the info panel has to say so rather than
# leave it a mystery, but the stored pad-10 image keeps displaying;
# only an explicit batch recompute would ever produce a pad-4 one.
win._fft_pad_factor = 4
win._update_scan_info_labels()
assert "differ from the stored cache" in win.lbl_frame_warn.text(), \
win.lbl_frame_warn.text()
assert "pad 10x" in win.lbl_frame_warn.text()
last_angle = win._current_angle
win._refresh_display()
assert wait_until(lambda: not win._job_running("compute")), "settled"
assert not no_fft, "no real FFT ran — the pad-10 cache still served the view"
assert np.allclose(win._current_image, expected[last_angle], atol=1e-3), \
"pad-10 cache still shown after the live pad control diverged"
finally:
win.close()
pump(300)
def test_viewer_shows_stored_dc_without_computing(dc_rig, monkeypatch):
"""Same for the DC half, with the background precompute silenced so the
file's stored block is the only thing that can be carrying the display."""
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
monkeypatch.setattr(type(win), "_start_dc_precompute", lambda self: None)
try:
win._load_file(str(dc_rig.path))
assert wait_until(lambda: win._sras is not None), "file loaded"
pump(120)
for ch in (CH3_IDX, CH4_IDX):
win.combo_channel.setCurrentIndex(ch)
for a in range(dc_rig.n_angles):
win.spin_angle.setValue(a)
pump(60)
assert (win._current_angle, win._current_ch) == (a, ch), \
f"angle {a} on channel {ch} displayed"
assert np.array_equal(win._current_image,
dc_rig.sras.cached_dc_mv(a, ch)), \
f"angle {a} channel {ch} shows the file's stored DC image"
assert dispatched == [], \
f"stored DC angles need no compute job, dispatched for {dispatched}"
finally:
win.close()
pump(300)
# ---------------------------------------------------------------------------
# Row-averaged FFT: cache_file("fft_rowavg", ...) and its on-disk provenance
# ---------------------------------------------------------------------------
def test_row_average_flag_and_n_round_trip(tmp_path):
path = tmp_path / "rowavg_roundtrip.sras"
gen.write(path, n_angles=2, seed=20, samples_per_frame=128)
err = cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=-1e9, row_avg_n=5)
assert err == "", err
sras = SrasFile(str(path))
assert sras.version == 7
assert sras.precomputed_row_avg_n == 5
assert all(x is not None for x in sras.precomputed_freq_mhz)
def test_fft_rowavg_mode_requires_positive_n_and_threshold(tmp_path):
path = tmp_path / "rowavg_bad_args.sras"
gen.write(path, n_angles=1, seed=27, samples_per_frame=64)
assert cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=0.0, row_avg_n=0)
assert cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=None, row_avg_n=5)
def test_raw_and_row_averaged_caches_never_cross_served(tmp_path):
"""The central regression this feature must never allow: a raw request
served a row-averaged image (or vice versa), or a request at one window
size served a cache stored at a different one."""
path = tmp_path / "cross_serve.sras"
gen.write(path, n_angles=1, seed=21, samples_per_frame=128)
err = cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=-1e9, row_avg_n=5)
assert err == "", err
sras = SrasFile(str(path))
assert cached_rf_image(sras, 0, None, apply_bg_sub=True, row_avg_n=0) is None, \
"a raw request must not be served a row-averaged cache"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True, row_avg_n=3) is None, \
"a request at the wrong window size must not be served either"
served = cached_rf_image(sras, 0, None, apply_bg_sub=True, row_avg_n=5)
assert served is not None
assert np.array_equal(served, sras.precomputed_freq_mhz[0])
def test_write_v7_cache_row_avg_n_carries_forward(tmp_path):
"""A later DC-only write must leave a previously-written row-averaged
FFT block -- including its row_avg_n -- byte-for-byte unchanged."""
path = tmp_path / "carry_forward.sras"
gen.write(path, n_angles=2, seed=22, samples_per_frame=64)
err = cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=-1e9, row_avg_n=7)
assert err == "", err
before = SrasFile(str(path))
assert before.precomputed_row_avg_n == 7
freq_before = [x.copy() for x in before.precomputed_freq_mhz]
err = cache_file(str(path), "dc", True)
assert err == "", err
after = SrasFile(str(path))
assert after.precomputed_row_avg_n == 7, "row_avg_n survives a DC-only write"
assert all(np.array_equal(after.precomputed_freq_mhz[a], freq_before[a])
for a in range(after.n_angles)), \
"the row-averaged FFT block itself is untouched by a DC-only write"
def test_row_average_never_touches_dc_images(tmp_path):
path = tmp_path / "dc_untouched.sras"
gen.write(path, n_angles=2, seed=23, samples_per_frame=64)
src = SrasFile(str(path))
expect_dc3 = [dc_image_mv(src, a, CH3_IDX) for a in range(src.n_angles)]
expect_dc4 = [dc_image_mv(src, a, CH4_IDX) for a in range(src.n_angles)]
assert cache_file(str(path), "dc", True) == ""
assert cache_file(str(path), "fft_rowavg", True,
dc_threshold_mv=-1e9, row_avg_n=6) == ""
after = SrasFile(str(path))
assert all(np.allclose(after.precomputed_dc3_mv[a], expect_dc3[a], atol=1e-4)
for a in range(after.n_angles))
assert all(np.allclose(after.precomputed_dc4_mv[a], expect_dc4[a], atol=1e-4)
for a in range(after.n_angles))
def test_row_average_never_modifies_raw_waveform_data(tmp_path):
path = tmp_path / "waveform_untouched.sras"
gen.write(path, n_angles=2, seed=24, samples_per_frame=64)
orig = tmp_path / "waveform_untouched_orig.sras"
gen.write(orig, n_angles=2, seed=24, samples_per_frame=64)
assert cache_file(str(path), "fft_rowavg", True,
dc_threshold_mv=-1e9, row_avg_n=5) == ""
o, n = SrasFile(str(orig)), SrasFile(str(path))
assert all(np.array_equal(np.asarray(o.data[a]), np.asarray(n.data[a]))
for a in range(o.n_angles)), \
"waveform data untouched by a row-averaged cache write"
def test_cach_v1_backward_compat_defaults_row_avg_n_zero(tmp_path):
"""A v1 CACH tail predates row-averaged FFT caching entirely (no
row_avg_n byte at all) -- readers must still parse it in full, treating
it as row_avg_n=0. This is what protects an existing real-world v7
file's already-stored FFT cache from silently becoming unusable after
this change ships."""
path = tmp_path / "v1_rowavg.sras"
gen.write(path, n_angles=2, seed=25, samples_per_frame=128)
assert cache_file(str(path), "fft", True) == ""
v2 = SrasFile(str(path))
freq, entries = v2.precomputed_freq_mhz, list(range(v2.n_angles))
tail_offset = v2._cache_tail_offset()
payload = struct.pack(fmt.CACH_HDR_FMT, fmt.CACH_MAGIC, 1, fmt.CACH_FLAG_FFT)
payload += struct.pack(fmt.SFFT_HDR_FMT_V1, fmt.SFFT_MAGIC,
fmt.SFFT_FLAG_BG_SUB, len(entries))
for a in entries:
payload += struct.pack(">H", a) + freq[a].astype(">f4").tobytes()
# See test_cach_v1_reads_as_natural_resolution: release the memmaps
# before write_bytes truncates, or Windows raises EINVAL.
del v2
head = path.read_bytes()[:tail_offset]
path.write_bytes(head + payload)
v1 = SrasFile(str(path))
assert v1.precomputed_row_avg_n == 0
assert all(np.array_equal(v1.precomputed_freq_mhz[a], freq[a]) for a in entries), \
"v1 images read back unchanged"
assert cached_rf_image(v1, 0, None, apply_bg_sub=True, row_avg_n=0) is not None
assert cached_rf_image(v1, 0, None, apply_bg_sub=True, row_avg_n=5) is None, \
"a v1 tail (predating this feature) can never satisfy a row-averaged request"
# ---------------------------------------------------------------------------
# Min peak frequency floor: serve-time masking, on-disk provenance, batch
# ---------------------------------------------------------------------------
def _biting_floor(stored: np.ndarray) -> float:
"""A floor that zeroes some-but-not-all of *stored*'s positive peaks:
the median distinct positive value, so pixels below it get masked and
pixels at/above it survive (the mask is a strict <)."""
positive = np.unique(stored[stored > 0])
assert len(positive) >= 2, "fixture must have varied peak frequencies"
return float(positive[len(positive) // 2])
def test_stored_image_served_with_raised_floor_masked(rig, no_fft):
"""The core of the 5 m/s bug fix: a floor above the stored one (here 0)
is re-applied when the stored image is served — pixels whose stored
peak falls below it come back as the 0.0 invalid sentinel, everything
else passes through, and no FFT runs. Both directly through
cached_rf_image and through compute_rf_image's fast path."""
assert rig.sras.precomputed_min_freq_mhz == 0.0, "batched without a floor"
stored = rig.sras.precomputed_freq_mhz[0]
floor = _biting_floor(stored)
expect = np.where(stored < floor, np.float32(0.0), stored)
img = cached_rf_image(rig.sras, 0, None, apply_bg_sub=True,
min_freq_mhz=floor)
assert img is not None, "an equal-or-higher floor is servable"
assert np.array_equal(img, expect)
assert (img == 0).any() and (img > 0).any(), \
"the floor bites some pixels but not all"
via_compute = compute_rf_image(rig.sras, 0, dc_threshold_mv=None,
apply_bg_sub=True, min_freq_mhz=floor)
assert np.array_equal(via_compute, expect), \
"compute_rf_image's fast path applies the same serve-time mask"
assert not no_fft, f"serving + masking must not run an FFT: {no_fft}"
def test_min_freq_floor_round_trips_and_gates_serving(tmp_path):
"""cache_file records the floor in the SFFT header and the accept rule
is asymmetric: an equal-or-higher request is servable, a lower one is
refused (the stored search never looked below its floor). 20.1 pins the
fixed-point kHz encoding — a float32 header field would read back as
20.10000038…, above the requested 20.1, and mismatch forever."""
path = tmp_path / "floor_roundtrip.sras"
gen.write(path, n_angles=2, seed=31, samples_per_frame=128)
floor = 20.1
assert cache_file(str(path), "fft", True, min_freq_mhz=floor) == ""
sras = SrasFile(str(path))
assert sras.precomputed_min_freq_mhz == floor, "exact fixed-point round-trip"
assert all(((img == 0) | (img >= floor)).all()
for img in sras.precomputed_freq_mhz), \
"no stored peak below the floor"
def reasons(f):
return compute.cache_mismatch_reasons(
sras, n_fft=None, apply_bg_sub=True, row_avg_n=0, min_freq_mhz=f)
assert reasons(floor) == []
assert reasons(floor + 5.0) == [], "a higher request is servable (masked)"
low = reasons(0.0)
assert low and "min-peak-freq floor" in low[0], \
"a lower request cannot be answered by the stored search"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True) is None, \
"default floor-0 request refused against a floored store"
assert cached_rf_image(sras, 0, None, apply_bg_sub=True,
min_freq_mhz=floor) is not None
def test_batch_recompute_resolves_not_masks(tmp_path, no_fft):
"""Re-batching an already-cached file at a raised floor must run the
real FFT and store re-resolved peaks — never let compute_rf_image's
fast path serve the file's own stale cache back to it and bake the
masked copy in as if it were a recompute (silent, permanent data
loss: a masked pixel's true above-floor peak is unrecoverable)."""
path = tmp_path / "rebatch.sras"
gen.write(path, n_angles=2, seed=32, samples_per_frame=256)
assert cache_file(str(path), "fft", True) == ""
first = SrasFile(str(path))
n_angles = first.n_angles
# On the header's kHz grid up front (as the spinbox value would be), so
# the recorded floor reads back equal — cache_file quantizes whatever it
# is given, and this test wants that to be the identity.
floor = round(_biting_floor(first.precomputed_freq_mhz[0]) * 1000) / 1000.0
bites = [img < floor for img in first.precomputed_freq_mhz]
assert bites[0].any(), "the floor must actually bite this fixture"
# What a real floored compute gives, from a view blinded to the cache.
first.precomputed_freq_mhz = [None] * n_angles
expected = [compute_rf_image(first, a, dc_threshold_mv=None,
apply_bg_sub=True, min_freq_mhz=floor)
for a in range(n_angles)]
del first # release memmaps before cache_file rewrites the tail
no_fft.clear()
assert cache_file(str(path), "fft", True, min_freq_mhz=floor) == ""
assert no_fft, "the re-batch ran a real FFT"
after = SrasFile(str(path))
assert after.precomputed_min_freq_mhz == floor
for a in range(n_angles):
assert np.array_equal(after.precomputed_freq_mhz[a], expected[a]), \
f"angle {a}: stored image is a real floored recompute"
assert (after.precomputed_freq_mhz[a][bites[a]] >= floor).all(), \
f"angle {a}: bitten pixels re-resolved above the floor, not zeroed"
def test_min_freq_carries_forward_through_dc_write(tmp_path):
"""A later DC-only write must leave the FFT block's recorded floor
untouched, like row_avg_n and pad_factor."""
path = tmp_path / "floor_carry.sras"
gen.write(path, n_angles=2, seed=33, samples_per_frame=128)
assert cache_file(str(path), "fft", True, min_freq_mhz=75.0) == ""
assert cache_file(str(path), "dc", True) == ""
after = SrasFile(str(path))
assert after.precomputed_min_freq_mhz == 75.0, \
"floor survives a DC-only write"
def test_cach_v3_backward_compat_defaults_floor_zero(tmp_path):
"""A v3 CACH tail predates the min peak frequency floor entirely (no
min_freq_khz field) — readers must still parse it in full, treating it
as floor 0: servable as-is at floor 0, and serve-maskable at any higher
one. This is what protects existing real-world v7 caches from silently
becoming unusable after the v4 bump ships."""
path = tmp_path / "v3_floor.sras"
gen.write(path, n_angles=2, seed=34, samples_per_frame=128)
assert cache_file(str(path), "fft", True) == ""
# Rewrite the tail as a genuine CACH v3 block (no min_freq field).
v4 = SrasFile(str(path))
freq, entries = v4.precomputed_freq_mhz, list(range(v4.n_angles))
tail_offset = v4._cache_tail_offset()
payload = struct.pack(fmt.CACH_HDR_FMT, fmt.CACH_MAGIC, 3, fmt.CACH_FLAG_FFT)
payload += struct.pack(fmt.SFFT_HDR_FMT_V3, fmt.SFFT_MAGIC,
fmt.SFFT_FLAG_BG_SUB, len(entries), 0, 1)
for a in entries:
payload += struct.pack(">H", a) + freq[a].astype(">f4").tobytes()
# See test_cach_v1_reads_as_natural_resolution: release the memmaps
# before write_bytes truncates, or Windows raises EINVAL.
del v4
head = path.read_bytes()[:tail_offset]
path.write_bytes(head + payload)
v3 = SrasFile(str(path))
assert v3.precomputed_min_freq_mhz == 0.0
assert v3.precomputed_pad_factor == 1 and v3.precomputed_bg_sub is True
assert all(np.array_equal(v3.precomputed_freq_mhz[a], freq[a])
for a in entries), "v3 images read back unchanged"
assert cached_rf_image(v3, 0, None, apply_bg_sub=True) is not None
floor = _biting_floor(freq[0])
served = cached_rf_image(v3, 0, None, apply_bg_sub=True,
min_freq_mhz=floor)
assert served is not None
assert np.array_equal(served,
np.where(freq[0] < floor, np.float32(0.0), freq[0]))
def test_min_freq_validation(tmp_path):
path = tmp_path / "floor_bad.sras"
gen.write(path, n_angles=1, seed=35, samples_per_frame=64)
assert cache_file(str(path), "fft", True, min_freq_mhz=-1.0), \
"negative floor must be an error, not a write"
assert cache_file(str(path), "fft", True, min_freq_mhz=float("nan")), \
"NaN floor must be an error, not a write"
sras = SrasFile(str(path))
with pytest.raises(ValueError):
sras.write_v7_cache(new_min_freq_mhz=-0.5)
def test_viewer_reapplies_floor_to_stored_images_without_computing(
rig, no_fft, monkeypatch):
"""The session-cache poisoning bug behind the '95 MHz peak but 5 m/s'
report: changing 'Min peak freq' against a stored cache used to re-file
the identical un-floored image under a key claiming the new floor — the
UI looked updated, the pixels weren't. Now the floor really is
re-applied on serve (masked, still no compute), and clearing it
restores the unmasked image, still without computing."""
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._load_file(str(rig.path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(rig.n_angles))), \
"DC precompute finished"
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX), "CH1 displayed"
assert np.allclose(win._current_image, rig.fresh[0], atol=1e-3)
# Round to the spinbox's 3-decimal granularity; the chosen bin value
# still survives its own (strict-<) floor after rounding down.
floor = round(_biting_floor(rig.fresh[0]), 3)
expect = np.where(rig.fresh[0] < floor, np.float32(0.0), rig.fresh[0])
dispatched.clear()
no_fft.clear()
win.spin_min_freq_mhz.setValue(floor)
win._on_min_freq_changed()
pump(120)
assert np.allclose(win._current_image, expect, atol=1e-3), \
"raised floor re-masks the stored image on serve"
assert (win._current_image == 0).any() and (win._current_image > 0).any()
assert dispatched == [] and not no_fft, \
f"re-masked serve needs no compute (jobs={dispatched}, fft={no_fft})"
key = (0, win.spin_threshold_mv.value(), win.spin_min_freq_mhz.value())
assert key in win._fft_cache
assert np.allclose(win._fft_cache[key], expect, atol=1e-3), \
"the session cache holds the value its key claims"
win.spin_min_freq_mhz.setValue(0.0)
win._on_min_freq_changed()
pump(120)
assert np.allclose(win._current_image, rig.fresh[0], atol=1e-3), \
"clearing the floor restores the unmasked stored image"
assert dispatched == [] and not no_fft
finally:
win.close()
pump(300)
def test_viewer_batch_fft_records_the_floor(tmp_path, no_fft, monkeypatch):
"""Convert → Batch Compute FFT with a floor set: the live spinbox value
reaches cache_file, lands in the reloaded file's provenance and the
info panel, and the viewer then serves the floored cache without
recomputing — the tooltip's promised remedy, end to end."""
path = tmp_path / "floor_gui.sras"
gen.write(path, n_angles=2, seed=36, samples_per_frame=256)
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._load_file(str(path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(win._sras.n_angles))), \
"DC precompute finished"
floor = 100.0
win.spin_min_freq_mhz.setValue(floor)
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")):
win._on_batch_compute("fft")
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
assert wait_until(lambda: win._sras is not None
and win._sras.version == 7), "file reloaded as v7"
pump(200)
assert win._sras.precomputed_min_freq_mhz == floor, \
"the viewer's floor reached the stored provenance"
assert "floor ≥ 100 MHz" in win.lbl_frame_warn.text()
no_fft.clear()
dispatched.clear()
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX), "CH1 displayed"
pump(120)
assert dispatched == [] and not no_fft, \
f"floored cache serves the view directly (jobs={dispatched}, fft={no_fft})"
img = win._current_image
assert ((img == 0) | (img >= floor)).all(), \
"no displayed peak below the floor"
finally:
win.close()
pump(300)
def test_viewer_batch_row_average_dispatch(tmp_path, monkeypatch, no_fft):
"""Driving the new 'Batch Compute Row-Averaged FFT and Store' action
end-to-end through the real menu handler: dialog values reach the
worker, the worker reaches cache_file, and the written file is
self-describing afterward. Also the exact scenario the row-averaged-FFT
recompute bug reported: before the fix, the display always asked for
row_avg_n=0 regardless of what the file actually had stored, so viewing
an angle after this batch action saw a phantom mismatch and launched a
full raw recompute on every view switch. This asserts that no longer
happens -- the stored row-averaged image is shown directly, with no
dispatched compute job and no real FFT."""
path = tmp_path / "rowavg_gui.sras"
gen.write(path, n_angles=2, seed=26, samples_per_frame=128)
class _StubDialog:
def __init__(self, *a, **k):
pass
def exec(self):
return QDialog.DialogCode.Accepted
def get_half_width(self):
return 6
def get_threshold_mv(self):
return -1e9 # mask nothing, keep the comparison simple
monkeypatch.setattr("sras_viewer.main_window.RowAverageFftOptionsDialog", _StubDialog)
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._load_file(str(path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert wait_until(lambda: all((a, CH4_IDX) in win._dc_cache
for a in range(win._sras.n_angles))), \
"DC precompute finished"
with patch("sras_viewer.main_window.QFileDialog.getOpenFileNames",
return_value=([str(path)], "")):
win._on_batch_compute_row_avg()
assert wait_until(lambda: not win._job_running("batch"), 60000), "batch ran"
assert wait_until(lambda: win._sras is not None
and win._sras.version == 7), "file reloaded as v7"
pump(200)
assert win._sras.precomputed_row_avg_n == 6
assert "row-averaged n=6" in win.lbl_frame_warn.text(), win.lbl_frame_warn.text()
expected = compute.cached_rf_image(win._sras, 0, dc_threshold_mv=None,
apply_bg_sub=win.chk_bg_sub.isChecked(),
row_avg_n=6)
assert expected is not None, "the batch write left a readable row-averaged cache"
# The regression this batch action used to leave unfixed: viewing an
# angle afterward must show the stored row-averaged image directly,
# never fall through to a real (raw) recompute.
no_fft.clear()
dispatched.clear()
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX), "CH1 displayed"
for a in range(win._sras.n_angles):
win.spin_angle.setValue(a)
pump(60)
assert np.allclose(win._current_image,
compute.cached_rf_image(
win._sras, a,
dc_threshold_mv=win.spin_threshold_mv.value(),
apply_bg_sub=win.chk_bg_sub.isChecked(),
row_avg_n=6), atol=1e-3), \
f"angle {a} shows the stored row-averaged image"
assert dispatched == [] and not no_fft, \
f"no recompute for row-averaged angles (jobs={dispatched}, fft={no_fft})"
finally:
win.close()
pump(300)
def test_compute_worker_fallback_honors_row_avg_n(tmp_path, monkeypatch, no_fft):
"""The other half of the row-averaged recompute bug: _stored_fft_image
isn't the only path that can serve a view. When the DC4 mask isn't known
yet without I/O (e.g. background DC precompute hasn't reached this angle),
_stored_fft_image's allow_dc_recompute=False guard bails and
_refresh_display falls through to _start_compute's ComputeWorker instead.
That worker must still ask compute_rf_image for the file's own
row_avg_n -- not silently default to 0 -- so its own internal cache
fast path also serves the stored row-averaged image rather than running
a real, non-averaged FFT."""
path = tmp_path / "rowavg_fallback.sras"
gen.write(path, n_angles=2, seed=33, samples_per_frame=128)
# Only fft_rowavg -- deliberately no DC block, exactly what "Batch
# Compute Row-Averaged FFT and Store" leaves on disk by itself.
err = cache_file(str(path), "fft_rowavg", True, dc_threshold_mv=-1e9, row_avg_n=5)
assert err == "", err
app = QApplication.instance() or QApplication([]) # noqa: F841
win = SrasViewerWindow()
win.show()
# Keep _dc_cache empty so _stored_fft_image can't resolve a mask without
# I/O and _refresh_display must fall through to _start_compute.
monkeypatch.setattr(type(win), "_start_dc_precompute", lambda self: None)
dispatched = []
original_start = type(win)._start_compute
monkeypatch.setattr(type(win), "_start_compute",
lambda self: (dispatched.append(self.spin_angle.value()),
original_start(self))[1])
try:
win._load_file(str(path))
assert wait_until(lambda: win._sras is not None), "file loaded"
assert win._sras.precomputed_row_avg_n == 5
# The initial default view (CH4, DC) has nothing cached either, so
# it dispatches its own one-off DC compute for angle 0 on load --
# unrelated to this bug. Let that settle, then clear it so no DC4 is
# available for any angle, simulating "background DC precompute
# hasn't reached this angle yet".
assert wait_until(lambda: not win._job_running("compute")), "initial DC view settled"
dispatched.clear()
win._dc_cache.clear()
no_fft.clear()
win.combo_channel.setCurrentIndex(CH1_IDX)
assert wait_until(lambda: win._current_ch == CH1_IDX
and not win._job_running("compute")), "CH1 displayed"
assert dispatched == [0], \
"with no DC cache available yet, the fallback compute must run"
assert not no_fft, \
f"the fallback's own compute_rf_image call must still hit the " \
f"stored row-averaged cache internally: {no_fft}"
expected = compute.cached_rf_image(
win._sras, 0, dc_threshold_mv=win.spin_threshold_mv.value(),
apply_bg_sub=win.chk_bg_sub.isChecked(), row_avg_n=5)
assert expected is not None
assert np.allclose(win._current_image, expected, atol=1e-3), \
"the displayed image is the stored row-averaged one, not a raw recompute"
finally:
win.close()
pump(300)
View File
+109
View File
@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""Benchmark the FFT peak-search path: serial vs pooled.
Reports wall time, waveforms/s, CPU utilization (utime+stime over wall, in
cores), the estimated per-thread resident pyFFTW plan footprint at each pad
factor (see sras_compute._fft_block_for), and verifies pooled output against
the serial reference.
Usage:
python tools/bench_fft.py # synthetic, pads 1/8/40
python tools/bench_fft.py --pads 40 --spf 2500 --rows 8 --frames 1024
python tools/bench_fft.py --real /path/big.sras --real-rows 32 --pads 40
"""
import argparse
import resource
import sys
import tempfile
import time
from pathlib import Path
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
import sras_compute as compute # noqa: E402
from sras_compute import compute_rf_image # noqa: E402
from sras_format import SrasFile # noqa: E402
import tools.make_test_sras as gen # noqa: E402
from tools.check_equivalence import row_slice # noqa: E402
def _timed(fn):
r0 = resource.getrusage(resource.RUSAGE_SELF)
t0 = time.perf_counter()
out = fn()
wall = time.perf_counter() - t0
r1 = resource.getrusage(resource.RUSAGE_SELF)
cpu = (r1.ru_utime - r0.ru_utime) + (r1.ru_stime - r0.ru_stime)
return out, wall, cpu / max(wall, 1e-9)
def _plan_mb(spf: int, n_len: int, n_workers: int) -> float:
"""Estimated resident pyFFTW plan-buffer footprint across the whole
pool at this transform length (see sras_compute._fft_block_for)."""
block = compute._fft_block_for(spf, n_len)
bytes_per_wf = 4 * spf + 8 * (n_len // 2 + 1)
return block * bytes_per_wf * n_workers / (1024 * 1024)
def bench(sras, pads):
n_wf = sum(int(sras.n_rows[a]) * int(sras.n_frames[a])
for a in range(sras.n_angles))
spf = sras.samples_per_frame
n_workers = compute._MAX_WORKERS
print(f"{n_wf} waveforms x {spf} samples, {sras.n_angles} angle(s), "
f"{n_workers} workers")
print(f"{'pad':>4} {'variant':>8} {'wall':>9} {'wf/s':>10} {'util':>6} "
f"{'plan MB':>9} match")
for pad in pads:
n_fft = spf * pad if pad > 1 else None
n_len = n_fft if n_fft is not None else spf
plan_mb = _plan_mb(spf, n_len, n_workers)
def run(**kw):
imgs = [compute_rf_image(sras, a, dc_threshold_mv=None,
apply_bg_sub=True, n_fft=n_fft, **kw)
for a in range(sras.n_angles)]
return np.concatenate([i.ravel() for i in imgs])
ref, wall, util = _timed(lambda: run(max_workers=1))
rows = [("serial", ref, wall, util, True)]
img, wall, util = _timed(lambda: run())
rows.append(("pooled", img, wall, util, bool(np.array_equal(img, ref))))
for label, img, wall, util, ok in rows:
print(f"{pad:>4} {label:>8} {wall:>8.2f}s {n_wf / wall:>10.0f} "
f"{util:>5.1f}x {plan_mb:>8.1f} {'OK' if ok else 'MISMATCH'}")
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--pads", default="1,8,40",
help="comma-separated pad factors (default 1,8,40)")
p.add_argument("--spf", type=int, default=2500)
p.add_argument("--rows", type=int, default=8)
p.add_argument("--frames", type=int, default=1024)
p.add_argument("--real", help="path to a real .sras file")
p.add_argument("--real-rows", type=int, default=32,
help="rows of angle 0 to use from the real file")
args = p.parse_args()
pads = [int(x) for x in args.pads.split(",")]
if args.real:
sras = row_slice(SrasFile(args.real), 0, args.real_rows)
sras.data = [sras.data[0]]
sras.n_angles = 1
bench(sras, pads)
else:
with tempfile.TemporaryDirectory(prefix="sras_bench_") as tmp:
path = Path(tmp) / "bench.sras"
gen.write(path, n_angles=1, seed=0, samples_per_frame=args.spf,
geometry=[(args.rows, args.frames)])
bench(SrasFile(str(path)), pads)
if __name__ == "__main__":
main()
+194
View File
@@ -0,0 +1,194 @@
#!/usr/bin/env python3
"""Golden-output equivalence harness for compute-path refactors.
Computes a battery of DC / FFT / alignment outputs and prints a stable hash
for each. Run it before a refactor to capture a baseline, then again after
and diff the two reports — every line must match.
Hashes canonicalise to native little-endian float64 before hashing, so a
deliberate dtype/byte-order change that preserves values does not show up as
a false mismatch.
Usage:
python tools/check_equivalence.py --out baseline.txt
python tools/check_equivalence.py --out after.txt --real /path/to/big.sras
diff baseline.txt after.txt
"""
import argparse
import copy
import hashlib
import sys
from pathlib import Path
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from sras_format import SrasFile, CH3_IDX, CH4_IDX, adc_to_mv # noqa: E402
from sras_compute import ( # noqa: E402
apply_alignment, compute_angle_alignment, compute_dc_image, compute_rf_image,
)
import tools.make_test_sras as gen # noqa: E402
def h(arr) -> str:
"""Stable hash of an array's *values*, independent of dtype/byte order."""
a = np.ascontiguousarray(np.asarray(arr, dtype=np.float64))
return hashlib.sha256(a.tobytes()).hexdigest()[:16]
def row_slice(sras: SrasFile, angle_idx: int, n_rows: int) -> SrasFile:
"""A shallow view of *sras* restricted to the first *n_rows* rows of
*angle_idx*, so the huge real file can be exercised in seconds."""
view = copy.copy(sras)
n = min(int(n_rows), int(sras.n_rows[angle_idx]))
view.n_rows = np.array(sras.n_rows, copy=True)
view.n_rows[angle_idx] = n
view.data = list(sras.data)
view.data[angle_idx] = sras.data[angle_idx][:n]
view.y_pos_per_angle = list(sras.y_pos_per_angle)
view.y_pos_per_angle[angle_idx] = sras.y_pos_per_angle[angle_idx][:n]
return view
def report(lines: list[str], label: str, value: str):
lines.append(f"{label:<58} {value}")
def check_file(path: Path, lines: list[str], tag: str,
angles: list[int], n_rows: int | None):
sras = SrasFile(str(path))
report(lines, f"[{tag}] version/n_angles",
f"v{sras.version} n={sras.n_angles}")
report(lines, f"[{tag}] geometry",
f"rows={list(map(int, sras.n_rows))} frames={list(map(int, sras.n_frames))}")
report(lines, f"[{tag}] calibration",
" ".join(f"{m:.6g}/{o:.6g}/{z:.6g}" for m, o, z in
zip(sras.ch_ymult_mv, sras.ch_yoff_adc, sras.ch_yzero_mv)))
report(lines, f"[{tag}] angles_deg", h(sras.angles_deg))
if sras.background is not None:
report(lines, f"[{tag}] background", h(sras.background))
for a in angles:
if a >= sras.n_angles:
continue
s = row_slice(sras, a, n_rows) if n_rows else sras
report(lines, f"[{tag}] x_axis_mm(a={a})", h(s.x_axis_mm(a)))
report(lines, f"[{tag}] y_positions_mm(a={a})", h(s.y_positions_mm(a)))
for ch, name in ((CH3_IDX, "CH3"), (CH4_IDX, "CH4")):
raw = compute_dc_image(s, a, ch)
report(lines, f"[{tag}] dc_adc(a={a},{name})", h(raw))
mv = adc_to_mv(raw, s.ch_ymult_mv[ch], s.ch_yoff_adc[ch],
s.ch_yzero_mv[ch])
report(lines, f"[{tag}] dc_mv(a={a},{name})", h(mv))
dc4 = adc_to_mv(compute_dc_image(s, a, CH4_IDX),
s.ch_ymult_mv[CH4_IDX], s.ch_yoff_adc[CH4_IDX],
s.ch_yzero_mv[CH4_IDX])
thresholds = [-1e9, float(np.median(dc4))]
for bg in (False, True):
if bg and s.background is None:
continue
for pad in (1, 2, 4, 8, 40):
n_fft = s.samples_per_frame * pad if pad > 1 else None
for ti, thr in enumerate(thresholds):
img = compute_rf_image(s, a, dc_threshold_mv=thr,
apply_bg_sub=bg, n_fft=n_fft)
report(lines,
f"[{tag}] rf(a={a},bg={int(bg)},pad={pad},thr{ti})",
h(img))
# dc4_mv passthrough must give the identical result
img2 = compute_rf_image(s, a, dc_threshold_mv=thr,
apply_bg_sub=bg, n_fft=n_fft,
dc4_mv=dc4)
same = "SAME" if h(img) == h(img2) else "DIFFER"
report(lines,
f"[{tag}] rf-dc4arg(a={a},bg={int(bg)},pad={pad},thr{ti})",
same)
def check_alignment(path: Path, lines: list[str], tag: str):
sras = SrasFile(str(path))
if sras.n_angles < 2:
return
dc4 = adc_to_mv(compute_dc_image(sras, 0, CH4_IDX),
sras.ch_ymult_mv[CH4_IDX], sras.ch_yoff_adc[CH4_IDX],
sras.ch_yzero_mv[CH4_IDX])
thr = float(np.median(dc4))
res = compute_angle_alignment(sras, 0, thr)
report(lines, f"[{tag}] align canvas_shape", str(res.canvas_shape))
report(lines, f"[{tag}] align canvas_origin",
f"{res.canvas_origin_mm[0]:.9g},{res.canvas_origin_mm[1]:.9g}")
report(lines, f"[{tag}] align canvas_pitch",
f"{res.canvas_dx_mm:.9g},{res.canvas_dy_mm:.9g}")
for a in sorted(res.per_angle):
t = res.per_angle[a]
report(lines, f"[{tag}] align shift_mm(a={a})",
f"{t.shift_mm[0]:.9g},{t.shift_mm[1]:.9g}")
report(lines, f"[{tag}] align rot(a={a})", f"{t.rotation_deg:.9g}")
report(lines, f"[{tag}] align matrix(a={a})", h(t.matrix))
report(lines, f"[{tag}] align offset(a={a})", h(t.offset))
img = compute_dc_image(sras, a, CH4_IDX)
report(lines, f"[{tag}] align resampled(a={a})",
h(apply_alignment(res, a, img)))
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--out", required=True, help="report file to write")
p.add_argument("--real", help="optional path to a real .sras file")
p.add_argument("--real-rows", type=int, default=2,
help="rows per angle to sample from the real file")
p.add_argument("--real-angles", type=int, default=2,
help="how many angles to sample from the real file")
p.add_argument("--scratch", default=".",
help="directory for generated synthetic files")
args = p.parse_args()
lines = [f"# numpy: {np.__version__}"]
scratch = Path(args.scratch)
synth = scratch / "equiv_synth.sras"
gen.write(synth, n_angles=4, seed=0, samples_per_frame=64)
check_file(synth, lines, "synth", angles=[0, 1, 2, 3], n_rows=None)
check_alignment(synth, lines, "synth")
# A second synthetic with an odd sample count, to catch off-by-one in
# rfft bin handling and chunk-boundary arithmetic.
synth_odd = scratch / "equiv_synth_odd.sras"
gen.write(synth_odd, n_angles=2, seed=7, samples_per_frame=37)
check_file(synth_odd, lines, "odd", angles=[0, 1], n_rows=None)
# A legacy v4 file exercises the uniform-geometry legacy layout through
# the same DC/FFT battery.
synth_v4 = scratch / "equiv_synth_v4.sras"
gen.write_legacy(synth_v4, version=4, n_angles=2, n_rows=6,
n_frames=14, samples_per_frame=48, seed=5)
check_file(synth_v4, lines, "v4", angles=[0, 1], n_rows=None)
# A big-endian int16 v6 file (real acquisitions are >i2; the other
# synthetics are int8).
synth_i16 = scratch / "equiv_synth_i16.sras"
gen.write(synth_i16, n_angles=2, seed=9, samples_per_frame=64, bps=2)
check_file(synth_i16, lines, "int16", angles=[0, 1], n_rows=None)
if args.real:
real = Path(args.real)
if real.exists():
check_file(real, lines, "real",
angles=list(range(args.real_angles)),
n_rows=args.real_rows)
else:
lines.append(f"# real file not found: {real}")
Path(args.out).write_text("\n".join(lines) + "\n")
print("\n".join(lines))
print(f"\nWrote {args.out} ({len(lines)} lines)")
if __name__ == "__main__":
main()
+321
View File
@@ -0,0 +1,321 @@
#!/usr/bin/env python3
"""Generate small synthetic .sras files for testing.
Writes v6 files (per-angle geometry, ragged waveform blocks) matching
scan_format.md, with deterministic pseudo-random waveform content so a test
can compute expected DC/FFT images independently of the reader under test.
Usage:
python tools/make_test_sras.py out.sras [--angles 3] [--seed 0]
"""
import argparse
import struct
import sys
from pathlib import Path
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
# Single source of truth for the byte layout: the reader's own constants.
# The byte *assembly* below stays independent, so a writer bug can't be
# masked by a matching reader bug.
from sras_format import HDR_FMT as HDR_FMT_LEGACY # noqa: E402
from sras_format import GEO_FMT_V6, HDR_FMT_V6 # noqa: E402
from sras_compute import _rotation_matrix as _rot # noqa: E402
# Per-angle (n_rows, n_frames) — deliberately different per angle so ragged
# geometry handling is actually exercised.
_GEOMETRY = [(5, 7), (4, 11), (6, 9), (3, 13), (7, 6)]
_SAMPLE_RATE_HZ = 6.25e9
_VELOCITY_MM_S = 20.0
_LASER_FREQ_HZ = 1000.0
_ROW_SPACING_MM = 0.05
def _preamble(ymult_v: float, yoff_adc: float, yzero_v: float) -> bytes:
"""A Tektronix WFMOutpre string in verbose (keyword) form — the reader
pulls YMULT/YOFF/YZERO out of it by name, so the keywords must be
present literally. YMULT/YZERO are in volts, as the scope reports them."""
return (
":WFMOUTPRE:BYT_NR 1;BIT_NR 8;ENCDG BIN;BN_FMT RI;BYT_OR MSB;"
'WFID "Ch1, DC coupling";NR_PT 2500;PT_FMT Y;'
"XINCR 1.6000E-10;XZERO 0.0E0;XUNIT \"s\";"
f"YMULT {ymult_v:.6E};YOFF {yoff_adc:.6E};YZERO {yzero_v:.6E};"
'YUNIT "V"'
).encode("utf-8")
def build(n_angles: int, seed: int, samples_per_frame: int,
geometry: list[tuple[int, int]] | None = None,
bps: int = 1) -> tuple[bytes, dict]:
rng = np.random.default_rng(seed)
src_geom = geometry or _GEOMETRY
geom = [src_geom[a % len(src_geom)] for a in range(n_angles)]
n_ch = 3
angles_deg = np.linspace(0.0, 60.0, n_angles, dtype=np.float32)
# Distinct calibration per channel so a swapped-channel bug is visible.
cal = [
(1.5625e-3, -87.04, 0.0),
(2.0000e-3, -60.00, 1.0e-3),
(2.5000e-3, -40.00, -2.0e-3),
]
out = bytearray()
out += struct.pack(
HDR_FMT_V6, b"SRAS", 6, n_angles,
0.0, 0.0, 1.0, 1.0, _ROW_SPACING_MM,
_VELOCITY_MM_S, _LASER_FREQ_HZ,
samples_per_frame, _SAMPLE_RATE_HZ, bps, n_ch,
)
out += angles_deg.astype(">f4").tobytes()
x_starts = []
for a, (n_rows, n_frames) in enumerate(geom):
x_start = -0.5 + 0.1 * a
x_starts.append(x_start)
out += struct.pack(GEO_FMT_V6, x_start, 1.0, n_frames, n_rows)
y_positions = []
for a, (n_rows, _) in enumerate(geom):
y = (0.2 * a + np.arange(n_rows) * _ROW_SPACING_MM).astype(np.float32)
y_positions.append(y)
out += y.astype(">f4").tobytes()
for ymult_v, yoff, yzero_v in cal:
p = _preamble(ymult_v, yoff, yzero_v)
out += struct.pack(">H", len(p)) + p
background = rng.integers(-8, 9, size=samples_per_frame, dtype=np.int8)
out += struct.pack(">I", samples_per_frame) + background.tobytes()
# Waveform data. CH1 gets a sinusoid at a per-pixel frequency so the FFT
# peak is predictable; CH3/CH4 get per-pixel DC levels so the mean is too.
t = np.arange(samples_per_frame)
waveforms = []
for a, (n_rows, n_frames) in enumerate(geom):
block = np.empty((n_rows, n_ch, n_frames, samples_per_frame), dtype=np.int8)
for r in range(n_rows):
for f in range(n_frames):
bin_idx = 3 + ((a + r + f) % 17)
phase = 2 * np.pi * bin_idx * t / samples_per_frame
block[r, 0, f] = np.clip(
np.round(60 * np.sin(phase)), -128, 127).astype(np.int8)
block[r, 1, f] = np.int8((a * 7 + r * 3 + f) % 100 - 50)
block[r, 2, f] = np.int8((a * 5 + r * 11 + f * 2) % 120 - 60)
waveforms.append(block)
# bps=2 stores the same values big-endian int16, exercising the
# reader's >i2 memmap path.
out += (block.astype(">i2") if bps == 2 else block).tobytes()
meta = {
"n_angles": n_angles,
"geometry": geom,
"angles_deg": angles_deg,
"x_starts": x_starts,
"y_positions": y_positions,
"cal": cal,
"background": background,
"waveforms": waveforms,
"samples_per_frame": samples_per_frame,
"sample_rate_hz": _SAMPLE_RATE_HZ,
}
return bytes(out), meta
def write(path: Path, n_angles: int = 3, seed: int = 0,
samples_per_frame: int = 64,
geometry: list[tuple[int, int]] | None = None,
bps: int = 1) -> dict:
payload, meta = build(n_angles, seed, samples_per_frame, geometry, bps=bps)
path.write_bytes(payload)
return meta
# ---------------------------------------------------------------------------
# Rotating-sample scan: one shape, imaged at several known rotations
# ---------------------------------------------------------------------------
#
# The scan the angle-alignment path actually has to solve: every angle images
# the *same* sample at a different known rotation and offset, and a correct
# alignment stacks them all back into one shape. Two properties are
# deliberately hostile:
#
# * every angle gets a different window size and a different, meaningless
# stage x_start / y0 — alignment must ignore per-angle stage coordinates
# entirely, so any code that reads them will visibly fail here;
# * the pixel grid is strongly anisotropic (5 µm along x, 50 µm along y),
# like the real instrument, so any registration that rotates raw indices
# instead of millimetres shears the image and cannot converge.
_ROT_DX_MM = 0.005 # x pitch, from velocity/laser_freq below
_ROT_DY_MM = 0.05 # row spacing
_ROT_BG_MV = 4.0
_ROT_FG_MV = 160.0
# How far the sample sits from the rotation axis. Non-zero on purpose: on the
# real instrument every angle's scan window is centred on the rotation axis
# while the sample is not, so each scan sees the sample somewhere else along a
# circle. That offset is exactly what a wrong rotation pivot turns into a ring
# of scans instead of a stack, so a centred test sample would hide the bug.
_ROT_SAMPLE_OFFSET_MM = (0.55, 0.40)
def _sample_shape_mv(u: np.ndarray, v: np.ndarray) -> np.ndarray:
"""An asymmetric test sample in its own mm frame, chirally distinct at
every rotation (no 180° ambiguity) and with structure at several radii so
rotation is well determined."""
u = u - _ROT_SAMPLE_OFFSET_MM[0]
v = v - _ROT_SAMPLE_OFFSET_MM[1]
img = np.full(u.shape, _ROT_BG_MV, dtype=np.float32)
img[((u / 0.85) ** 2 + (v / 0.40) ** 2) <= 1.0] = _ROT_FG_MV # bar
img[(np.abs(u - 0.55) <= 0.22) & (np.abs(v - 0.62) <= 0.22)] = _ROT_FG_MV # nub
img[((u + 0.75) ** 2 + (v + 0.30) ** 2) <= 0.20 ** 2] = _ROT_FG_MV # dot
return img
def write_rotating(path: Path, n_angles: int = 5, samples_per_frame: int = 4,
seed: int = 0) -> dict:
"""Write a v6 file whose CH4 DC image is one sample seen at n_angles known
rotations, and return the ground truth each angle should register to.
``truth[a] = (rotation_deg, (shift_x_mm, shift_y_mm))`` is the rigid map
from angle *a*'s local mm (origin at its own array center) to angle 0's —
exactly what ``register_angle_to_reference`` is supposed to recover.
"""
rng = np.random.default_rng(seed)
n_ch, bps = 3, 1
cal = [(1.5625e-3, -87.04, 0.0), (2.0e-3, -60.0, 1.0e-3), (2.5e-3, -40.0, -2.0e-3)]
ymult_mv, yoff, yzero_mv = cal[2][0] * 1000, cal[2][1], cal[2][2] * 1000
stage_angles, geom, x_starts, y_starts, thetas, offsets = [], [], [], [], [], []
for a in range(n_angles):
stage = -37.0 * a # what the rotation stage reports
stage_angles.append(stage)
# The true image rotation is the negative of the stage's reported
# angle: the stage's positive sense is the opposite of math-positive
# (x toward y) in scan mm. Nothing may depend on knowing that — the
# registration search tries both signs.
thetas.append(-stage)
offsets.append((0.0, 0.0) if a == 0
else (float(rng.uniform(-0.3, 0.3)), float(rng.uniform(-0.3, 0.3))))
# A different window per angle, all centred on the same array center —
# the real instrument grows each angle's axis-aligned bounding box to
# cover the rotated ROI. Sized so the off-axis sample stays inside every
# window at every angle, keeping the expected result unambiguous.
geom.append((88 + 8 * a, 780 + 60 * a))
# Meaningless per-angle stage positions: correct alignment never reads
# them, so scattering them proves it.
x_starts.append(float(20.0 + rng.uniform(-6.0, 6.0)))
y_starts.append(float(30.0 + rng.uniform(-6.0, 6.0)))
out = bytearray()
out += struct.pack(
HDR_FMT_V6, b"SRAS", 6, n_angles,
x_starts[0], y_starts[0], 1.0, 1.0, _ROT_DY_MM,
_VELOCITY_MM_S, _VELOCITY_MM_S / _ROT_DX_MM, # velocity/freq -> 5 µm pitch
samples_per_frame, _SAMPLE_RATE_HZ, bps, n_ch,
)
out += np.array(stage_angles, dtype=">f4").tobytes()
for a, (n_rows, n_frames) in enumerate(geom):
out += struct.pack(GEO_FMT_V6, x_starts[a], 1.0, n_frames, n_rows)
for a, (n_rows, _) in enumerate(geom):
out += (y_starts[a] + np.arange(n_rows) * _ROT_DY_MM).astype(">f4").tobytes()
for ymult_v, yoff_a, yzero_v in cal:
p = _preamble(ymult_v, yoff_a, yzero_v)
out += struct.pack(">H", len(p)) + p
background = rng.integers(-8, 9, size=samples_per_frame, dtype=np.int8)
out += struct.pack(">I", samples_per_frame) + background.tobytes()
truth, dc4_images = {}, []
for a, (n_rows, n_frames) in enumerate(geom):
# Local mm of every pixel, measured from this angle's own array center.
lx = (np.arange(n_frames) - (n_frames - 1) / 2.0) * _ROT_DX_MM
ly = (np.arange(n_rows) - (n_rows - 1) / 2.0) * _ROT_DY_MM
gx, gy = np.meshgrid(lx, ly)
# local = R(theta) @ sample + offset, so sample = R(theta)^T @ (local - offset)
rel = np.stack([gx - offsets[a][0], gy - offsets[a][1]], axis=-1)
s = rel @ _rot(thetas[a]) # == rel @ R^T.T == R^T @ rel
dc4 = _sample_shape_mv(s[..., 0], s[..., 1])
dc4_images.append(dc4)
inv = _rot(-thetas[a])
truth[a] = (-thetas[a],
tuple(float(v) for v in -(inv @ np.array(offsets[a]))))
adc4 = np.clip(np.round((dc4 - yzero_mv) / ymult_mv + yoff), -128, 127).astype(np.int8)
block = np.zeros((n_rows, n_ch, n_frames, samples_per_frame), dtype=np.int8)
block[:, 2] = adc4[:, :, None] # CH4 carries the sample
block[:, 1] = 10 # CH3 flat
block[:, 0] = rng.integers(-40, 41, size=(n_rows, n_frames, samples_per_frame),
dtype=np.int8) # CH1 noise
out += block.tobytes()
path.write_bytes(bytes(out))
return {"n_angles": n_angles, "geometry": geom, "stage_angles_deg": stage_angles,
"truth": truth, "dc4_mv": dc4_images, "x_starts": x_starts,
"y_starts": y_starts, "dx_mm": _ROT_DX_MM, "dy_mm": _ROT_DY_MM}
def write_legacy(path: Path, version: int = 4, n_angles: int = 2,
n_rows: int = 4, n_frames: int = 10,
samples_per_frame: int = 32, seed: int = 0) -> dict:
"""Write a v2/v3/v4 file: uniform geometry, one flat waveform block."""
rng = np.random.default_rng(seed)
n_ch, bps = 3, 1
out = bytearray()
out += struct.pack(
HDR_FMT_LEGACY, b"SRAS", version, n_angles, n_rows,
-0.5, 1.0, _VELOCITY_MM_S, _LASER_FREQ_HZ,
n_frames, samples_per_frame, _SAMPLE_RATE_HZ, bps, n_ch,
)
angles = np.linspace(0.0, 45.0, n_angles, dtype=np.float32)
out += angles.astype(">f4").tobytes()
y = (np.arange(n_rows) * _ROW_SPACING_MM).astype(np.float32)
out += y.astype(">f4").tobytes()
if version >= 3:
for ymult_v, yoff, yzero_v in ((1.5625e-3, -87.04, 0.0),
(2.0e-3, -60.0, 1.0e-3),
(2.5e-3, -40.0, -2.0e-3))[:n_ch]:
p = _preamble(ymult_v, yoff, yzero_v)
out += struct.pack(">H", len(p)) + p
background = rng.integers(-8, 9, size=samples_per_frame, dtype=np.int8)
if version >= 4:
out += struct.pack(">I", samples_per_frame) + background.tobytes()
data = rng.integers(-100, 101,
size=(n_angles, n_rows, n_ch, n_frames, samples_per_frame),
dtype=np.int8)
out += data.tobytes()
path.write_bytes(bytes(out))
return {"version": version, "n_angles": n_angles, "n_rows": n_rows,
"n_frames": n_frames, "samples_per_frame": samples_per_frame,
"n_channels": n_ch, "data": data, "angles_deg": angles,
"y_positions": y, "background": background}
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("output")
p.add_argument("--angles", type=int, default=3)
p.add_argument("--seed", type=int, default=0)
p.add_argument("--spf", type=int, default=64, help="samples per frame")
args = p.parse_args()
out = Path(args.output)
meta = write(out, args.angles, args.seed, args.spf)
print(f"Wrote {out} ({out.stat().st_size:,} bytes)")
print(f" angles : {meta['n_angles']}")
print(f" geometry : {meta['geometry']}")
print(f" spf : {meta['samples_per_frame']}")
if __name__ == "__main__":
main()