dev-auto-align
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Thomas Ales
|
6e8c1cb7a2 |
Auto-align: level the sample on the DC bias levels from the camera window
The operator frames a good spot, confirms the two DC levels the detector reads there, and the rig then measures its own tilt: step 1.5 mm either side on X and then on Y, and tilt the platform until those levels come back. The correction that fixes an offset point is the correction that levels the whole travel — height error and tilt effect are both proportional to the offset — so the procedure ends by applying it and leaving it applied. Both directions are measured from the same starting tilt and averaged, which makes their disagreement a flatness read-out rather than something averaged away silently. core/auto_align.py holds the geometry and the search, Qt-free. The three T-axes' azimuths are the whole geometry: T1 lies along +X so it alone tilts along X, and T0/T2 move as an equal-and-opposite pair to tilt along Y without touching X (tilt_response derives that, and the tests pin it — an axis map that drifts would still converge, on the wrong axis). The search is a secant null on the split-detector difference: probe once to learn what a microstep is worth, sign included, then step at the null. It refuses to servo on a scope that has not re-triggered, escalates a probe that reads as no response before calling an axis dead, and stops at a per-axis travel limit. gui/align_bridge.py runs it on a worker thread; stopping is a threading.Event rather than a queued command, because the worker is inside a long handler for the whole run. The camera window carries the button and the progress window, and locks the scan panel and the jog pads while a run owns the stage. Adds immediate MEAN measurements and an acquisition count to the scope driver, and read_bias_mv to core/scope_inspect — the one scalar the inspection state was missing. KNOWN_ISSUES.md records what only the rig can settle: the probe step, the travel limit, the hold current, and whether the piston the X phase applies alongside its tilt matters. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
aa06fa1460 |
Per-angle background capture: v7/v11 .sras layout
A multi-angle scan runs for hours, but every angle was referenced against
one background captured before the first row of the first angle. That
reference has drifted by the last angle, and comparing angles — the whole
point of a multi-angle scan — was comparing each one against a noise floor
measured at whichever angle came first.
Every angle now captures its own. Before each angle's rows, the operator is
prompted to switch the Genesis laser off, the engine averages a fresh CH1
record, and the operator switches it back on. The data block therefore reads
[background][scan][background][scan] …, one pair per angle.
Format v7 (scan) and v11 (SAW check) carry the background inside the data
block, one length-prefixed block ahead of each angle's rows; the single
block that sat between the preambles and the data is gone. Per-angle offsets
now come from a walk of the data block at parse time rather than arithmetic
over the geometry table, and an angle whose background is not fully on disk
is the frontier — nothing of it was written yet.
v6/v10 files still read: SrasFile hands their one background to every angle,
so readers never branch on the version. Nothing writes them, and a resume
refuses them, since a re-acquired angle writes a block the old layout has no
room for. A resumed v7 angle rewrites its background in place, and the
engine checks the new block fits the room the file has before writing it —
anything else would shift every row behind it.
Two fixes made along the way:
* QtScanController never accepted file_version, so every scan launched
from the app raised TypeError at construction.
* angle_status() left its cursor parked at the frontier, so every angle
past it reported the frontier's own data_offset — which handed a resumed
scan the same write position for several angles. Two recorded offsets in
tests/golden/sras_expected.json are corrected accordingly.
The v6 goldens stay as parser fixtures; the writer is now locked against
bytes the test lays out from scan_format.md itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Thomas Ales
|
844fcd0297 |
SAW quality check: one middle row per angle, and a viewer that overlays them
A full multi-angle scan takes hours, and a rig whose angles disagree produces all of them before anyone finds out. This adds a test mode that acquires one row per angle — the row-wise middle of the ROI — and a viewer that puts every angle's SAW frequency on one graph. The default 80×50 mm ROI at 5 angles goes from 1461 rows to 5. Why the middle row answers an alignment question at all: build_plan centres every angle's rotated bounding box on the same nominal ROI centre, so each angle's middle row crosses that one point on the sample. All the angles measure the same material, so a spread in their frequencies belongs to the rig rather than to where each row happened to land. test_every_angles_middle_row_ crosses_the_roi_centre pins that premise, since the whole comparison rests on it and nothing else in the geometry code would notice it breaking. core/saw_check.py — both halves of the mode, kept together because neither is much use alone. middle_row_plan() reduces a ScanPlan to one row per angle (n_rows // 2, the upper of two centre rows when even); frequency_traces() and alignment_summary() turn the resulting file back into per-angle frequency traces and the scalars an operator is actually asking about — the spread of the per-angle medians, the worst drift along a row, the sparsest row. The verdict thresholds are labelled as rules of thumb, not physics: an anisotropic sample genuinely varies with angle, so a wide spread is a prompt to look at the curves rather than a verdict. Format v10: byte-identical to v6, one row per angle. The version byte earns its keep because the two are otherwise indistinguishable — a v6 scan aborted after its first row is not a check, and a reader guessing from the row count would read a failed scan as a deliberate measurement. create_scan_file() enforces the one-row rule at write time, since nothing downstream can recover from a v10 file that breaks it. ScanEngine gains file_version and is otherwise untouched: the acquisition, the abort/pause path and the background capture are the scan's, unchanged. sras_scan_manager.py now carries the source file's version through an export instead of stamping v6 on everything, which the wider reader would otherwise have made a lie. saw_check_viewer.py — frequency along the row, one curve per angle, over a common offset axis so the curves lie on the same piece of sample; a summary of each angle's median ±1σ against angle; and the per-angle numbers in a table. Analysis parameters (DC threshold, background, time gate) recompute on a worker thread; display ones (smoothing, axis, MHz↔m/s) only redraw. A full v6 scan opens too — the same middle row is pulled out of it — so a finished scan can be re-examined with the check's own read-out. In the app, a check finishes by handing the operator the file and an "Open Viewer" button rather than shutting the rig down the way a completed scan does. Burst mode is not offered: one row per angle means every burst would be a single row, so it buys nothing and still pays for the gate preflight. 137 tests passing, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
23546a03f7 |
Pre-scan angle inspection: park the rig per angle, read the response on the scope
A 9-angle scan takes hours, and an angle that responds poorly still produces rows that look structurally fine in the file — the SAW packet is just not there. This lets the operator walk the angles first, parking the rig at a random point in each, and judge the response before committing to the run. Nothing reads the scope. The operator inspects the instrument directly, so there is no transfer path, no plotting, and no waveform crossing the module boundary — test_inspection_never_reads_a_waveform_back pins that, since it is the kind of premise a later change erodes without noticing. core/scope_inspect.py — the scope state worth looking at, which is not the scan's state: - plain rising-edge trigger on CH2 at 2.0 V, not the scan's logic AND of the laser pulse and the stage gate, so a stationary stage still triggers - FastFrame off, SAMPLE (no averaging) — a weak or intermittent response is exactly what is being looked for, and averaging would hide it - free-running (STOPAfter RUNSTop + STATE RUN) so the trace keeps updating while the operator looks at it - CH1 keeps the acquisition front-end verbatim, so what is on screen is what a scan would record - CH3/CH4 become bias monitors sharing one scale and position, since the comparison is by eye and only works if a division means the same on each. 100 mV/div with ground 3.5 divisions below centre puts 0–700 mV on screen with headroom on an 8- or 10-division graticule (the signal never goes negative, hence moving the trace down). core/angle_inspect.py — AngleInspector, headless and Qt-free like ScanEngine. Points are drawn from the angle's own bounding box: Y from its actual row positions and X uniformly across its data window, so the point is somewhere the scan would really sample rather than merely inside the box. New Point re-rolls without rotating, which is what separates a bad spot on the sample from a bad angle. The stage gate is held off throughout, and the rotator goes home on stop. gui/inspect_bridge.py — QtAngleInspector on the existing QueueWorker base. Inspection is click-driven rather than one long run, so the worker blocks on its queue between commands and an open window costs nothing. BBD position polling is suppressed while inspecting, for the same reason the scan does it: the shared TX queue. sc3_aui_app.py — AngleInspectWindow (angle list, prev/next, New Point) driven off the plan currently entered in the scan panel, so it inspects exactly the scan about to be run. Navigation locks while the stage moves. The list syncs via itemClicked rather than currentRowChanged, so echoing the worker's position back does not re-trigger the move it is reporting. README picks up the new modules, and scope_burst.py which the previous merge left out of the structure listing. 114 tests passing, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
52fdcdd9f3 |
Make row packing toggleable: pad (default) or strict abort
A mis-triggered row cannot be written as it arrived — v6 declares n_frames per row in the header and has no per-row length field, so a short or long row would shift every later row in the file. Until now the only policy was to square it up, which keeps the scan running but leaves the affected row indistinguishable from a good one afterwards: nothing in the file records that it was padded. strict_rows selects the other trade. On any frame-count mismatch the scan stops instead of writing the row, so a data run either produces rows that mean what the header says they mean or fails loudly. Default stays pad, so existing behaviour is unchanged. _warn_frame_delta becomes _check_frame_delta, since it now decides rather than just reports. Both acquisition paths already call it before writing anything for the row (CH1 leads SCAN_CHANNELS, and the burst path checks every row up front), so an abort leaves the file on a whole-row boundary rather than a half-written row — test_strict_row_packing_writes_nothing_for_the_failed_row pins that. Plumbed through QtScanController to a checkbox in the scan panel, persisted in ScanDefaults alongside burst_mode. scan_format.md documents both policies and notes that the choice is not recorded in the file. The row-clipping setup in the padding test is now a _clip_one_row helper, reused by the strict tests. 92 tests passing, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
ef8c0feb91 |
Expose burst acquisition in the app; refresh scan_format acquisition docs
Wires the burst_mode flag through QtScanController to ScanEngine and adds a checkbox to the scan panel. The setting persists via ScanDefaults like the other scan fields, defaulting to off — per-row acquisition stays the default path until burst mode has run on the rig and the gate-off preflight has settled which TRIGOUT value idles the pin low. scan_format.md — the acquisition settings table had drifted from the code it claimed to describe: it attributed the settings to sc3_aui_app.py (they moved to core/scope_sras.py in the Phase 2 extraction), listed a 1.24 V trigger level and 0 % offset where the code sets 0.500 V and HORizontal:POSition 30, and did not mention the logic-AND scan trigger at all. Corrected, pointed at the module that actually owns them, and noted that none of it affects byte layout — only where the acoustic packet lands inside a frame. Added an acquisition-paths section: the two paths write byte-identical files and the choice is a runtime flag that is not recorded in the file, so a reader never needs to care which produced it. Documents where row boundaries come from in a burst and that either path squares rows up to n_frames. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
116c9c07c7 |
Burst acquisition: many whole rows per FastFrame acquisition
Per-row acquisition pays a full arm/stop/transfer round trip for every row, and the transfer is one IEEE-488.2 block read per frame (~16k frames a row). Burst mode runs one FastFrame acquisition across as many complete rows as the scope's frame memory holds and pulls each burst in a single CURVe? transaction, amortising the round trip over the whole burst. It is opt-in (ScanEngine(burst_mode=...), default False) and writes byte-identical files to the per-row path — test_burst_and_serial_produce_ identical_files runs the same plan both ways and compares the bytes, which is the property the whole feature rests on. core/scope_burst.py — the new policy module. Everything that computes rather than talks to hardware is a free function, so sizing and row-splitting are testable without a rig: rows_per_burst() (rounds down, since a partial row can't be written, and clamps to a transfer-buffer budget), split_row_counts(), normalize_row(), frame_means_block(). The hard part is that a burst carries no row markers — the scope returns one flat run of frames. Boundaries come from ACQuire:NUMFRAMESACQuired? sampled after each acquiring pass while the stage gate is already low, rebased on a baseline read back at RUN rather than assuming the counter resets. A counter that goes backwards means the acquisition restarted mid-burst and is now a hard error instead of silently misattributing every later row. core/scan_engine.py — the row loop splits into _scan_rows_serial and _scan_rows_burst. The wire is channel-major and the file is row-major with channels inner, so _write_burst deinterleaves by writing one channel at a time to strided offsets; peak memory stays at a single channel's burst instead of the whole thing. _gate_off_preflight is what makes this trustworthy on real hardware. The BBD value that idles the trigger output low is not settled by the protocol docs (see TRIGOUT_GATE_OFF), and getting it wrong fills every burst with flyback frames that silently shift the file. The scope already measures the gate on CH3, so the check needs no bench probe: one gated-off flyback must acquire nothing, and one gated pass must acquire something — the second half is what stops a dark laser from making the first half pass vacuously. It runs once per scan and costs two row-times. Two fixes fall out of this work and apply to both paths: - Rows are now squared up to the declared n_frames (short rows zero-padded, long rows truncated, both warned). v6 commits to n_frames per row in the header and has no per-row length field, so an over- or under-triggered row used to shift every later row in the file. - The X trigger output is returned to idle in the run() finally block. The per-row path left TRIGOUT_MAXV armed for the rest of the session, so the gate line kept being driven on every later jog. core/scope_sras.py — pins DATa:ENCdg RIBinary and DATa:WIDth 1 during setup instead of inheriting front-panel state. The file header hardcodes bytes_per_sample=1; a scope left on 2 bytes would have corrupted every frame written. frames_acquired/frame_means move to scope_burst, where the offset- based variants serve both paths. tests/fakes.py — FakeStage and FakeScope are now wired together the way the rig is: a gated X move at scan velocity feeds frames into a running acquisition at the real 20 kHz / 100 mm/s rate, direction-agnostic. Both paths therefore derive frame counts from one model, which is what makes the byte-identity comparison meaningful, and a gate the engine forgets to drop shows up as extra frames instead of passing silently. Frame content is a function of (channel, index) alone, so the same frame sequence yields the same bytes however it is chopped into transfers. 87 tests passing, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
afe33249d1 |
Phase 4: extract headless ScanEngine; de-Qt the T3R driver
The headline of the refactor. Scan orchestration no longer lives inside a QObject that reaches through Qt workers for its hardware handles. core/scan_engine.py — ScanEngine(stage, scope, rotator, plan, out_path, resume, callbacks). Takes the concrete drivers, blocks in run(), reports via plain callables, and prompts through an injected blocking callable. No Qt import anywhere in the path (test_engine_imports_without_qt proves it), so a simpler GUI or a CLI can drive the identical acquisition. Supporting extractions, all Qt-free: - core/scope_sras.py — SCPI policy: channel profiles, trigger programming, background average, per-row FastFrame transfer - core/rotation.py — RotationAxis + RotationSettings (the GR_* constants) - core/scan_resume.py — frontier contiguity rule + settings compatibility - gui/scan_bridge.py — QtScanController, exposing exactly the signal surface the old ScanWorker had, so MainWindow's connections are unchanged hardware/t3r_driver.py is now Qt-free: a plain Signal class, a threading reader, and a polling thread instead of QObject/QThread/QTimer. gui/qt_t3r.py re-emits its callbacks as queued Qt signals for the panels. Fixes carried by the extraction: - rotation waits on the driver's MOTION_DONE event instead of time.sleep(estimate + 0.5) - abort during an operator prompt now takes effect; the old _prompt_event.wait() had no timeout and could not be interrupted - the poll timer is a thread, so an I/O error tearing down the driver no longer calls QTimer.stop() from the wrong thread - T3RDriver.disconnect() renamed close(); it shadowed QObject.disconnect() - per-frame DC means use np.frombuffer over the joined block instead of struct.unpack per frame (~16k tuple allocations per row) tests/fakes.py + test_scan_engine.py (14 tests) assert the exact command sequence, file layout, resume seeking, abort/pause, and geometry rejection before any hardware call; test_scan_resume.py covers the frontier rule. 58 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
d2734c45d6 |
Phase 3: viewer reads v6 via mmap; extract analysis core
The viewer could only parse v2-v4 headers while the app has been writing v6 for some time — it could not open ANY file the current app produces. It now uses core.sras_format directly (v6 only, per user decision). New core/sras_analysis.py (Qt-free): ChannelCalibration, image reducers, and SawPipeline. sras_viewer.py keeps only Qt. Memory (measured, 92 MB synthetic scan, separate processes): old eager path +305 MB read()+slice-copy+astype+float32 mean new mmap path + 31 MB zero-copy view + mean(dtype=) -> identical DC image; old scaled at ~3.3x file size, new at image size - load_angle() returns a read-only mmap view instead of reading the whole data block, then copying it twice - SAW sweeps keep one scalar per pixel (process_shot_metrics) instead of retaining 5 full arrays x pixel count in a results list - CH1 float32 materializes only for pixels passing the DC mask - matched filter caches the template FFT instead of recomputing per pixel - opening a new file drops every reference to the old one (compute/ template/diagnostic workers used to pin the previous multi-GB mapping) Responsiveness: - 250 ms debounce coalesces spinbox storms into one recompute - grating change is a display-time scalar multiply, not a full FFT rerun - colormap/clim reuse the AxesImage (set_data/set_clim) instead of clf() + rebuilding the colorbar; draw_idle() throughout - SAW diagnostics (21 pipeline runs) and CSV export moved off the GUI thread Also: ragged per-angle geometry is respected (v6 angles differ in rows/ frames), truncated scans show only rows present on disk, dead decimation path and v2 fallback branch removed, scipy added to viewer requirements. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Thomas Ales
|
dff9f69d78 |
Phase 2: extract headless core modules (sras_format, scan_geometry, config)
- core/sras_format.py: THE v6 implementation — create_scan_file (writer, byte-identical to the old one, enforced against the Phase-0 goldens), SrasFile parser with frontier/truncation walk, and zero-copy mmap load_angle/load_row views for multi-GB files - core/scan_geometry.py: ScanPlan/AngleGeometry dataclasses, build_plan (rotated-bbox trig from MainWindow._build_scan_params), travel-limit validate_plan (limits now a StageLimits dataclass, not literals buried in the worker), format_eta + EtaEstimator (bounded deque) - core/config.py: ScanDefaults dataclass replaces the module-import-time dict globals. FIXES: editing any main-window port used to rewrite aui_defaults.json without helios_port, silently reverting the Helios port every time (test_helios_port_survives_partial_update covers it). Also drops the inert laser_freq_hz plumbing — scans always used the LASER_FREQ_HZ constant. - hardware/serial_util.py: shared 8N1 open + scored port enumeration (promoted from t3r_control_panel); helios_laser and the panel use it - sc3_aui_app.py and sras_scan_manager.py migrated onto core (three format implementations down to one); ScanWorker now takes a ScanPlan - tests: byte-identical writer vs golden, frontier over every truncation variant, mmap==eager, geometry vs golden fixtures + invariants, config round-trip. 28 passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |