Adds a second acquisition path that runs one FastFrame acquisition across as
many whole rows as the scope's frame memory holds, transferring each burst in
a single CURVe? transaction instead of one block read per frame per row.
Off by default; both paths write byte-identical files.
- d6a5626 driver support (bulk transfer, MAXFRames?, per-row trigger gating)
and two read_raw fixes: a short read on the length digits, and #0
indeterminate-length blocks, which a raw socket cannot delimit by EOI
- 116c9c0 core/scope_burst.py, the split row loop, and the on-rig gate-off
preflight that resolves the undocumented BBD trigger-idle value
- ef8c0fe GUI checkbox, persisted default, and corrected scan_format docs
Also fixes, on both paths: rows are squared up to the declared n_frames
(v6 has no per-row length field, so a mis-triggered row shifted every later
row in the file), the transfer format is pinned rather than inherited from
the front panel (the header hardcodes bytes_per_sample=1), and the X trigger
output is returned to idle when a scan ends.
87 tests passing, ruff clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wires the burst_mode flag through QtScanController to ScanEngine and adds a
checkbox to the scan panel. The setting persists via ScanDefaults like the
other scan fields, defaulting to off — per-row acquisition stays the default
path until burst mode has run on the rig and the gate-off preflight has
settled which TRIGOUT value idles the pin low.
scan_format.md — the acquisition settings table had drifted from the code it
claimed to describe: it attributed the settings to sc3_aui_app.py (they moved
to core/scope_sras.py in the Phase 2 extraction), listed a 1.24 V trigger
level and 0 % offset where the code sets 0.500 V and HORizontal:POSition 30,
and did not mention the logic-AND scan trigger at all. Corrected, pointed at
the module that actually owns them, and noted that none of it affects byte
layout — only where the acoustic packet lands inside a frame.
Added an acquisition-paths section: the two paths write byte-identical files
and the choice is a runtime flag that is not recorded in the file, so a
reader never needs to care which produced it. Documents where row boundaries
come from in a burst and that either path squares rows up to n_frames.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per-row acquisition pays a full arm/stop/transfer round trip for every row,
and the transfer is one IEEE-488.2 block read per frame (~16k frames a row).
Burst mode runs one FastFrame acquisition across as many complete rows as the
scope's frame memory holds and pulls each burst in a single CURVe?
transaction, amortising the round trip over the whole burst.
It is opt-in (ScanEngine(burst_mode=...), default False) and writes
byte-identical files to the per-row path — test_burst_and_serial_produce_
identical_files runs the same plan both ways and compares the bytes, which is
the property the whole feature rests on.
core/scope_burst.py — the new policy module. Everything that computes rather
than talks to hardware is a free function, so sizing and row-splitting are
testable without a rig: rows_per_burst() (rounds down, since a partial row
can't be written, and clamps to a transfer-buffer budget), split_row_counts(),
normalize_row(), frame_means_block().
The hard part is that a burst carries no row markers — the scope returns one
flat run of frames. Boundaries come from ACQuire:NUMFRAMESACQuired? sampled
after each acquiring pass while the stage gate is already low, rebased on a
baseline read back at RUN rather than assuming the counter resets. A counter
that goes backwards means the acquisition restarted mid-burst and is now a
hard error instead of silently misattributing every later row.
core/scan_engine.py — the row loop splits into _scan_rows_serial and
_scan_rows_burst. The wire is channel-major and the file is row-major with
channels inner, so _write_burst deinterleaves by writing one channel at a
time to strided offsets; peak memory stays at a single channel's burst
instead of the whole thing.
_gate_off_preflight is what makes this trustworthy on real hardware. The BBD
value that idles the trigger output low is not settled by the protocol docs
(see TRIGOUT_GATE_OFF), and getting it wrong fills every burst with flyback
frames that silently shift the file. The scope already measures the gate on
CH3, so the check needs no bench probe: one gated-off flyback must acquire
nothing, and one gated pass must acquire something — the second half is what
stops a dark laser from making the first half pass vacuously. It runs once
per scan and costs two row-times.
Two fixes fall out of this work and apply to both paths:
- Rows are now squared up to the declared n_frames (short rows zero-padded,
long rows truncated, both warned). v6 commits to n_frames per row in the
header and has no per-row length field, so an over- or under-triggered row
used to shift every later row in the file.
- The X trigger output is returned to idle in the run() finally block. The
per-row path left TRIGOUT_MAXV armed for the rest of the session, so the
gate line kept being driven on every later jog.
core/scope_sras.py — pins DATa:ENCdg RIBinary and DATa:WIDth 1 during setup
instead of inheriting front-panel state. The file header hardcodes
bytes_per_sample=1; a scope left on 2 bytes would have corrupted every frame
written. frames_acquired/frame_means move to scope_burst, where the offset-
based variants serve both paths.
tests/fakes.py — FakeStage and FakeScope are now wired together the way the
rig is: a gated X move at scan velocity feeds frames into a running
acquisition at the real 20 kHz / 100 mm/s rate, direction-agnostic. Both
paths therefore derive frame counts from one model, which is what makes the
byte-identity comparison meaningful, and a gate the engine forgets to drop
shows up as extra frames instead of passing silently. Frame content is a
function of (channel, index) alone, so the same frame sequence yields the
same bytes however it is chopped into transfers.
87 tests passing, ruff clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for burst acquisition: the scope needs to report and transfer a
whole multi-row FastFrame acquisition, and the BBD needs to gate its trigger
output per row rather than staying armed for the scan.
tektronix_base
- get_fastframe_max_frames() exposes HORizontal:FASTframe:MAXFRames?, which
is what sizes a burst once the horizontal settings are fixed.
- transfer_fastframe_bulk() pulls a burst as one contiguous buffer. Unlike
transfer_fastframe it does not assume how the scope frames the response:
it accumulates until the expected byte count is reached, so one large IEEE
block and one block per frame both work.
- set_data_encoding() / set_data_width() make the transfer format settable
instead of inherited from whatever the front panel was left on.
- read_raw() had two real defects. The length-digit read used a bare recv()
and only checked the length afterwards, so a short read raised "Failed to
read data length" on a perfectly good transfer; it now goes through a
_recv_exact() helper, as does the trailing separator. And a #0
indeterminate-length block was parsed as int("") -> ValueError. #0 is
normally delimited by EOI, which a raw socket never sees, so read_raw now
takes expected_bytes to size it. The bulk transfer relies on this.
pybbd202
- arm_scan_gate(axis, armed) raises and drops the max-velocity trigger
output the scope's AND-gate uses. A burst spans several rows with the
scope running throughout, so the gate must be low for the flyback or the
return move reaches max velocity and injects frames between rows.
- set_trigger_verified() reads the mode back after setting it. set_trigger
is fire-and-forget over the shared TX queue; burst mode toggles the gate
between every row, where a dropped change silently corrupts the file
rather than failing loudly.
- set_trigger_gate_off() so the scan can leave the output idle on exit.
- TRIGOUT_GATE_OFF is deliberately marked unverified. §7.6 of the BBD203
protocol doc describes `mode` as an enumeration capping at 0x11, which
contradicts the bitmask this driver actually sends (TRIGOUT_MAXV = 0x90,
known working), so the doc cannot settle which value idles the pin low.
The engine's preflight check resolves it on the rig instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hardware/.DS_Store kept showing up as untracked throughout the refactor;
.gitignore had no rule for it. The file itself is left on disk (Finder
regenerates it) — it is simply ignored now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Collapsed Args:/Returns:/Raises: blocks that only restated the
signature (364 lines): tektronix_base 48% -> ~20% doc density,
helios_laser and uc480_camera likewise. Only docstrings whose entire
body was those sections were touched.
- Preserved verbatim the comments that carry hardware knowledge the code
can't express: uc480's USB split-transaction contention note (with its
measured fps), the IS_ALLOW_STARTER_FW_UPLOAD segfault explanation, the
QImage-copy rationale, and tektronix's NUMFRAMESACQuired warning.
- README: project structure, quick start, and every usage example now
describe code that exists (they referenced hardware/bbd202.py,
CoherentHOPSLaser, get_curve_binary, and 'python -m scanengine.app',
none of which do). Added a headless-scan example and a read-a-scan-file
example, since reuse without the GUI is the point of the refactor.
- SETUP: structure section defers to README instead of keeping a second
stale copy; documents the vendored uEye SDK and the Genesis quarantine.
- ruff is now clean repo-wide: fixed the remaining raise-from, unused
loop variables, placeholder f-strings, and a non-strict zip; the
widget-layout semicolon idiom is an explicit config ignore rather than
22 standing warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gui/qt_workers.py — one QueueWorker base replaces the per-device command
queue + dispatch + signal boilerplate. The loop blocks on the queue
instead of waking 10-20x/second forever (test_idle_worker_does_not_spin
asserts an idle worker burns ~no CPU). PollingQueueWorker adds
self-rescheduling polling: the next poll is queued only after the
previous finishes, so a device slower than the interval can't accumulate
a backlog (test_polling_never_overlaps_or_backs_up).
Helios responsiveness — the concrete bug that motivated the above: a free
running 1 s QTimer queued a status poll that took ~2 s, so the queue grew
for as long as the panel stayed connected.
- helios_laser._query reads until the CR terminator instead of sleeping a
fixed 0.05 + 0.2 s per query
- one _query_int() helper replaces five copies of parse-with-logging
- polling is now driven by the worker; HeliosWindow's QTimer is gone
- dropped __del__, which disabled the laser and wrote to the serial port
from the garbage collector at an unpredictable time
helios_test_app.py — the worker was moveToThread'd but every call site
invoked its methods directly, so all serial I/O (including the sleeps)
ran on the GUI thread; Query All froze the UI for ~2 s. Calls now go
through a queued signal to a pyqtSlot. Also: connect/disconnect cycles
leaked a QThread + worker + 9 connections each time; 16 copies of the
not-connected guard collapse to _require_connection(); the Query Power
button called a method that has never existed (AttributeError popup) and
is now disabled and documented in KNOWN_ISSUES.
DCBiasImageWidget preallocates its image and uses set_data/set_clim, so
the live preview stops rebuilding the array and the whole artist tree per
row (O(rows^2) over a scan).
bbd20x: connect() now raises when no bays respond instead of reporting
success on the wrong port; disconnect() joins with a timeout so a wedged
reader can't hang shutdown; one _channel_for() helper replaces four
copy-pasted axis mappings; hardcoded travel limits become TRAVEL_MM; the
joke error strings are gone.
gui/widgets.py adds the shared ConnectionBar / PortSelector / bounded
LogConsole / StatusGrid for the test benches to adopt. 65 tests passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The headline of the refactor. Scan orchestration no longer lives inside a
QObject that reaches through Qt workers for its hardware handles.
core/scan_engine.py — ScanEngine(stage, scope, rotator, plan, out_path,
resume, callbacks). Takes the concrete drivers, blocks in run(), reports
via plain callables, and prompts through an injected blocking callable.
No Qt import anywhere in the path (test_engine_imports_without_qt proves
it), so a simpler GUI or a CLI can drive the identical acquisition.
Supporting extractions, all Qt-free:
- core/scope_sras.py — SCPI policy: channel profiles, trigger programming,
background average, per-row FastFrame transfer
- core/rotation.py — RotationAxis + RotationSettings (the GR_* constants)
- core/scan_resume.py — frontier contiguity rule + settings compatibility
- gui/scan_bridge.py — QtScanController, exposing exactly the signal
surface the old ScanWorker had, so MainWindow's connections are unchanged
hardware/t3r_driver.py is now Qt-free: a plain Signal class, a threading
reader, and a polling thread instead of QObject/QThread/QTimer.
gui/qt_t3r.py re-emits its callbacks as queued Qt signals for the panels.
Fixes carried by the extraction:
- rotation waits on the driver's MOTION_DONE event instead of
time.sleep(estimate + 0.5)
- abort during an operator prompt now takes effect; the old
_prompt_event.wait() had no timeout and could not be interrupted
- the poll timer is a thread, so an I/O error tearing down the driver no
longer calls QTimer.stop() from the wrong thread
- T3RDriver.disconnect() renamed close(); it shadowed QObject.disconnect()
- per-frame DC means use np.frombuffer over the joined block instead of
struct.unpack per frame (~16k tuple allocations per row)
tests/fakes.py + test_scan_engine.py (14 tests) assert the exact command
sequence, file layout, resume seeking, abort/pause, and geometry
rejection before any hardware call; test_scan_resume.py covers the
frontier rule. 58 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The viewer could only parse v2-v4 headers while the app has been writing
v6 for some time — it could not open ANY file the current app produces.
It now uses core.sras_format directly (v6 only, per user decision).
New core/sras_analysis.py (Qt-free): ChannelCalibration, image reducers,
and SawPipeline. sras_viewer.py keeps only Qt.
Memory (measured, 92 MB synthetic scan, separate processes):
old eager path +305 MB read()+slice-copy+astype+float32 mean
new mmap path + 31 MB zero-copy view + mean(dtype=)
-> identical DC image; old scaled at ~3.3x file size, new at image size
- load_angle() returns a read-only mmap view instead of reading the whole
data block, then copying it twice
- SAW sweeps keep one scalar per pixel (process_shot_metrics) instead of
retaining 5 full arrays x pixel count in a results list
- CH1 float32 materializes only for pixels passing the DC mask
- matched filter caches the template FFT instead of recomputing per pixel
- opening a new file drops every reference to the old one (compute/
template/diagnostic workers used to pin the previous multi-GB mapping)
Responsiveness:
- 250 ms debounce coalesces spinbox storms into one recompute
- grating change is a display-time scalar multiply, not a full FFT rerun
- colormap/clim reuse the AxesImage (set_data/set_clim) instead of
clf() + rebuilding the colorbar; draw_idle() throughout
- SAW diagnostics (21 pipeline runs) and CSV export moved off the GUI thread
Also: ragged per-angle geometry is respected (v6 angles differ in rows/
frames), truncated scans show only rows present on disk, dead decimation
path and v2 fallback branch removed, scipy added to viewer requirements.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- core/sras_format.py: THE v6 implementation — create_scan_file (writer,
byte-identical to the old one, enforced against the Phase-0 goldens),
SrasFile parser with frontier/truncation walk, and zero-copy mmap
load_angle/load_row views for multi-GB files
- core/scan_geometry.py: ScanPlan/AngleGeometry dataclasses, build_plan
(rotated-bbox trig from MainWindow._build_scan_params), travel-limit
validate_plan (limits now a StageLimits dataclass, not literals buried
in the worker), format_eta + EtaEstimator (bounded deque)
- core/config.py: ScanDefaults dataclass replaces the module-import-time
dict globals. FIXES: editing any main-window port used to rewrite
aui_defaults.json without helios_port, silently reverting the Helios
port every time (test_helios_port_survives_partial_update covers it).
Also drops the inert laser_freq_hz plumbing — scans always used the
LASER_FREQ_HZ constant.
- hardware/serial_util.py: shared 8N1 open + scored port enumeration
(promoted from t3r_control_panel); helios_laser and the panel use it
- sc3_aui_app.py and sras_scan_manager.py migrated onto core (three
format implementations down to one); ScanWorker now takes a ScanPlan
- tests: byte-identical writer vs golden, frontier over every truncation
variant, mmap==eager, geometry vs golden fixtures + invariants, config
round-trip. 28 passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- uc480_camera: drop never-called _capture_paused/get_framerate (the
hardware question _capture_paused encoded is now in KNOWN_ISSUES.md)
- t3r_protocol: drop read_reg/write_reg/decode_reg/Reg (commands never
wired into the driver)
- bbd20x: drop _update0x0212 (never dispatched) and 8 of 9 unused
trigger convenience wrappers; apt_constants: drop TriggerBitsStepper
(servo-only rig)
- ruff --fix: 35 unused imports across all apps; drop unused T3R_BAUD
- genesis_core.py: quarantine warning header; docs/genesis_verification.md
bench checklist for the 7 divergences vs tools/genesis_laser_gui.py
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified by repo-wide name search + transitive closure over internal
calls: the live apps use 25 methods (plus raw write/query); everything
else — cosmetic label styling, unused getters/setters, transfer_waveform,
acquire_waveform — had no callers. Also: linear-time chunk join in
read_raw instead of quadratic bytes += concat, and a typed except on its
debug path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
app.py was an abandoned skeleton (15 'pass # TODO' handlers, loads a
deleted .qss); everything else was reachable only from it:
ui_mainwindow.py (pyuic6 artifact), sc3-new.ui, motion_worker.py,
genesis_worker.py, coherent_hops_laser.py (stubs), scanning/ (dead C#
port + unused plan generator), config.json, plus helios_diagnostic.py
(sends wrong protocol commands) and helios_terminal.py (worse duplicate
of helios_test_app's Terminal tab).
hardware/__init__.py no longer wildcard-imports every driver, so the
stage driver imports without the uEye camera SDK installed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ruff config, offscreen smoke tests for all 7 GUI apps/panels
- golden v6 .sras fixtures (complete + 4 truncation variants) generated
by the pre-refactor writer, with expected header/frontier JSON
- golden geometry fixtures from the pre-refactor _build_scan_params
- consistency tests proving current code reproduces the goldens
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Merged four separate hardware control projects into unified platform
- Created unified requirements.txt with all dependencies
- Added comprehensive .gitignore
- Added project overview README
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Thomas Ales