tka/scanengine-3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 5: shared worker base, self-rescheduling polls, driver robustness

Browse Source 
gui/qt_workers.py — one QueueWorker base replaces the per-device command
queue + dispatch + signal boilerplate. The loop blocks on the queue
instead of waking 10-20x/second forever (test_idle_worker_does_not_spin
asserts an idle worker burns ~no CPU). PollingQueueWorker adds
self-rescheduling polling: the next poll is queued only after the
previous finishes, so a device slower than the interval can't accumulate
a backlog (test_polling_never_overlaps_or_backs_up).

Helios responsiveness — the concrete bug that motivated the above: a free
running 1 s QTimer queued a status poll that took ~2 s, so the queue grew
for as long as the panel stayed connected.
- helios_laser._query reads until the CR terminator instead of sleeping a
  fixed 0.05 + 0.2 s per query
- one _query_int() helper replaces five copies of parse-with-logging
- polling is now driven by the worker; HeliosWindow's QTimer is gone
- dropped __del__, which disabled the laser and wrote to the serial port
  from the garbage collector at an unpredictable time

helios_test_app.py — the worker was moveToThread'd but every call site
invoked its methods directly, so all serial I/O (including the sleeps)
ran on the GUI thread; Query All froze the UI for ~2 s. Calls now go
through a queued signal to a pyqtSlot. Also: connect/disconnect cycles
leaked a QThread + worker + 9 connections each time; 16 copies of the
not-connected guard collapse to _require_connection(); the Query Power
button called a method that has never existed (AttributeError popup) and
is now disabled and documented in KNOWN_ISSUES.

DCBiasImageWidget preallocates its image and uses set_data/set_clim, so
the live preview stops rebuilding the array and the whole artist tree per
row (O(rows^2) over a scan).

bbd20x: connect() now raises when no bays respond instead of reporting
success on the wrong port; disconnect() joins with a timeout so a wedged
reader can't hang shutdown; one _channel_for() helper replaces four
copy-pasted axis mappings; hardcoded travel limits become TRAVEL_MM; the
joke error strings are gone.

gui/widgets.py adds the shared ConnectionBar / PortSelector / bounded
LogConsole / StatusGrid for the test benches to adopt. 65 tests passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Thomas Ales
2026-07-28 11:21:29 -05:00
parent afe33249d1
commit 44febe34b8
8 changed files with 833 additions and 444 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sc3_aui_app.py
+150 -214
View File
@@ -5,7 +5,6 @@ Loads sc3-aui-main.ui, sc3-aui-camera.ui, sc3-aui-scanprogress.ui via uic
and wires up T3R, BBD202, oscilloscope, and camera hardware workers.
"""
import queue
import struct
import sys
import time
@@ -14,7 +13,7 @@ from threading import Thread
import numpy as np
from PyQt6 import uic
from PyQt6.QtCore import QObject, QThread, QTimer, Qt, pyqtSignal, pyqtSlot
from PyQt6.QtCore import QObject, QThread, QTimer, Qt, pyqtSignal
from PyQt6.QtGui import QImage, QPixmap
from PyQt6.QtWidgets import (
QApplication, QDialog, QDialogButtonBox, QFileDialog, QHBoxLayout, QLabel,
@@ -38,6 +37,7 @@ from core.scan_resume import is_compatible, plan_resume
from core.scope_sras import SAMPLE_RATE_HZ, configure_channels
from core.sras_format import SCAN_CHANNELS, SrasFile, plan_from_header
from gui.qt_t3r import QtT3RAdapter
from gui.qt_workers import PollingQueueWorker, QueueWorker
from gui.scan_bridge import QtScanController
from hardware.helios_laser import HeliosLaser
from hardware.pybbd202 import AXIS_X, AXIS_Y, ThorlabsServoDriver
@@ -53,7 +53,13 @@ BBD_DEFAULT_JOG_MM = 0.5 # default jog step for BBD202
class DCBiasImageWidget(FigureCanvas):
"""Live 2D DC-bias image: rows × frames, matching sras_viewer's compute_dc_image view."""
"""Live 2D DC-bias image: rows × frames, matching the viewer's DC image.
The image buffer is allocated once per scan and updated in place. The
previous version rebuilt the whole array and the entire matplotlib
artist tree on every row, so the live preview got steadily slower as a
scan progressed (O(rows²) work for one scan).
"""
def __init__(self, parent=None):
fig = Figure(tight_layout=True)
@@ -65,36 +71,59 @@ class DCBiasImageWidget(FigureCanvas):
QSizePolicy.Policy.Expanding,
)
self.ax = fig.add_subplot(111)
self._row_data: list[np.ndarray] = []
self._img: np.ndarray | None = None
self._im = None
self.ax.set_title("DC Bias Preview (ADC counts)")
self.ax.set_xlabel("Frame")
self.ax.set_ylabel("Row")
self.ax.text(0.5, 0.5, "Waiting for data…",
transform=self.ax.transAxes, ha="center", va="center", color="gray")
self.draw()
def add_row(self, row_index: int, values):
"""Store per-frame DC mean values for row_index (1-based) and refresh the image."""
idx = row_index - 1
while len(self._row_data) <= idx:
self._row_data.append(np.empty(0, dtype=np.float32))
self._row_data[idx] = np.asarray(values, dtype=np.float32)
# Build rectangular image — pad shorter rows with NaN so they render distinctly
max_len = max(len(r) for r in self._row_data)
img = np.full((len(self._row_data), max_len), np.nan, dtype=np.float32)
for i, row in enumerate(self._row_data):
img[i, :len(row)] = row
self._placeholder = None
self._reset_axes()
def _reset_axes(self):
self.ax.clear()
self.ax.set_title("DC Bias Preview (ADC counts)")
self.ax.set_xlabel("Frame")
self.ax.set_ylabel("Row")
self.ax.imshow(
img, aspect="auto", origin="upper",
cmap="viridis", interpolation="nearest",
)
self._placeholder = self.ax.text(
0.5, 0.5, "Waiting for data…", transform=self.ax.transAxes,
ha="center", va="center", color="gray")
self._im = None
self.draw_idle()
def begin_scan(self, n_rows: int, n_frames: int):
"""Preallocate the image for a scan of this shape."""
self._img = np.full((max(1, n_rows), max(1, n_frames)), np.nan,
dtype=np.float32)
self._reset_axes()
def add_row(self, row_index: int, values):
"""Store per-frame DC means for row_index (1-based) and refresh."""
row = np.asarray(values, dtype=np.float32)
idx = row_index - 1
# Grow only if the scan turned out taller/wider than announced.
if self._img is None:
self.begin_scan(idx + 1, len(row))
if idx >= self._img.shape[0] or len(row) > self._img.shape[1]:
grown = np.full((max(idx + 1, self._img.shape[0]),
max(len(row), self._img.shape[1])),
np.nan, dtype=np.float32)
grown[:self._img.shape[0], :self._img.shape[1]] = self._img
self._img = grown
self._im = None # extent changed; rebuild the artist once
self._img[idx, :len(row)] = row
if self._im is None:
if self._placeholder is not None:
self._placeholder.remove()
self._placeholder = None
self._im = self.ax.imshow(
self._img, aspect="auto", origin="upper",
cmap="viridis", interpolation="nearest",
)
else:
self._im.set_data(self._img)
finite = self._img[np.isfinite(self._img)]
if finite.size:
self._im.set_clim(float(finite.min()), float(finite.max()))
self.draw_idle()
@@ -103,61 +132,35 @@ BBD_JOG_ACCEL_MM_S2 = 50.0
# ── BBD202 worker ─────────────────────────────────────────────────────────────
class BBD202Worker(QObject):
class BBD202Worker(PollingQueueWorker):
"""Manages ThorlabsServoDriver (MLS203-1) in a worker thread."""
connected = pyqtSignal()
disconnected = pyqtSignal()
connection_failed = pyqtSignal(str)
position_updated = pyqtSignal(float, float) # x_mm, y_mm
homed_status = pyqtSignal(bool, bool) # x_homed, y_homed
error_occurred = pyqtSignal(str)
homed_status = pyqtSignal(bool, bool) # x_homed, y_homed
POLL_INTERVAL_S = 0.2
def __init__(self):
super().__init__()
super().__init__(poll_interval_s=self.POLL_INTERVAL_S)
self.controller: ThorlabsServoDriver | None = None
self._cmd_q: queue.Queue = queue.Queue()
self._running = False
self.is_connected = False
self.scanning_active = False # pause polling during scan
self.scanning_active = False # pause polling during a scan
self._jog_step = BBD_DEFAULT_JOG_MM
self._last_x: float | None = None
self._last_y: float | None = None
self._last_xh: bool | None = None
self._last_yh: bool | None = None
self._last_poll_t = 0.0
self._last_pos: tuple[float, float] | None = None
self._last_homed: tuple[bool, bool] | None = None
self._handlers.update({
"connect": self._do_connect,
"disconnect": self._do_disconnect,
"jog": self._do_jog,
"home_all": self._do_home_all,
"enable_all": self._do_enable_all,
})
@pyqtSlot()
def run(self):
self._running = True
while self._running:
try:
cmd = self._cmd_q.get(timeout=0.05)
self._dispatch(cmd)
except queue.Empty:
pass
if self.is_connected and self.controller and not self.scanning_active:
self._poll()
def _on_stop(self):
if self.controller:
try:
self.controller.disconnect()
except Exception:
pass
def _dispatch(self, cmd: dict):
t = cmd["type"]
if t == "connect":
self._do_connect(cmd["port"])
elif t == "disconnect":
self._do_disconnect()
elif t == "jog":
self._do_jog(cmd["axis"], cmd["direction"])
elif t == "home_all":
self._do_home_all()
elif t == "enable_all":
self._do_enable_all()
elif t == "stop":
self._running = False
def _do_connect(self, port: str):
try:
self.controller = ThorlabsServoDriver()
@@ -173,10 +176,12 @@ class BBD202Worker(QObject):
acceleration=BBD_JOG_ACCEL_MM_S2)
self.is_connected = True
self.connected.emit()
self.start_polling()
except Exception as e:
self.connection_failed.emit(str(e))
def _do_disconnect(self):
self.stop_polling()
if self.controller:
try:
self.controller.disconnect()
@@ -194,8 +199,6 @@ class BBD202Worker(QObject):
self.controller.move_axis_relative(dest, self._jog_step * direction, timeout=2.0)
except TimeoutError:
pass
except Exception as e:
self.error_occurred.emit(str(e))
def _do_home_all(self):
if not self.controller:
@@ -214,79 +217,54 @@ class BBD202Worker(QObject):
self.controller.toggle_enabled_state(AXIS_X)
self.controller.toggle_enabled_state(AXIS_Y)
def _poll(self):
now = time.time()
if now - self._last_poll_t < 0.2:
def _poll_once(self):
"""Emit position/homed updates, but only when they actually change."""
if self.scanning_active or not self.controller:
return
self._last_poll_t = now
try:
x = self.controller.positions[0]
y = self.controller.positions[1]
if x != self._last_x or y != self._last_y:
self._last_x = x
self._last_y = y
self.position_updated.emit(x, y)
xh = self.controller.am_homed[0]
yh = self.controller.am_homed[1]
if xh != self._last_xh or yh != self._last_yh:
self._last_xh = xh
self._last_yh = yh
self.homed_status.emit(xh, yh)
pos = (self.controller.positions[0], self.controller.positions[1])
if pos != self._last_pos:
self._last_pos = pos
self.position_updated.emit(*pos)
homed = (self.controller.am_homed[0], self.controller.am_homed[1])
if homed != self._last_homed:
self._last_homed = homed
self.homed_status.emit(*homed)
except Exception:
pass
# ── Queue API ─────────────────────────────────────────────────────────────
def queue_connect(self, port: str):
self._cmd_q.put({"type": "connect", "port": port})
self._enqueue("connect", port=port)
def queue_disconnect(self):
self._cmd_q.put({"type": "disconnect"})
self._enqueue("disconnect")
def queue_jog(self, axis: str, direction: int):
self._cmd_q.put({"type": "jog", "axis": axis, "direction": direction})
self._enqueue("jog", axis=axis, direction=direction)
def queue_home_all(self):
self._cmd_q.put({"type": "home_all"})
self._enqueue("home_all")
def queue_enable_all(self):
self._cmd_q.put({"type": "enable_all"})
def stop_worker(self):
self._cmd_q.put({"type": "stop"})
self._enqueue("enable_all")
# ── Oscilloscope worker ───────────────────────────────────────────────────────
class OscopeWorker(QObject):
class OscopeWorker(QueueWorker):
"""Manages TektronixOscilloscopeBase in a worker thread."""
connected = pyqtSignal()
disconnected = pyqtSignal()
connection_failed = pyqtSignal(str)
error_occurred = pyqtSignal(str)
def __init__(self):
super().__init__()
self.scope: TektronixOscilloscopeBase | None = None
self._cmd_q: queue.Queue = queue.Queue()
self._running = False
self.is_connected = False
self._handlers.update({
"connect": self._do_connect,
"disconnect": self._do_disconnect,
})
@pyqtSlot()
def run(self):
self._running = True
while self._running:
try:
cmd = self._cmd_q.get(timeout=0.1)
t = cmd["type"]
if t == "connect":
self._do_connect(cmd["ip"])
elif t == "disconnect":
self._do_disconnect()
elif t == "stop":
self._running = False
except queue.Empty:
pass
def _on_stop(self):
if self.scope:
try:
self.scope.disconnect()
@@ -314,67 +292,48 @@ class OscopeWorker(QObject):
self.disconnected.emit()
def queue_connect(self, ip: str):
self._cmd_q.put({"type": "connect", "ip": ip})
self._enqueue("connect", ip=ip)
def queue_disconnect(self):
self._cmd_q.put({"type": "disconnect"})
def stop_worker(self):
self._cmd_q.put({"type": "stop"})
self._enqueue("disconnect")
# ── Helios laser worker ───────────────────────────────────────────────────────
class HeliosWorker(QObject):
"""Manages HeliosLaser in a worker thread using a command queue."""
connected = pyqtSignal()
disconnected = pyqtSignal()
connection_failed = pyqtSignal(str)
class HeliosWorker(PollingQueueWorker):
"""Manages HeliosLaser in a worker thread using a command queue.
Status polling is self-rescheduling: the next poll is queued only once
the previous completes. A free-running 1 s timer used to queue a ~2 s
job, so the queue grew without bound and the panel fell further behind
the longer it stayed connected.
"""
enabled_updated = pyqtSignal(bool)
current_updated = pyqtSignal(int)
diode_temp_updated = pyqtSignal(float)
pstage_temp_updated = pyqtSignal(float)
qswitch_temp_updated = pyqtSignal(float)
status_registers_updated = pyqtSignal(object, object, object) # LER, LCE, CCE (int|None)
error_occurred = pyqtSignal(str)
POLL_INTERVAL_S = 1.0
def __init__(self):
super().__init__()
super().__init__(poll_interval_s=self.POLL_INTERVAL_S)
self._laser: HeliosLaser | None = None
self._cmd_q: queue.Queue = queue.Queue()
self._running = False
self.is_connected = False
self._handlers.update({
"connect": self._do_connect,
"disconnect": self._do_disconnect,
"set_enable": self._do_set_enable,
"set_current": self._do_set_current,
})
@pyqtSlot()
def run(self):
self._running = True
while self._running:
try:
cmd = self._cmd_q.get(timeout=0.1)
self._dispatch(cmd)
except queue.Empty:
pass
def _on_stop(self):
if self._laser:
try:
self._laser.disconnect()
except Exception:
pass
def _dispatch(self, cmd: dict):
t = cmd["type"]
if t == "connect":
self._do_connect(cmd["port"])
elif t == "disconnect":
self._do_disconnect()
elif t == "set_enable":
self._do_set_enable(cmd["enable"])
elif t == "set_current":
self._do_set_current(cmd["current_ma"])
elif t == "poll_status":
self._do_poll_status()
elif t == "stop":
self._running = False
def _do_connect(self, port: str):
try:
self._laser = HeliosLaser(timeout=1.0)
@@ -384,12 +343,13 @@ class HeliosWorker(QObject):
return
self.is_connected = True
self.connected.emit()
self._do_poll_status()
self.start_polling()
except Exception as e:
self._laser = None
self.connection_failed.emit(str(e))
def _do_disconnect(self):
self.stop_polling()
if self._laser:
try:
self._laser.set_laser_enable(False)
@@ -406,75 +366,52 @@ class HeliosWorker(QObject):
def _do_set_enable(self, enable: bool):
if not self._laser:
return
try:
self._laser.set_laser_enable(enable)
self.enabled_updated.emit(self._laser.is_laser_enabled())
except Exception as e:
self.error_occurred.emit(str(e))
self._laser.set_laser_enable(enable)
self.enabled_updated.emit(self._laser.is_laser_enabled())
def _do_set_current(self, current_ma: int):
if not self._laser:
return
try:
self._laser.set_current_ma(current_ma)
self.current_updated.emit(current_ma)
except Exception as e:
self.error_occurred.emit(str(e))
self._laser.set_current_ma(current_ma)
self.current_updated.emit(current_ma)
def _do_poll_status(self):
def _poll_once(self):
"""Read the full status set. Individual reads are allowed to fail
(a timed-out register shouldn't suppress the rest of the panel)."""
if not self._laser or not self._laser.is_connected:
return
try:
ler, lce, cce = self._laser.get_status_registers()
self.status_registers_updated.emit(ler, lce, cce)
self.status_registers_updated.emit(*self._laser.get_status_registers())
except Exception:
pass
try:
self.enabled_updated.emit(self._laser.is_laser_enabled())
except Exception:
pass
try:
current = self._laser.get_current_ma()
if current is not None:
self.current_updated.emit(current)
except Exception:
pass
try:
t = self._laser.get_diode_temp_c()
if t is not None:
self.diode_temp_updated.emit(t)
except Exception:
pass
try:
t = self._laser.get_power_stage_temp_c()
if t is not None:
self.pstage_temp_updated.emit(t)
except Exception:
pass
try:
t = self._laser.get_qswitch_temp_c()
if t is not None:
self.qswitch_temp_updated.emit(t)
except Exception:
pass
for getter, signal in (
(self._laser.get_current_ma, self.current_updated),
(self._laser.get_diode_temp_c, self.diode_temp_updated),
(self._laser.get_power_stage_temp_c, self.pstage_temp_updated),
(self._laser.get_qswitch_temp_c, self.qswitch_temp_updated),
):
try:
value = getter()
if value is not None:
signal.emit(value)
except Exception:
pass
def queue_connect(self, port: str):
self._cmd_q.put({"type": "connect", "port": port})
self._enqueue("connect", port=port)
def queue_disconnect(self):
self._cmd_q.put({"type": "disconnect"})
self._enqueue("disconnect")
def queue_set_enable(self, enable: bool):
self._cmd_q.put({"type": "set_enable", "enable": enable})
self._enqueue("set_enable", enable=enable)
def queue_set_current(self, current_ma: int):
self._cmd_q.put({"type": "set_current", "current_ma": current_ma})
def queue_poll_status(self):
self._cmd_q.put({"type": "poll_status"})
def stop_worker(self):
self._cmd_q.put({"type": "stop"})
self._enqueue("set_current", current_ma=current_ma)
# ── Camera window popup ───────────────────────────────────────────────────────
@@ -633,17 +570,13 @@ class HeliosWindow(QWidget):
self.setWindowTitle("Helios Laser")
self._worker = worker
self._poll_timer = QTimer(self)
self._poll_timer.setInterval(1000)
self._poll_timer.timeout.connect(self._worker.queue_poll_status)
# Polling is driven by the worker itself (self-rescheduling), so
# there is no timer here to outpace the device.
self._wire_signals()
self._update_controls(False)
self._set_emission_indicator(False)
def closeEvent(self, event):
self._poll_timer.stop()
if self._worker.is_connected:
self._worker.queue_disconnect()
super().closeEvent(event)
@@ -683,7 +616,6 @@ class HeliosWindow(QWidget):
DEFAULTS.save()
self._worker.queue_connect(port)
else:
self._poll_timer.stop()
self._worker.queue_disconnect()
def _on_connected(self):
@@ -691,10 +623,8 @@ class HeliosWindow(QWidget):
self.helios_connect_toggle.setText("Disconnect")
self.helios_status_label.setText("Connected")
self._update_controls(True)
self._poll_timer.start()
def _on_disconnected(self):
self._poll_timer.stop()
self.helios_connect_toggle.blockSignals(True)
self.helios_connect_toggle.setChecked(False)
self.helios_connect_toggle.setText("Connect")
@@ -905,6 +835,10 @@ class ScanProgressWindow(QWidget):
def update_dc_bias_plot(self, row: int, values):
self.bias_image_widget.add_row(row, values)
def begin_scan(self, n_rows: int, n_frames: int):
"""Size the DC preview for the scan about to run."""
self.bias_image_widget.begin_scan(n_rows, n_frames)
# ── Scan worker ───────────────────────────────────────────────────────────────
@@ -1342,6 +1276,8 @@ class MainWindow(QMainWindow):
0, plan.per_angle[ai0].n_rows,
0 if resume is None else ai0 + 1, plan.n_angles
)
self._scan_progress.begin_scan(plan.per_angle[ai0].n_rows,
plan.per_angle[ai0].n_frames)
self._scan_progress.show()
self._scan_thread.start()