scanengine-3/README.md

# scanengine-3

SRAS Scanning and Instrumentation Control Platform

## Overview

scanengine-3 is a unified platform for scanning acoustic microscopy and precision instrumentation control. It integrates multiple hardware control modules into a single cohesive PyQt6-based application.

### Key Features

- **Stage Control**: ThorLabs BBD202/BBD203 motor controller with 3-axis positioning
- **Laser Systems**: Helios pulsed laser and Genesis CW laser control
- **Data Acquisition**: Tektronix oscilloscope integration with fast-frame support
- **Scan Planning**: Automated raster scan generation and execution
- **Per-Angle Background**: every angle opens with its own background
  capture (Genesis off, Helios on), stored ahead of that angle's data
- **Angle Inspection**: Park the rig at random points across a plan's angles
  to check the SAW response on the scope before committing to a long scan
- **SAW Quality Check**: Acquire one row per angle — the row-wise middle of
  the ROI — as a v11 `.sras`, then compare every angle's SAW frequency on one
  graph to judge the alignment before a full run
- **Auto-Align**: Level the sample from the camera window — step the stage
  1.5 mm either side on X and then Y, tilt the T-axes until the DC bias levels
  read what they read at the reference point, and leave the correction applied
- **Real-time Monitoring**: Live status updates and progress tracking

## Hardware Components

### Motion Control
- **ThorLabs BBD202/BBD203 Motor Controller**
  - 3-channel APT protocol driver
  - Precision positioning with encoder feedback
  - Programmable velocity and acceleration
  - Trigger output support for synchronized data acquisition

### Laser Systems
- **Helios Laser System**
  - Frequency control (16.7-125 kHz)
  - Current control (0-7000 mA)
  - Multiple pulse modes
  - Temperature and power monitoring

### Data Acquisition
- **Tektronix MSO/DPO Series Oscilloscopes**
  - Direct socket communication (no VISA overhead)
  - Fast-frame acquisition for high-speed scanning
  - Multi-channel waveform capture
  - Configurable triggering

### Rotation / Focus
- **T3R four-channel stepper controller**
  - Focus axis plus the GR rotation stage (12.5:1 gear train)
  - Custom binary framing protocol over USB serial

## Project Structure

The codebase is split so that everything needed to run a scan is importable
without PyQt6 or any vendor SDK — `core/` is the headless engine, `gui/` is
the shared Qt layer, and the root scripts are entry points.

```
scanengine-3/
├── core/                          # Headless: no PyQt6, no vendor SDKs
│   ├── scan_engine.py             # ScanEngine — full acquisition sequence
│   ├── scan_geometry.py           # ScanPlan, rotated-bbox planning, limits
│   ├── scan_resume.py             # Resume planning (frontier rule)
│   ├── scope_sras.py              # Oscilloscope SCPI policy for SRAS
│   ├── scope_burst.py             # Burst-mode FastFrame sizing + row splitting
│   ├── scope_inspect.py           # Scope setup for inspection + bias read-back
│   ├── angle_inspect.py           # AngleInspector — park on a point per angle
│   ├── auto_align.py              # AutoAligner — tilt the sample level on the DC levels
│   ├── saw_check.py               # Middle-row SAW check: plan + alignment read-out
│   ├── rotation.py                # GR rotation axis settings + moves
│   ├── sras_format.py             # v7/v11 .sras writer, v6/v10 reader (mmap)
│   ├── sras_analysis.py           # Image reducers + SAW matched filter
│   └── config.py                  # ScanDefaults ⇄ aui_defaults.json
│
├── hardware/                      # Device drivers (Qt-free)
│   ├── serial_util.py             # Shared 8N1 open + port enumeration
│   ├── t3r_driver.py              # T3R stepper controller
│   ├── t3r_protocol.py            # T3R frame encode/decode
│   ├── helios_laser.py            # Helios pulsed laser
│   ├── tektronix_base.py          # Tektronix oscilloscope (raw SCPI)
│   ├── uc480_camera.py            # IDS/ThorLabs uEye camera (returns QImage)
│   ├── genesis_core.py            # Genesis laser — QUARANTINED, see below
│   └── pybbd202/                  # ThorLabs BBD202 stage (APT protocol)
│
├── gui/                           # Shared PyQt6 layer
│   ├── scan_bridge.py             # QtScanController over core.scan_engine
│   ├── inspect_bridge.py          # QtAngleInspector over core.angle_inspect
│   ├── align_bridge.py            # QtAutoAligner over core.auto_align
│   ├── qt_t3r.py                  # Qt adapter over the T3R driver
│   ├── qt_workers.py              # QueueWorker / PollingQueueWorker bases
│   ├── jog_panel.py               # T3R + BBD202 jog controls (camera window)
│   └── widgets.py                 # ConnectionBar, LogConsole, PortSelector…
│
├── sc3_aui_app.py                 # Main acquisition application
├── sras_viewer.py                 # Scan data viewer
├── saw_check_viewer.py            # SAW check viewer: every angle's frequency, one graph
├── sras_scan_manager.py           # CLI: inspect/export/delete angles
├── t3r_control_panel.py           # T3R panel (used by the main app)
├── helios_test_app.py             # Per-device test benches
├── bbd202_test_app.py
├── camera_test_app.py
├── sc3-aui-*.ui                   # Qt Designer files loaded at runtime
│
├── tests/                         # pytest suite
│   ├── golden/                    # legacy v6 .sras + geometry fixtures
│   ├── fakes.py                   # Recording fake stage/scope/rotator
│   └── test_*.py
│
├── docs/
│   ├── hardware/                  # Driver notes
│   ├── protocols/                 # Vendor protocol PDFs
│   └── genesis_verification.md    # Bench checklist (see KNOWN_ISSUES.md)
│
├── lib/                           # Vendored IDS uEye SDK (not in git)
├── aui_defaults.json              # Persisted ports / scope IP / save dir
├── scan_format.md                 # .sras binary format specification
├── KNOWN_ISSUES.md                # Open questions needing the hardware
└── requirements.txt
```

## Quick Start

### Installation

```bash
# Clone or navigate to project directory
cd scanengine-3

# Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

### Running the Application

```bash
# Main acquisition application
python sc3_aui_app.py

# Scan data viewer
python sras_viewer.py

# SAW quality check viewer (every angle's frequency on one graph)
python saw_check_viewer.py path/to/scan-sawcheck.sras

# Inspect / export / delete angles in a .sras file
python sras_scan_manager.py path/to/scan.sras

# Per-device test benches
python helios_test_app.py
python bbd202_test_app.py
python camera_test_app.py

# Genesis laser control tool
python tools/genesis_laser_control.py
```

### Running the tests

The suite is hardware-free: fake drivers and committed fixtures stand in
for the rig.

```bash
pip install pytest ruff
python -m pytest tests/ -q
```

## Dependencies

- **PyQt6** (>=6.4.0) - GUI framework
- **pyserial** (>=3.5) - Serial communication
- **pyvisa** (>=1.13.0) - VISA instrument control
- **pyvisa-py** (>=0.7.0) - Pure Python VISA backend
- **pyftdi** (>=0.54.0) - FTDI USB device support
- **numpy** (>=1.20.0) - Array processing
- **scipy** (>=1.10) - Signal processing (viewer SAW pipeline)
- **matplotlib** (>=3.7) - Plotting (viewer, live scan preview)
- **pyueye** (>=4.95.0) - IDS uEye camera SDK bindings (camera only)

## Known hardware caveats

`hardware/genesis_core.py` is quarantined: it diverges from the reference
implementation in `tools/genesis_laser_gui.py` in ways that need the laser
on the bench to settle. See [KNOWN_ISSUES.md](KNOWN_ISSUES.md) and
[docs/genesis_verification.md](docs/genesis_verification.md) before
changing either file.

## Usage Examples

### Running a scan without any GUI

The acquisition sequence lives in `core.scan_engine` and takes plain
drivers plus callbacks, so a script (or a future simpler GUI) can drive the
identical scan the main app runs:

```python
from pathlib import Path
from core.scan_engine import ScanCallbacks, ScanEngine
from core.scan_geometry import build_plan
from core.rotation import RotationAxis
from hardware.pybbd202 import ThorlabsServoDriver
from hardware.tektronix_base import TektronixOscilloscopeBase
from hardware.t3r_driver import T3RDriver

plan = build_plan(x_start=10.0, y_start=10.0, x_delta=20.0, y_delta=10.0,
                  num_angles=3, row_spacing=0.25,
                  laser_freq_hz=20000.0, velocity_mm_s=100.0)

stage = ThorlabsServoDriver(); stage.connect("/dev/ttyUSB0")
scope = TektronixOscilloscopeBase("192.168.100.105"); scope.connect()
t3r = T3RDriver(); t3r.open("/dev/ttyACM0")

engine = ScanEngine(stage, scope, RotationAxis(t3r), plan,
                    Path("/data/SRAS/demo.sras"),
                    callbacks=ScanCallbacks(on_status=print,
                                            prompt=lambda t, m: input(f"{t}: {m} ")))
result = engine.run()          # blocking; engine.abort() is thread-safe
print(f"wrote {result.rows_written} rows to {result.path}")
```

### Running a SAW quality check

Same engine, same hardware sequence — the plan is reduced to one row per
angle and the result is tagged v11 so the viewer knows it is a check rather
than a scan cut short:

```python
from core.saw_check import alignment_summary, frequency_traces, middle_row_plan
from core.sras_format import VERSION_SAW_CHECK, SrasFile

check = middle_row_plan(plan)              # the plan above: 163 rows → 3
engine = ScanEngine(stage, scope, RotationAxis(t3r), check,
                    Path("/data/SRAS/demo-sawcheck.sras"),
                    callbacks=ScanCallbacks(on_status=print),
                    file_version=VERSION_SAW_CHECK)
engine.run()

with SrasFile("/data/SRAS/demo-sawcheck.sras") as sras:
    traces = frequency_traces(sras, dc_threshold_mv=50.0)
    for t in traces:
        print(f"{t.angle_deg:+7.1f}°  {t.median_mhz:.2f} MHz  "
              f"drift {t.drift_mhz_per_mm:+.3f} MHz/mm")
    print(alignment_summary(traces).describe())
```

`saw_check_viewer.py` is the same read-out with the curves drawn.

### Levelling the sample (auto-align)

Two phases, because the operator sits between them: `prepare()` configures
the rig and reads the DC levels where the stage stands, and `run()` only
starts once those levels have been confirmed as the ones to hold.

```python
from core.auto_align import AlignCallbacks, AutoAligner

aligner = AutoAligner(stage, scope, t3r,
                      callbacks=AlignCallbacks(on_status=print))
reference = aligner.prepare()              # scope + T-axes configured, one reading
print(reference.describe())                # "is the image correct?" happens here
result = aligner.run()                     # X on T1, then Y on T0/T2
print(result.describe())
aligner.stop()                             # stage parked; the tilt stays applied
```

The scope has to be cabled CH1 SAW / CH2 trigger / CH3 DC 1 / CH4 DC 2 — the
same channels a scan uses, except that CH3 carries the DC monitor here rather
than the max-velocity gate. Nothing rewires it; the app asks the operator to
confirm the cabling, and refuses to servo on a scope that is not triggering.

In the main app the button is in the camera window, because judging the image
is the first step of the procedure.

### Reading a scan file

`SrasFile` memory-maps the data block, so opening a multi-gigabyte scan
costs only the pages actually touched:

```python
from core.sras_format import SrasFile
from core.sras_analysis import CH4_IDX, ChannelCalibration, compute_dc_image

with SrasFile("/data/SRAS/demo.sras") as sras:
    print(sras.header.n_angles, "angles")
    for st in sras.angle_status():          # handles aborted/partial files
        print(f"  angle {st.index}: {st.n_rows_available}/{st.n_rows} rows ({st.status})")

    view = sras.load_angle(0)               # (rows, channels, frames, samples)
    calib = ChannelCalibration.from_preambles(sras.preambles)
    dc_mv = calib.adc_to_mv(compute_dc_image(view, CH4_IDX), CH4_IDX)
```

### Stage control

```python
from hardware.pybbd202 import AXIS_X, AXIS_Y, ThorlabsServoDriver

stage = ThorlabsServoDriver()
stage.connect("/dev/ttyUSB0")               # raises if no bay responds
stage.enable_axis(AXIS_X)
stage.home_axis(AXIS_X, timeout=120.0)
stage.move_axis_absolute(AXIS_X, 25.0, timeout=30.0)
```

### Oscilloscope acquisition

```python
from core.scope_sras import configure_acquisition, configure_channels
from hardware.tektronix_base import TektronixOscilloscopeBase

scope = TektronixOscilloscopeBase("192.168.100.105", port=4000)
scope.connect()
configure_channels(scope)                   # standard SRAS front-end setup
samples_per_frame = configure_acquisition(scope)
```

### Laser control

```python
from hardware.helios_laser import HeliosLaser

laser = HeliosLaser()
laser.connect("/dev/ttyUSB1")
laser.set_current_ma(1200)
laser.set_laser_enable(True)
print(laser.get_diode_temp_c(), "°C")
laser.disconnect()                          # always explicit — no __del__
```

### Camera control

```python
from hardware.uc480_camera import UC480Camera, find_camera_bus_conflicts

find_camera_bus_conflicts()                 # warns about USB bus contention
camera = UC480Camera(camera_id=1)
camera.initialize()
camera.start_capture()
```

## Configuration

### Persisted settings
`aui_defaults.json` holds the ports, scope IP, and save directory the main
app last used. It is read and written through `core.config.ScanDefaults`,
which always writes every field — see KNOWN_ISSUES.md history for why
partial writes were a problem.

### Fixed acquisition settings
Scan velocity, laser frequency, sample rate, and the ramp geometry are
constants in `core/scan_engine.py` and `core/scope_sras.py`, not user
settings; a `.sras` file records them so resume can refuse a mismatch.

### Serial port configuration
- **BBD202**: USB serial, APT protocol (`/dev/ttyUSB*`)
- **T3R**: USB serial, custom binary framing (`/dev/ttyACM*`)
- **Helios**: RS-232 (9600 baud, 8N1)
- **Genesis**: USB serial, I2C-over-serial
- **Oscilloscope**: Ethernet/LXI (TCP socket on port 4000)

## Development

### Adding New Hardware

1. Create driver module in `hardware/` directory
2. Implement connection, control, and status methods
3. Add UI elements to main window or create new dialog
4. Connect signals in `main_window.py`

### Testing Without Hardware

All hardware modules include stub implementations or simulation modes. The GUI can be developed and tested without physical devices connected.

## Documentation

Detailed documentation available in project subdirectories:

- [BBD202/203 Driver Guide](docs/hardware/BBD203_DRIVER_README.md)
- [BBD202/203 Connection Guide](docs/hardware/BBD203_CONNECTION_GUIDE.md)
- [BBD202/203 Communications Protocol](docs/hardware/BBD203_Communications_Protocol.md)
- [Helios Laser Guide](docs/hardware/HELIOS_DRIVER_README.md)
- [Genesis Laser Guide](docs/hardware/GENESIS_LASER_README.md)
- [Laser Control Implementation Guide](docs/hardware/laser_control_implementation_guide.md)
- [Setup Instructions](SETUP.md)

## License

Copyright (C) 2025 Thomas Ales
Licensed under GNU General Public License v2.0

See LICENSE file for full license text.

## Support

For issues, questions, or contributions, please refer to the project documentation or contact the development team.

## Version

scanengine-3 v0.1.0 - Initial unified release