scanengine-3/scan_format.md

# SRAS Scan Binary Format — Version 6

Each `.sras` file contains **one complete scan**: all GR rotation angles and all
Y rows.  Files are named `{prefix}.sras`.

Starting in v6, each angle only scans the **bounding box of the nominal ROI
rotated by that specific angle** — not the worst case across all angles — so
`x_start`, `x_delta` (and therefore `n_frames`, the points/row count) and
`n_rows` all vary per angle. A 0°/180° scan of a wide, short ROI needs far
fewer rows than a 45° scan of the same ROI, and the file format reflects that
instead of forcing every angle to the largest bounding box.

---

## File Layout

```
[Global Header           — 49 bytes]
[Angle Table             — n_angles × 4 bytes  (float32 per angle, degrees)]
[Per-Angle Geometry Table— n_angles × 14 bytes (x_start f32, x_delta f32, n_frames u32, n_rows u16)]
[Row Table (ragged)      — sum(n_rows) × 4 bytes (float32 per row, angle-major)]
[Preamble Blocks         — n_channels × (uint16 length + UTF-8 WFMOutpre string)]
[Background Block        — uint32 n_bg_samples + n_bg_samples × int8 bytes]
[Waveform Data (ragged)  — per angle: n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bps bytes]
```

All multi-byte integers and floats use **big-endian** byte order
(`>` in Python's `struct` module).

---

## Global Header (49 bytes)

| Offset | Size | Type      | Field              | Description                                      |
|--------|------|-----------|--------------------|--------------------------------------------------|
|  0     |  4   | `4s`      | `magic`            | Always `SRAS` (0x53 0x52 0x41 0x53)              |
|  4     |  1   | `uint8`   | `version`          | Format version — `6`                             |
|  5     |  2   | `uint16`  | `n_angles`         | Number of GR rotation angles                     |
|  7     |  4   | `float32` | `x_start_nominal`  | Nominal (pre-rotation) X scan start, mm          |
| 11     |  4   | `float32` | `y_start_nominal`  | Nominal (pre-rotation) Y scan start, mm          |
| 15     |  4   | `float32` | `x_delta_nominal`  | Nominal (pre-rotation) X scan width, mm          |
| 19     |  4   | `float32` | `y_delta_nominal`  | Nominal (pre-rotation) Y scan height, mm         |
| 23     |  4   | `float32` | `row_spacing_mm`   | Y spacing between rows, mm                       |
| 27     |  4   | `float32` | `velocity_mm_s`    | Stage scan velocity in mm/s                      |
| 31     |  4   | `float32` | `laser_freq_hz`    | Laser repetition rate in Hz                      |
| 35     |  4   | `uint32`  | `samples_per_frame`| Time samples per waveform                        |
| 39     |  8   | `float64` | `sample_rate_hz`   | Oscilloscope sample rate in Hz (e.g. 6.25e9)     |
| 47     |  1   | `uint8`   | `bytes_per_sample` | Bytes per ADC sample: `1` = int8, `2` = int16    |
| 48     |  1   | `uint8`   | `n_channels`       | Number of channels recorded (currently `3`)       |

**Total header size:** 49 bytes — verified:
`struct.calcsize(">4sBHfffffffIdBB") == 49`.

The `*_nominal` fields describe the ROI as originally entered on the New Scan
page (XS/YS/XD/YD), **before** per-angle bounding-box expansion. They are for
reference/reconstruction only — the actual per-angle scan geometry used for
acquisition is in the Per-Angle Geometry Table below.

---

## Angle Table

Immediately after the header: **n_angles** big-endian float32 values, one per
GR angle (degrees, signed; magnitude 0–180, sign gives physical rotation
direction — negative for the current CW-rotating GR stage).

```
angle[0], angle[1], …, angle[n_angles - 1]
```

---

## Per-Angle Geometry Table

Immediately after the angle table: **n_angles** fixed-size records, one per
angle (same order as the angle table), each 14 bytes:

| Size | Type      | Field      | Description                                          |
|------|-----------|------------|-------------------------------------------------------|
| 4    | `float32` | `x_start`  | X scan start for this angle's bounding box, mm         |
| 4    | `float32` | `x_delta`  | X scan width for this angle's bounding box, mm         |
| 4    | `uint32`  | `n_frames` | A-scans per row for this angle (FastFrame count)       |
| 2    | `uint16`  | `n_rows`   | Number of Y rows scanned for this angle                |

Format string per record: `">ffIH"`.

---

## Row Table (ragged)

Immediately after the per-angle geometry table: for each angle in order,
that angle's `n_rows` big-endian float32 Y positions (mm), concatenated with
no padding between angles.

```
# angle 0's rows, then angle 1's rows, …
y_mm[0][0], …, y_mm[0][n_rows[0]-1], y_mm[1][0], …, y_mm[n_angles-1][n_rows[-1]-1]
```

Row-table boundaries for angle *a* are derived from the per-angle geometry
table: `sum(n_rows[0:a])` gives the starting index into the flattened array.

---

## Preamble Blocks

Immediately after the row table: **n_channels** length-prefixed UTF-8 strings,
one per channel in `SCAN_CHANNELS` order (CH1, CH3, CH4).  Each block is:

```
uint16  length   — byte length of the following UTF-8 string
bytes   preamble — WFMOutpre response string from the oscilloscope
```

The preamble captures per-channel scaling constants (YMULT, YOFF, YZERO) needed
to convert raw ADC values to volts.

---

## Background Block

Immediately after the preamble blocks: a single CH1 waveform captured with the
**Helios (generation) laser enabled** and the **Genesis (detection) laser
disabled**.  This provides a noise/background reference for subtraction during
post-processing.

```
uint32  n_bg_samples   — number of samples in the background waveform
int8[]  bg_data        — raw ADC samples (same encoding as waveform data)
```

`n_bg_samples` equals `samples_per_frame` under normal acquisition settings.

---

## Waveform Data (ragged)

Immediately after the background block.  Data is stored in **angle-major,
row-minor** order, but unlike earlier versions each angle contributes a
different number of rows (`n_rows[a]`) and a different number of frames per
row (`n_frames[a]`), both taken from that angle's Per-Angle Geometry Table
entry.  Within each row, channels are interleaved in ascending channel-index
order, with each channel's FastFrame data written in frame order.

```
for angle a in 0 … n_angles-1:
  for row in 0 … n_rows[a]-1:
    for channel in [CH1, CH3, CH4]:          # 3 channels, fixed order
      for frame in 0 … n_frames[a]-1:
        samples[0 … samples_per_frame-1]     # bps bytes each
```

Each sample is a raw signed ADC value.  With `bytes_per_sample = 1` this is
**int8** (−128 … +127).  With `bytes_per_sample = 2` this is **big-endian
int16**.

Total data size:
```
sum over angles a of: n_rows[a] × 3 × n_frames[a] × samples_per_frame × bytes_per_sample
```

> **Incomplete files:** If a scan is aborted the file is closed immediately and
> the data block will be shorter than the expected size. Readers should
> reconstruct the expected per-angle byte offsets from the Per-Angle Geometry
> Table and check `file_size` against the running total before reshaping —
> a fixed `(n_angles, n_rows, ...)` reshape (as in pre-v6 readers) will not
> work since row/frame counts are no longer uniform across angles.

---

## Spatial Mapping

The *k*-th waveform (frame) in a row corresponds to the *k*-th laser pulse that
hit the sample.  For a row belonging to angle *a*, the physical X position of
that pulse is:

```
x_k = x_start[a] + k * (velocity_mm_s / laser_freq_hz)
```

using that angle's `x_start` from the Per-Angle Geometry Table (not
`x_start_nominal`).

---

## Acquisition Settings (fixed by core/scope_sras.py)

| Parameter            | Value                                    |
|-----------------------|------------------------------------------|
| Setup trigger        | CH2, rising edge, 0.500 V (`TRIG_LEVEL_V`) |
| Scan trigger         | Logic AND, CH2 HIGH ∧ CH3 HIGH, 0.500 V  |
| Horizontal position  | 30 (`HORizontal:POSition`)               |
| Sample rate          | 6.25 GS/s  (160 ps/sample)               |
| Transfer format      | `DATa:ENCdg RIBinary`, `DATa:WIDth 1`    |
| Channels recorded    | CH1, CH3, CH4                            |
| Stage X velocity     | 100 mm/s                                 |
| Stage X acceleration | 1500 mm/s²                               |
| Stage X trigger out  | Logic-high at max velocity (`TRIGOUT_MAXV`) |
| Acquisition mode     | FastFrame, Normal trigger                |

None of these are stored in the file, so they do not affect byte layout — but
they do set where the acoustic packet lands inside each frame.  Read them from
`core/scope_sras.py`; earlier revisions of this table drifted from the code.

---

## Acquisition Paths

Two acquisition strategies write **byte-identical** files; the choice is a
runtime flag (`ScanEngine(burst_mode=…)`, exposed as a checkbox in the app) and
is not recorded in the file.

| | Per-row (default) | Burst |
|---|---|---|
| FastFrame acquisitions | one per row | one per `floor(max_frames / n_frames)` rows |
| Curve transfers | one per channel per row | one per channel per burst |
| Stage X trigger out | armed for the whole scan | armed per acquiring pass, dropped for the flyback |

Burst mode runs a single acquisition across several rows, so the return move
must not trigger: the trigger output is dropped before each flyback and
re-armed for each acquiring pass.  Row boundaries inside the burst come from
`ACQuire:NUMFRAMESACQuired?` sampled after each pass — the burst itself carries
no row markers.  See `core/scope_burst.py`.

Either path squares each row up to the declared `n_frames` (zero-padding a
short row, dropping the tail of a long one), because the format has no per-row
length field and a mismatch would shift every later row.

---

## Version History

| Version | Change |
|---------|--------|
| 1       | One file per row; header included `angle_idx`, `row_idx`, `angle_deg`, `y_mm`. |
| 2       | One file per scan; global header with `n_angles`/`n_rows`; separate angle and row tables; three channels (CH1, CH3, CH4) per row. |
| 3       | Added preamble blocks (WFMOutpre strings) after the row table, one length-prefixed UTF-8 block per channel. |
| 4       | Added background waveform block (CH1, Helios ON / Genesis OFF) after the preamble blocks; stored as `uint32` sample count followed by raw `int8` ADC bytes. |
| 5       | (skipped) |
| 6       | Each angle now scans only the bounding box of the nominal ROI rotated by that angle instead of the AABB-expanded worst case across all angles. Header no longer carries a single global `x_start`/`x_delta`/`n_rows` — replaced with `*_nominal` reference fields plus a new Per-Angle Geometry Table (`x_start`, `x_delta`, `n_frames`, `n_rows` per angle) and a ragged Row Table / Waveform Data block sized per angle. **Not compatible with v4 readers** (e.g. `sras_viewer.py`, which has not yet been updated for v6). |