Thomas Ales
52fdcdd9f3
A mis-triggered row cannot be written as it arrived — v6 declares n_frames per row in the header and has no per-row length field, so a short or long row would shift every later row in the file. Until now the only policy was to square it up, which keeps the scan running but leaves the affected row indistinguishable from a good one afterwards: nothing in the file records that it was padded. strict_rows selects the other trade. On any frame-count mismatch the scan stops instead of writing the row, so a data run either produces rows that mean what the header says they mean or fails loudly. Default stays pad, so existing behaviour is unchanged. _warn_frame_delta becomes _check_frame_delta, since it now decides rather than just reports. Both acquisition paths already call it before writing anything for the row (CH1 leads SCAN_CHANNELS, and the burst path checks every row up front), so an abort leaves the file on a whole-row boundary rather than a half-written row — test_strict_row_packing_writes_nothing_for_the_failed_row pins that. Plumbed through QtScanController to a checkbox in the scan panel, persisted in ScanDefaults alongside burst_mode. scan_format.md documents both policies and notes that the choice is not recorded in the file. The row-clipping setup in the padding test is now a _clip_one_row helper, reused by the strict tests. 92 tests passing, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
256 lines
11 KiB
Markdown
Executable File
256 lines
11 KiB
Markdown
Executable File
# SRAS Scan Binary Format — Version 6
|
||
|
||
Each `.sras` file contains **one complete scan**: all GR rotation angles and all
|
||
Y rows. Files are named `{prefix}.sras`.
|
||
|
||
Starting in v6, each angle only scans the **bounding box of the nominal ROI
|
||
rotated by that specific angle** — not the worst case across all angles — so
|
||
`x_start`, `x_delta` (and therefore `n_frames`, the points/row count) and
|
||
`n_rows` all vary per angle. A 0°/180° scan of a wide, short ROI needs far
|
||
fewer rows than a 45° scan of the same ROI, and the file format reflects that
|
||
instead of forcing every angle to the largest bounding box.
|
||
|
||
---
|
||
|
||
## File Layout
|
||
|
||
```
|
||
[Global Header — 49 bytes]
|
||
[Angle Table — n_angles × 4 bytes (float32 per angle, degrees)]
|
||
[Per-Angle Geometry Table— n_angles × 14 bytes (x_start f32, x_delta f32, n_frames u32, n_rows u16)]
|
||
[Row Table (ragged) — sum(n_rows) × 4 bytes (float32 per row, angle-major)]
|
||
[Preamble Blocks — n_channels × (uint16 length + UTF-8 WFMOutpre string)]
|
||
[Background Block — uint32 n_bg_samples + n_bg_samples × int8 bytes]
|
||
[Waveform Data (ragged) — per angle: n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bps bytes]
|
||
```
|
||
|
||
All multi-byte integers and floats use **big-endian** byte order
|
||
(`>` in Python's `struct` module).
|
||
|
||
---
|
||
|
||
## Global Header (49 bytes)
|
||
|
||
| Offset | Size | Type | Field | Description |
|
||
|--------|------|-----------|--------------------|--------------------------------------------------|
|
||
| 0 | 4 | `4s` | `magic` | Always `SRAS` (0x53 0x52 0x41 0x53) |
|
||
| 4 | 1 | `uint8` | `version` | Format version — `6` |
|
||
| 5 | 2 | `uint16` | `n_angles` | Number of GR rotation angles |
|
||
| 7 | 4 | `float32` | `x_start_nominal` | Nominal (pre-rotation) X scan start, mm |
|
||
| 11 | 4 | `float32` | `y_start_nominal` | Nominal (pre-rotation) Y scan start, mm |
|
||
| 15 | 4 | `float32` | `x_delta_nominal` | Nominal (pre-rotation) X scan width, mm |
|
||
| 19 | 4 | `float32` | `y_delta_nominal` | Nominal (pre-rotation) Y scan height, mm |
|
||
| 23 | 4 | `float32` | `row_spacing_mm` | Y spacing between rows, mm |
|
||
| 27 | 4 | `float32` | `velocity_mm_s` | Stage scan velocity in mm/s |
|
||
| 31 | 4 | `float32` | `laser_freq_hz` | Laser repetition rate in Hz |
|
||
| 35 | 4 | `uint32` | `samples_per_frame`| Time samples per waveform |
|
||
| 39 | 8 | `float64` | `sample_rate_hz` | Oscilloscope sample rate in Hz (e.g. 6.25e9) |
|
||
| 47 | 1 | `uint8` | `bytes_per_sample` | Bytes per ADC sample: `1` = int8, `2` = int16 |
|
||
| 48 | 1 | `uint8` | `n_channels` | Number of channels recorded (currently `3`) |
|
||
|
||
**Total header size:** 49 bytes — verified:
|
||
`struct.calcsize(">4sBHfffffffIdBB") == 49`.
|
||
|
||
The `*_nominal` fields describe the ROI as originally entered on the New Scan
|
||
page (XS/YS/XD/YD), **before** per-angle bounding-box expansion. They are for
|
||
reference/reconstruction only — the actual per-angle scan geometry used for
|
||
acquisition is in the Per-Angle Geometry Table below.
|
||
|
||
---
|
||
|
||
## Angle Table
|
||
|
||
Immediately after the header: **n_angles** big-endian float32 values, one per
|
||
GR angle (degrees, signed; magnitude 0–180, sign gives physical rotation
|
||
direction — negative for the current CW-rotating GR stage).
|
||
|
||
```
|
||
angle[0], angle[1], …, angle[n_angles - 1]
|
||
```
|
||
|
||
---
|
||
|
||
## Per-Angle Geometry Table
|
||
|
||
Immediately after the angle table: **n_angles** fixed-size records, one per
|
||
angle (same order as the angle table), each 14 bytes:
|
||
|
||
| Size | Type | Field | Description |
|
||
|------|-----------|------------|-------------------------------------------------------|
|
||
| 4 | `float32` | `x_start` | X scan start for this angle's bounding box, mm |
|
||
| 4 | `float32` | `x_delta` | X scan width for this angle's bounding box, mm |
|
||
| 4 | `uint32` | `n_frames` | A-scans per row for this angle (FastFrame count) |
|
||
| 2 | `uint16` | `n_rows` | Number of Y rows scanned for this angle |
|
||
|
||
Format string per record: `">ffIH"`.
|
||
|
||
---
|
||
|
||
## Row Table (ragged)
|
||
|
||
Immediately after the per-angle geometry table: for each angle in order,
|
||
that angle's `n_rows` big-endian float32 Y positions (mm), concatenated with
|
||
no padding between angles.
|
||
|
||
```
|
||
# angle 0's rows, then angle 1's rows, …
|
||
y_mm[0][0], …, y_mm[0][n_rows[0]-1], y_mm[1][0], …, y_mm[n_angles-1][n_rows[-1]-1]
|
||
```
|
||
|
||
Row-table boundaries for angle *a* are derived from the per-angle geometry
|
||
table: `sum(n_rows[0:a])` gives the starting index into the flattened array.
|
||
|
||
---
|
||
|
||
## Preamble Blocks
|
||
|
||
Immediately after the row table: **n_channels** length-prefixed UTF-8 strings,
|
||
one per channel in `SCAN_CHANNELS` order (CH1, CH3, CH4). Each block is:
|
||
|
||
```
|
||
uint16 length — byte length of the following UTF-8 string
|
||
bytes preamble — WFMOutpre response string from the oscilloscope
|
||
```
|
||
|
||
The preamble captures per-channel scaling constants (YMULT, YOFF, YZERO) needed
|
||
to convert raw ADC values to volts.
|
||
|
||
---
|
||
|
||
## Background Block
|
||
|
||
Immediately after the preamble blocks: a single CH1 waveform captured with the
|
||
**Helios (generation) laser enabled** and the **Genesis (detection) laser
|
||
disabled**. This provides a noise/background reference for subtraction during
|
||
post-processing.
|
||
|
||
```
|
||
uint32 n_bg_samples — number of samples in the background waveform
|
||
int8[] bg_data — raw ADC samples (same encoding as waveform data)
|
||
```
|
||
|
||
`n_bg_samples` equals `samples_per_frame` under normal acquisition settings.
|
||
|
||
---
|
||
|
||
## Waveform Data (ragged)
|
||
|
||
Immediately after the background block. Data is stored in **angle-major,
|
||
row-minor** order, but unlike earlier versions each angle contributes a
|
||
different number of rows (`n_rows[a]`) and a different number of frames per
|
||
row (`n_frames[a]`), both taken from that angle's Per-Angle Geometry Table
|
||
entry. Within each row, channels are interleaved in ascending channel-index
|
||
order, with each channel's FastFrame data written in frame order.
|
||
|
||
```
|
||
for angle a in 0 … n_angles-1:
|
||
for row in 0 … n_rows[a]-1:
|
||
for channel in [CH1, CH3, CH4]: # 3 channels, fixed order
|
||
for frame in 0 … n_frames[a]-1:
|
||
samples[0 … samples_per_frame-1] # bps bytes each
|
||
```
|
||
|
||
Each sample is a raw signed ADC value. With `bytes_per_sample = 1` this is
|
||
**int8** (−128 … +127). With `bytes_per_sample = 2` this is **big-endian
|
||
int16**.
|
||
|
||
Total data size:
|
||
```
|
||
sum over angles a of: n_rows[a] × 3 × n_frames[a] × samples_per_frame × bytes_per_sample
|
||
```
|
||
|
||
> **Incomplete files:** If a scan is aborted the file is closed immediately and
|
||
> the data block will be shorter than the expected size. Readers should
|
||
> reconstruct the expected per-angle byte offsets from the Per-Angle Geometry
|
||
> Table and check `file_size` against the running total before reshaping —
|
||
> a fixed `(n_angles, n_rows, ...)` reshape (as in pre-v6 readers) will not
|
||
> work since row/frame counts are no longer uniform across angles.
|
||
|
||
---
|
||
|
||
## Spatial Mapping
|
||
|
||
The *k*-th waveform (frame) in a row corresponds to the *k*-th laser pulse that
|
||
hit the sample. For a row belonging to angle *a*, the physical X position of
|
||
that pulse is:
|
||
|
||
```
|
||
x_k = x_start[a] + k * (velocity_mm_s / laser_freq_hz)
|
||
```
|
||
|
||
using that angle's `x_start` from the Per-Angle Geometry Table (not
|
||
`x_start_nominal`).
|
||
|
||
---
|
||
|
||
## Acquisition Settings (fixed by core/scope_sras.py)
|
||
|
||
| Parameter | Value |
|
||
|-----------------------|------------------------------------------|
|
||
| Setup trigger | CH2, rising edge, 0.500 V (`TRIG_LEVEL_V`) |
|
||
| Scan trigger | Logic AND, CH2 HIGH ∧ CH3 HIGH, 0.500 V |
|
||
| Horizontal position | 30 (`HORizontal:POSition`) |
|
||
| Sample rate | 6.25 GS/s (160 ps/sample) |
|
||
| Transfer format | `DATa:ENCdg RIBinary`, `DATa:WIDth 1` |
|
||
| Channels recorded | CH1, CH3, CH4 |
|
||
| Stage X velocity | 100 mm/s |
|
||
| Stage X acceleration | 1500 mm/s² |
|
||
| Stage X trigger out | Logic-high at max velocity (`TRIGOUT_MAXV`) |
|
||
| Acquisition mode | FastFrame, Normal trigger |
|
||
|
||
None of these are stored in the file, so they do not affect byte layout — but
|
||
they do set where the acoustic packet lands inside each frame. Read them from
|
||
`core/scope_sras.py`; earlier revisions of this table drifted from the code.
|
||
|
||
---
|
||
|
||
## Acquisition Paths
|
||
|
||
Two acquisition strategies write **byte-identical** files; the choice is a
|
||
runtime flag (`ScanEngine(burst_mode=…)`, exposed as a checkbox in the app) and
|
||
is not recorded in the file.
|
||
|
||
| | Per-row (default) | Burst |
|
||
|---|---|---|
|
||
| FastFrame acquisitions | one per row | one per `floor(max_frames / n_frames)` rows |
|
||
| Curve transfers | one per channel per row | one per channel per burst |
|
||
| Stage X trigger out | armed for the whole scan | armed per acquiring pass, dropped for the flyback |
|
||
|
||
Burst mode runs a single acquisition across several rows, so the return move
|
||
must not trigger: the trigger output is dropped before each flyback and
|
||
re-armed for each acquiring pass. Row boundaries inside the burst come from
|
||
`ACQuire:NUMFRAMESACQuired?` sampled after each pass — the burst itself carries
|
||
no row markers. See `core/scope_burst.py`.
|
||
|
||
### Row packing
|
||
|
||
The format has no per-row length field, so a row that over- or under-triggers
|
||
cannot be written as it arrived — that would shift every later row. Two
|
||
policies are selectable (`ScanEngine(strict_rows=…)`, a checkbox in the app),
|
||
and the choice is not recorded in the file:
|
||
|
||
| | Pad (default) | Strict |
|
||
|---|---|---|
|
||
| Short row | zero-padded to `n_frames`, warned | scan stops |
|
||
| Long row | trailing frames dropped, warned | scan stops |
|
||
|
||
Pad keeps a scan running through an occasional mis-trigger, at the cost that
|
||
the affected row is indistinguishable from a good one afterwards — nothing in
|
||
the file records that it was padded. Strict is for data runs where that
|
||
ambiguity is worse than a failed scan: it aborts before writing the row, so
|
||
the file always ends on a whole-row boundary.
|
||
|
||
---
|
||
|
||
## Version History
|
||
|
||
| Version | Change |
|
||
|---------|--------|
|
||
| 1 | One file per row; header included `angle_idx`, `row_idx`, `angle_deg`, `y_mm`. |
|
||
| 2 | One file per scan; global header with `n_angles`/`n_rows`; separate angle and row tables; three channels (CH1, CH3, CH4) per row. |
|
||
| 3 | Added preamble blocks (WFMOutpre strings) after the row table, one length-prefixed UTF-8 block per channel. |
|
||
| 4 | Added background waveform block (CH1, Helios ON / Genesis OFF) after the preamble blocks; stored as `uint32` sample count followed by raw `int8` ADC bytes. |
|
||
| 5 | (skipped) |
|
||
| 6 | Each angle now scans only the bounding box of the nominal ROI rotated by that angle instead of the AABB-expanded worst case across all angles. Header no longer carries a single global `x_start`/`x_delta`/`n_rows` — replaced with `*_nominal` reference fields plus a new Per-Angle Geometry Table (`x_start`, `x_delta`, `n_frames`, `n_rows` per angle) and a ragged Row Table / Waveform Data block sized per angle. **Not compatible with v4 readers** (e.g. `sras_viewer.py`, which has not yet been updated for v6). |
|
||
|