sras-viewer/scan_format.md

SRAS Scan Binary Format — Version 6 / 7

Each .sras file contains one complete scan: all GR rotation angles and all Y rows. Files are named {prefix}.sras.

Starting in v6, each angle only scans the bounding box of the nominal ROI rotated by that specific angle — not the worst case across all angles — so x_start, x_delta (and therefore n_frames, the points/row count) and n_rows all vary per angle. A 0°/180° scan of a wide, short ROI needs far fewer rows than a 45° scan of the same ROI, and the file format reflects that instead of forcing every angle to the largest bounding box.

v7 is byte-identical to v6 (same header, angle table, geometry table, row table, preamble blocks, background block, waveform data) plus an optional trailing Cache Tail holding precomputed per-angle DC and/or FFT images (see Cache Tail (v7) below) — only the header's version field and the presence of that trailing section differ. Scans come off the scope as v6; sras_viewer.py's "Convert" menu batch actions convert a file to v7 in place the first time either cache block is computed and stored.

---

File Layout

[Global Header           — 49 bytes]
[Angle Table             — n_angles × 4 bytes  (float32 per angle, degrees)]
[Per-Angle Geometry Table— n_angles × 14 bytes (x_start f32, x_delta f32, n_frames u32, n_rows u16)]
[Row Table (ragged)      — sum(n_rows) × 4 bytes (float32 per row, angle-major)]
[Preamble Blocks         — n_channels × (uint16 length + UTF-8 WFMOutpre string)]
[Background Block        — uint32 n_bg_samples + n_bg_samples × int8 bytes]
[Waveform Data (ragged)  — per angle: n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bps bytes]
[Cache Tail (optional)   — "CACH" + DC block (optional) + FFT block (optional); v7 only]

All multi-byte integers and floats use big-endian byte order (> in Python's struct module).

---

Global Header (49 bytes)

Offset	Size	Type	Field	Description
0	4	4s	magic	Always SRAS (0x53 0x52 0x41 0x53)
4	1	uint8	version	Format version — 6
5	2	uint16	n_angles	Number of GR rotation angles
7	4	float32	x_start_nominal	Nominal (pre-rotation) X scan start, mm
11	4	float32	y_start_nominal	Nominal (pre-rotation) Y scan start, mm
15	4	float32	x_delta_nominal	Nominal (pre-rotation) X scan width, mm
19	4	float32	y_delta_nominal	Nominal (pre-rotation) Y scan height, mm
23	4	float32	row_spacing_mm	Y spacing between rows, mm
27	4	float32	velocity_mm_s	Stage scan velocity in mm/s
31	4	float32	laser_freq_hz	Laser repetition rate in Hz
35	4	uint32	samples_per_frame	Time samples per waveform
39	8	float64	sample_rate_hz	Oscilloscope sample rate in Hz (e.g. 6.25e9)
47	1	uint8	bytes_per_sample	Bytes per ADC sample: 1 = int8, 2 = int16
48	1	uint8	n_channels	Number of channels recorded (currently 3)

Total header size: 49 bytes — verified: struct.calcsize(">4sBHfffffffIdBB") == 49.

The *_nominal fields describe the ROI as originally entered on the New Scan page (XS/YS/XD/YD), before per-angle bounding-box expansion. They are for reference/reconstruction only — the actual per-angle scan geometry used for acquisition is in the Per-Angle Geometry Table below.

---

Angle Table

Immediately after the header: n_angles big-endian float32 values, one per GR angle (degrees, 0–180).

angle[0], angle[1], …, angle[n_angles - 1]

---

Per-Angle Geometry Table

Immediately after the angle table: n_angles fixed-size records, one per angle (same order as the angle table), each 14 bytes:

Size	Type	Field	Description
4	float32	x_start	X scan start for this angle's bounding box, mm
4	float32	x_delta	X scan width for this angle's bounding box, mm
4	uint32	n_frames	A-scans per row for this angle (FastFrame count)
2	uint16	n_rows	Number of Y rows scanned for this angle

Format string per record: ">ffIH".

---

Row Table (ragged)

Immediately after the per-angle geometry table: for each angle in order, that angle's n_rows big-endian float32 Y positions (mm), concatenated with no padding between angles.

# angle 0's rows, then angle 1's rows, …
y_mm[0][0], …, y_mm[0][n_rows[0]-1], y_mm[1][0], …, y_mm[n_angles-1][n_rows[-1]-1]

Row-table boundaries for angle a are derived from the per-angle geometry table: sum(n_rows[0:a]) gives the starting index into the flattened array.

---

Preamble Blocks

Immediately after the row table: n_channels length-prefixed UTF-8 strings, one per channel in SCAN_CHANNELS order (CH1, CH3, CH4). Each block is:

uint16  length   — byte length of the following UTF-8 string
bytes   preamble — WFMOutpre response string from the oscilloscope

The preamble captures per-channel scaling constants (YMULT, YOFF, YZERO) needed to convert raw ADC values to volts.

---

Background Block

Immediately after the preamble blocks: a single CH1 waveform captured with the Helios (generation) laser enabled and the Genesis (detection) laser disabled. This provides a noise/background reference for subtraction during post-processing.

uint32  n_bg_samples   — number of samples in the background waveform
int8[]  bg_data        — raw ADC samples (same encoding as waveform data)

n_bg_samples equals samples_per_frame under normal acquisition settings.

---

Waveform Data (ragged)

Immediately after the background block. Data is stored in angle-major, row-minor order, but unlike earlier versions each angle contributes a different number of rows (n_rows[a]) and a different number of frames per row (n_frames[a]), both taken from that angle's Per-Angle Geometry Table entry. Within each row, channels are interleaved in ascending channel-index order, with each channel's FastFrame data written in frame order.

for angle a in 0 … n_angles-1:
  for row in 0 … n_rows[a]-1:
    for channel in [CH1, CH3, CH4]:          # 3 channels, fixed order
      for frame in 0 … n_frames[a]-1:
        samples[0 … samples_per_frame-1]     # bps bytes each

Each sample is a raw signed ADC value. With bytes_per_sample = 1 this is int8 (−128 … +127). With bytes_per_sample = 2 this is big-endian int16.

Total data size:

sum over angles a of: n_rows[a] × 3 × n_frames[a] × samples_per_frame × bytes_per_sample

Incomplete files: If a scan is aborted the file is closed immediately and the data block will be shorter than the expected size. Readers should reconstruct the expected per-angle byte offsets from the Per-Angle Geometry Table and check file_size against the running total before reshaping — a fixed (n_angles, n_rows, ...) reshape (as in pre-v6 readers) will not work since row/frame counts are no longer uniform across angles.

---

Spatial Mapping

The k-th waveform (frame) in a row corresponds to the k-th laser pulse that hit the sample. For a row belonging to angle a, the physical X position of that pulse is:

x_k = x_start[a] + k * (velocity_mm_s / laser_freq_hz)

using that angle's x_start from the Per-Angle Geometry Table (not x_start_nominal).

---

Cache Tail (v7)

Present iff version == 7 and file_size > cache_offset, where:

cache_offset = data_offset + Σ over angles a of:
                 n_rows[a] × n_channels × n_frames[a] × samples_per_frame × bytes_per_sample

i.e. exactly data_offset + waveform_bytes — the same "Total data size" formula as Waveform Data above. This offset is derivable from the header and Per-Angle Geometry Table alone and does not depend on which cache block(s) are present, so a writer can always seek straight there without reading or touching any waveform byte before it.

Unlike the (removed) v5 PREC section, which stored one omnibus per-angle entry (FFT + both DC channels together) in a dense, uniform-geometry array, the v7 Cache Tail splits DC and FFT into two independent sub-blocks — each sized per-angle from the Per-Angle Geometry Table, each independently present, and each independently updatable in any order, any number of times, without disturbing the other. This matches sras_viewer.py's "Convert" menu, which exposes DC and FFT store as two separate batch actions.

CACH outer header (6 bytes, ">4sBB")

Offset	Size	Type	Field	Description
0	4	char[4]	cach_magic	CACH (ASCII). Missing/wrong magic → treat file as having no cache.
4	1	u8	cach_version	Cache format version. Currently 2; readers also accept 1 (a 1 tail predates row-averaged FFT caching — see the SFFT block below and CACH tail version history). Any other value → treat the file as uncached (unlike v5's PREC section, which read but never validated its version byte).
5	1	u8	block_flags	Bit 0 = DC block (SDCB) follows. Bit 1 = FFT block (SFFT) follows, immediately after the DC block if both are present. Bits 2–7 reserved, must be zero on write.

DC block SDCB (present iff block_flags & 0x01)

7-byte block header, format ">4sBH":

Offset (rel)	Size	Type	Field	Description
0	4	char[4]	magic	SDCB
4	1	u8	reserved	0, reserved for future use
5	2	u16	n_stored	Number of angle entries that follow, 0 ≤ n_stored ≤ n_angles

followed by n_stored entries, each:

u16                          angle_idx   — index into the angle table (0-based)
f32[n_rows[angle_idx] × n_frames[angle_idx]]  dc3_mv  — CH3 waveform mean, mV, row-major
f32[n_rows[angle_idx] × n_frames[angle_idx]]  dc4_mv  — CH4 waveform mean, mV, row-major

Entries may appear in any order and need not be contiguous from angle 0 — this supports storing (or re-storing) a subset of angles, or an interrupted batch run leaving only some angles cached. Readers bounds-check angle_idx < n_angles on each entry and stop parsing on an out-of-range value, same as v5's PREC section.

FFT block SFFT (present iff block_flags & 0x02)

Block header layout depends on cach_version:

• cach_version 1: 7 bytes, format ">4sBH" — magic, flags, n_stored.
• cach_version 2: 8 bytes, format ">4sBHB" — magic, flags, n_stored, row_avg_n. Always written by current code; a cach_version 1 tail (no trailing byte) is still read, with row_avg_n taken as 0 for every entry it stores.

Offset (rel)	Size	Type	Field	Description
0	4	char[4]	magic	SFFT
4	1	u8	flags	Bit 0 = bg_sub_applied — background waveform was subtracted from CH1 before the FFT when these images were computed. Bit 1 = row_averaged — peak_freq_mhz came from same-row, distance-weighted averaged CH1 waveforms rather than raw per-pixel ones; row_avg_n (below) is the neighbor half-width used. Bits 2–7 reserved.
5	2	u16	n_stored	Number of angle entries that follow
7	1	u8	row_avg_n	cach_version 2 only. Same-row neighbor half-width, in pixels, that peak_freq_mhz was averaged over before its FFT; 0 = raw (unaveraged). Meaningful only when flags bit 1 is set — a cach_version 1 tail has no such byte and is always row_avg_n = 0.

followed by n_stored entries, each:

u16                          angle_idx      — index into the angle table (0-based)
f32[n_rows[angle_idx] × n_frames[angle_idx]]  peak_freq_mhz  — CH1 FFT peak frequency, MHz, row-major

peak_freq_mhz for a raw store (row_avg_n == 0) is computed without any DC-threshold masking (i.e. the FFT is run on every pixel unconditionally, same as v5's PREC convention). Readers apply the DC4 threshold at display time:

pixel is valid  ⟺  dc4_mv[r][f]  ≥  threshold_mv
display_value   =   peak_freq_mhz[r][f]  if valid, else 0

using the DC4 image from the DC block if that angle is also cached there, else computed on demand.

For a row-averaged store (row_avg_n > 0), the DC4 threshold is applied during the store — a pixel below threshold is left at 0 and never contributes to any neighbor's average — since neighbor validity can't be deferred to display time the way plain masking can. The threshold value itself is not recorded, only that averaging happened and at what window size. Readers still apply their own live DC4 threshold at display time exactly as for a raw store, using whatever mask they currently have.

Readers must fall back to real-time FFT computation (ignoring stored peak_freq_mhz) under the same conditions as v5's PREC fast path: time-domain gating is active, zero-padding (n_fft ≠ samples_per_frame) is requested, the reader's background-subtraction setting doesn't match flags.bg_sub_applied, or the reader's requested row_avg_n doesn't match the stored value exactly — a raw request must never be served a row-averaged store, or vice versa, and a request at one window size must never be served a store at another.

In-place write ordering

A writer updating a file's Cache Tail must write the payload (CACH header + whichever block(s) are present) before flipping the header's version byte to 7, and truncate() the file to the new payload's end immediately after writing it. If the process is interrupted between the payload write and the version-byte flip, the file is still valid v6 — v6 parsing only bounds-checks each angle's offset + nbytes ≤ file_size, it never asserts exactly how many bytes follow the last angle's waveform block — so the interrupted write leaves harmless trailing bytes rather than a corrupt file, and the next successful write overwrites them via the same deterministic cache_offset.

CACH tail version history

Distinct from the outer .sras file version byte (top of this document), which has stayed 7 since the Cache Tail was introduced — this is the inner cach_version byte inside the CACH header itself.

cach_version	Change
1	Initial Cache Tail: SDCB (DC) and SFFT (FFT, 7-byte header) blocks.
2	SFFT header grows one byte, row_avg_n — the same-row neighbor half-width the stored peak_freq_mhz was averaged over before its FFT, 0 = raw. Readers still accept a cach_version 1 tail, treated as row_avg_n = 0 for every angle it stores, so files cached before this change keep working without a recompute.

---

Acquisition Settings (fixed by sc3_aui_app.py)

Parameter	Value
Oscilloscope trigger	CH2, rising edge, 1.24 V
Trigger offset	0 % (trigger at left edge)
Sample rate	6.25 GS/s (160 ps/sample)
Channels recorded	CH1, CH3, CH4
Stage X velocity	100 mm/s
Stage X acceleration	1500 mm/s²
Stage X trigger out	Logic-high at max velocity
Acquisition mode	FastFrame, Normal trigger

---

Version History

Version	Change
1	One file per row; header included angle_idx, row_idx, angle_deg, y_mm.
2	One file per scan; global header with n_angles/n_rows; separate angle and row tables; three channels (CH1, CH3, CH4) per row.
3	Added preamble blocks (WFMOutpre strings) after the row table, one length-prefixed UTF-8 block per channel.
4	Added background waveform block (CH1, Helios ON / Genesis OFF) after the preamble blocks; stored as uint32 sample count followed by raw int8 ADC bytes.
5	(skipped)
6	Each angle now scans only the bounding box of the nominal ROI rotated by that angle instead of the AABB-expanded worst case across all angles. Header no longer carries a single global x_start/x_delta/n_rows — replaced with *_nominal reference fields plus a new Per-Angle Geometry Table (x_start, x_delta, n_frames, n_rows per angle) and a ragged Row Table / Waveform Data block sized per angle. Not compatible with pre-v6 readers that assume uniform geometry. sras_viewer.py reads v6 natively (per-angle n_rows/n_frames/x_start).
7	Adds an optional trailing Cache Tail (CACH section, see Cache Tail (v7)) after the ragged waveform data, holding independently-present, independently-updatable per-angle DC (SDCB: dc3_mv + dc4_mv) and FFT (SFFT: peak_freq_mhz) blocks, so previously-computed images redisplay instantly instead of being recomputed. Header / angle table / geometry table / row table / preamble blocks / background block / waveform data are byte-identical to v6 — only the version byte and the optional Cache Tail differ. Scans still come off the scope as v6; sras_viewer.py's "Convert" menu ("Batch Compute DC and Store" / "Batch Compute FFT and Store") converts a file to v7 in place on first use, or updates an existing v7 file's cache blocks, without rewriting any waveform bytes. Supersedes the removed "Pre-process and Save as v5" workflow, which was never available for v6 sources since the flat v5 PREC layout can't represent per-angle geometry.