@qtsurfer/lastra

TypeScript reader for the Lastra columnar time series format.

Features

• Read .lastra files written by the Java LastraWriter
• Selective column access — only requested columns are decompressed
• Built-in decoders: ALP, Gorilla, Pongo (doubles), delta-varint (timestamps), ZSTD/gzip (binary)
• Apache Arrow interop — convert to Arrow Table for use with DuckDB-WASM, Perspective, Arquero, etc.
• Browser + Node.js compatible
• Zero-copy Float64Array output for decoded doubles

Install

npm install @qtsurfer/lastra

Usage

import { LastraReader } from '@qtsurfer/lastra';

// From fetch response
const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// Series data
const ts = reader.readSeriesLong('ts');           // Float64Array (timestamps)
const close = reader.readSeriesDouble('close');    // Float64Array (prices)

// Column metadata
const meta = reader.getSeriesColumn('ema1').metadata;
// { indicator: 'ema', periods: '10' }

// Events (signals with independent timestamps)
const signalTs = reader.readEventLong('ts');
const types = reader.readEventBinary('type');      // (Uint8Array | null)[]
const data = reader.readEventBinary('data');       // ZSTD-compressed JSON

// Decode event strings
const decoder = new TextDecoder();
types.forEach((t, i) => {
  if (t) console.log(decoder.decode(t), data[i] ? JSON.parse(decoder.decode(data[i]!)) : null);
});

API

LastraReader

Property / Method	Returns	Description
seriesRowCount	number	Number of series rows
eventsRowCount	number	Number of event rows
seriesColumns	ColumnInfo[]	Series column descriptors
eventColumns	ColumnInfo[]	Event column descriptors
readSeriesLong(name)	Float64Array	Decode a LONG series column
readSeriesDouble(name)	Float64Array	Decode a DOUBLE series column
readSeriesBinary(name)	(Uint8Array|null)[]	Decode a BINARY series column
readEventLong(name)	Float64Array	Decode a LONG event column
readEventDouble(name)	Float64Array	Decode a DOUBLE event column
readEventBinary(name)	(Uint8Array|null)[]	Decode a BINARY event column
readEventBinaryAsync(name)	Promise<...>	Async decode (native gzip in browser)
getSeriesColumn(name)	ColumnInfo	Get column descriptor with metadata
getEventColumn(name)	ColumnInfo	Get event column descriptor

ColumnInfo

interface ColumnInfo {
  name: string;
  dataType: DataType;
  codec: Codec;
  metadata: Record<string, string>;
}

Codec Support

Codec	Type	Description	Bundle cost
DELTA_VARINT	LONG	Delta-of-delta + zigzag varint (timestamps)	0 kB
ALP	DOUBLE	Adaptive lossless floating-point (~3-4 bits/value)	0 kB
GORILLA	DOUBLE	XOR compression (Facebook VLDB 2015)	0 kB
PONGO	DOUBLE	Decimal-aware erasure + Gorilla XOR (best for prices)	0 kB
VARLEN	BINARY	Variable-length encoding	0 kB
VARLEN_ZSTD	BINARY	Variable-length + ZSTD	~3.8 kB gz
VARLEN_GZIP	BINARY	Variable-length + gzip (async, native DecompressionStream)	0 kB
RAW	LONG/DOUBLE	Uncompressed / Float64Array zero-copy	0 kB

Arrow Interop

Convert Lastra data to an Apache Arrow Table for integration with any Arrow-compatible tool.

import { LastraReader, lastraToArrow, lastraToArrowColumns } from '@qtsurfer/lastra';

const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// All series columns
const table = lastraToArrow(reader);

// Selected columns only (others are not decompressed)
const partial = lastraToArrowColumns(reader, ['t', 'cls', 'vol']);

// One-liner: buffer → Arrow Table
import { readLastraAsArrow } from '@qtsurfer/lastra';
const table2 = readLastraAsArrow(buffer);

The resulting Arrow Table can be registered in DuckDB-WASM, used with Perspective, Arquero, Observable Plot, or any tool that accepts Arrow IPC.

Cross-Language Testing

Test fixtures are generated by the Java LastraWriter and read by the TS reader, ensuring wire format compatibility. Includes real BTC/USDT ticker data with mixed codecs (ALP + Gorilla + Pongo).

License

Copyright 2026 Wualabs LTD. Apache License 2.0 — see LICENSE.