vortex-go - source code release

I'm happy to share:

A native Go implementation of the Vortex columnar file format.

Vortex is a next-generation columnar file format (Linux Foundation project) designed for analytical workloads. It achieves significantly faster scans and better compression than Apache Parquet through adaptive encoding selection and lazy materialisation.

vortex-go provides full read and write support from pure Go — no CGo, no Rust FFI.

Key Features

  • 25 encoding decoders — ALP, BitPacking, Dict, FoR, FSST, Pcodec, RLE, Zstd, and more
  • Compressed writer — automatic encoding selection (dict, FoR+BitPacking, Pcodec, zone maps)
  • Column projection — read only the columns you need, skip the rest
  • Zone map pruning — skip entire chunks based on min/max filter predicates
  • Full cross-compatibility — files readable by the Rust reference, Python vortex, and DuckDB
  • CLI toolsvtxinfo for file inspection, vtxconvert for Parquet-to-Vortex conversion
  • Zero external dependencies at runtime — only Go standard library plus FlatBuffers/Protobuf

Performance

On a 7-column schema at 1M rows (Intel 14th Gen, Go 1.24):

MetricParquetVortex (Go)Improvement
File size45.6 MB19.3 MB58% smaller
Full scan188 ms40 ms5× faster
Projected scan (2/7 cols)188 ms16 ms12× faster
Throughput5.3M rows/s25M rows/s5× faster

Zone map pruning on 100K rows (10 chunks, filter selects 10%): 8.5× faster than full scan.

The source code is available at:  https://github.com/LaurieRhodes/vortex-go