# parquet

> parquet, a magmalake tin: The first native Apache Parquet reader and writer in Mojo.

[magmalake](https://magmalake.dev/) [parquet](https://magmalake.dev/s/parquet)

The first native Apache Parquet reader and writer in Mojo. It decodes the footer, the page headers, the levels and the values itself, and hands the result back in Arrow memory layout, exportable over the Arrow C Data Interface. That interface is the boundary, not a full Arrow library: there are no compute kernels, no IPC and no Flight.

Install

mojoshelf:

```
pixi shelf add parquet-mojo
```

`pixi shelf` comes with mojoshelf: [install it first](https://mojoshelf.org/getting-started).

plain pixi:

```
pixi init app -c https://conda.modular.com/max -c conda-forge && cd app
sed -i.bak 's/^\[workspace\]/[workspace]\npreview = ["pixi-build"]/' pixi.toml
pixi add "mojo-compiler==1.2.0"
# Without mojoshelf: the tin as a git dependency at its published commit.
# Transitive dependencies may be wrong: pixi takes the tin's own git deps as
# pinned in its pixi.toml, which may not match what other tins pin, and then
# fails with "duplicate records". mojoshelf picks revisions that agree.
pixi add parquet-mojo --git https://github.com/magmalake/parquet.mojo --rev c15f51aee06ccf73102810e7de143745abba6c90
```

No mojoshelf needed. If pixi reports duplicate records, the mojoshelf tab picks revisions that agree.

Version

`0.11.7` on mojoshelf

Source

[GitHub](https://github.com/magmalake/parquet.mojo) [README](https://github.com/magmalake/parquet.mojo#readme) [Benchmark history](https://magmalake.github.io/parquet.mojo/benchmarks/)

## Works today

- Every physical and logical type, every encoding, v1 and v2 pages
- UNCOMPRESSED, SNAPPY and GZIP with no FFI; ZSTD, BROTLI and LZ4 with parquet-full
- Nested lists, maps and structs reconstructed from definition and repetition levels
- Row-group, page-index and bloom-filter pruning; field-id projection for Iceberg
- Arrow C Data Interface export, verified by pyarrow importing the arrays

## Not yet

- More than 2 GiB of string data in one Parquet column chunk

Checked against pyarrow, value-exact on 33 fixtures; pyarrow reads back every file it writes

Value-exact against pyarrow on 33 fixtures — every value of every column — and pyarrow reads back every file magmalake writes.

The Arrow export imports through pyarrow.Array.\_import\_from\_c.

### Building it found bugs upstream

Holding three implementations against each other surfaces disagreements, and not all of them turned out to be my fault. I cut each one below down to a standalone script or unit test with no magmalake code in it, then either reported it upstream or — where someone had got there first — matched it to the report that already existed.

- [Apache Arrow apache/arrow#51097  reported](https://github.com/apache/arrow/issues/51097) The Parquet writer undercounts nulls for a fixed-width leaf under list<struct> when some lists are null or empty. The BYTE\_ARRAY leaf beside it in the same struct is right, so two leaves disagree about the same level records.

- [parquet-full](https://magmalake.dev/s/parquet-full) parquet · plus ZSTD, BROTLI and LZ4
- [avro](https://magmalake.dev/s/avro) object container files, both ways
- [arrow-mlake](https://magmalake.dev/s/arrow-mlake) Arrow layout · C Data Interface · IPC
- [roaring](https://magmalake.dev/s/roaring) 32/64-bit · deletion vectors
- [Every tin](https://magmalake.dev/tins)

[A magmalake project](https://magmalake.dev/)

Apache 2.0 · © 2026 Marius Seritan