Tins · Storage & databases
objectstore
Storage and HTTP for Iceberg tables: a FileIO abstraction over local files, HTTP(S) range reads and S3, with the pooled HTTP transport the rest of the stack was missing.
Install
plain pixi:
pixi init app -c https://conda.modular.com/max -c conda-forge && cd app
sed -i.bak 's/^\[workspace\]/[workspace]\npreview = ["pixi-build"]/' pixi.toml
pixi add "mojo-compiler==1.2.0"
# Without mojoshelf: the tin as a git dependency at its published commit.
# Transitive dependencies may be wrong: pixi takes the tin's own git deps as
# pinned in its pixi.toml, which may not match what other tins pin, and then
# fails with "duplicate records". mojoshelf picks revisions that agree.
pixi add objectstore-mojo --git https://github.com/magmalake/objectstore.mojo --rev 41fac2800882e5a64da09729262dfa22bbd4ad13No mojoshelf needed. If pixi reports duplicate records, the mojoshelf tab picks revisions that agree.
Version
0.4.1 on mojoshelf
What it does
Works today
- S3 with SigV4, vended credentials, presigned URLs and multipart upload
- Pooled libcurl transport, so range reads reuse a connection
- Pure-Mojo SHA-256 and HMAC on hardware crypto paths
- GCS, Azure and plain HTTP range reads alongside local files
Correctness
Checked against AWS SigV4 suite 37/37, and S3 verified end-to-end against MinIO in CI
37 of 37 cases of the official AWS SigV4 test suite, every stage.
S3 is verified end-to-end against MinIO in CI, not only on a developer's machine.