Infino
★ star on github Get started for free

Architecture

Infino writes Parquet to object storage to power search. Keep a single copy of your data so every tool in your stack reads the same source of truth.

writes Parquet parquet on object storage s3 · gcs · azure blob INFINO READER hybrid search BM25 + vector analytical SQL joins & aggregates query DSL elastic compatibility local cache RAM NVMe LAKE FORMATS iceberg · delta lake · hudi ANALYTICS snowflake · databricks · duckdb · trino · clickhouse i i i i i i
Infino writes BM25 and vector indexes into the Parquet files. file layout →
Hybrid search reads BM25 and vector index regions inside each Parquet file. example query →
SQL runs on DataFusion with BM25, vector, and hybrid search as table-valued functions. SQL example →
A supported Query DSL subset lets compatible Elasticsearch and OpenSearch clients query Parquet. migration path →
Object storage holds the durable data; RAM and NVMe cache hot byte ranges. cache path →
Files remain standard Parquet columns readable by Snowflake, Databricks, DuckDB, Trino, and ClickHouse. warehouse fit →

hover a section to learn more

Get started for free
infino explain files

Data and indexes share one Parquet file

Each chunk of a table is one self-contained file, a superfile.

  • Columns and both search indexes, keyword and vector, live in the same file.
  • It's a valid Apache Parquet file, so DuckDB, pandas, and Spark open the same bytes.

Superfile format, on GitHub →

ONE FILE, FOUR REGIONS a valid Parquet file scalar columns Parquet row groups full-text index vector index footer DuckDB · pandas · Spark columns + footer only infino reader reads it all: BM25 · vector · SQL
infino explain manifest

Files are never edited, only added

A table is its manifest: the many superfiles it currently points to. Infino calls that whole, manifest-tracked table a supertable.

  • If you have used Iceberg or Delta Lake, this is the same idea: immutable data files plus a small pointer that commits atomically. Here the files also carry the search indexes.
  • Switching manifests is one atomic step, so a reader sees the table before a write or after it.

Supertable layer, on GitHub →

manifest v1 · [ A, B ] NEW FILE A B C manifest v2 · current · [ A, B, C ] commit (atomic)
infino explain cache

Storage and compute are separate

The engine stores nothing permanently.

  • It fetches the byte ranges a query needs and keeps frequently used data on local disk, the way a warehouse caches hot partitions.
cold only in object storage warm resident on local disk hot pages in RAM object storage
infino deploy

Run it managed, or run it yourself

The same build runs on any object store, chosen once at startup.

  • Amazon S3, Azure Blob Storage, Google Cloud Storage, or a local disk.
  • Run it serverless on Infino Cloud: multi-tenant, metered on usage, with a free tier, support and SLAs.
  • Or run the same engine yourself, on-premise or in your cloud.

Infino Cloud →

One copy of your data, searched in place.

Get started for free Talk to an engineer

Free tier · No credit card to start

or run it yourself: pip install infino · Apache-2.0 · quickstart →

Go deeper: accelerating full-text search queries · opening a file on S3 used to take two round trips · docs ↗