Vector Embeddings

This guide covers practical decisions for working with vector embeddings in ArcadeDB: choosing dimensions, creating indexes, tuning parameters, and combining vector search with other query types.

Choosing an Embedding Model

Your embedding model determines the dimensions parameter for the index:

Model Dimensions Notes

Model	Dimensions	Notes
OpenAI `text-embedding-3-small`	1536	General purpose, high quality
OpenAI `text-embedding-3-large`	3072	Highest quality, largest memory footprint
Sentence Transformers `all-MiniLM-L6-v2`	384	Fast, open source, good quality
Sentence Transformers `all-mpnet-base-v2`	768	Better quality, slower
Cohere `embed-english-v3.0`	1024	Good balance of quality and size
CLIP (image + text)	512	Multi-modal image/text

OpenAI text-embedding-3-small

1536

General purpose, high quality

OpenAI text-embedding-3-large

3072

Highest quality, largest memory footprint

Sentence Transformers all-MiniLM-L6-v2

384

Fast, open source, good quality

Sentence Transformers all-mpnet-base-v2

768

Better quality, slower

Cohere embed-english-v3.0

1024

Good balance of quality and size

CLIP (image + text)

512

Multi-modal image/text

Start with 384 dimensions (MiniLM) for prototyping. Move to 768+ for production quality. Use quantization to manage memory at higher dimensions.

Creating a Vector Index

Recommended index creation with INT8 quantization:

CREATE VERTEX TYPE Document
CREATE PROPERTY Document.content STRING
CREATE PROPERTY Document.embedding LIST OF FLOAT

CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8'
}

INT8 quantization is recommended for all production workloads. It provides 2.5x faster search and 4x lower memory usage with negligible accuracy loss (see concepts/vector-search.adoc#quantization-performance). Only omit quantization for very small datasets (< 10K vectors) where maximum precision matters.

Production-ready index with additional tuning:

CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8',
  maxConnections: 16,
  beamWidth: 100
}

Choosing a Similarity Function

Function	Choose When	Avoid When
COSINE	Using text embedding models (most common). Vectors may have varying magnitudes.	Vectors represent absolute quantities (distances, counts).
DOT_PRODUCT	Vectors are already L2-normalized. You need maximum query speed.	Vectors are not normalized (results will be incorrect).
EUCLIDEAN	Working with spatial data, sensor readings, or continuous measurements.	Comparing text embeddings of different lengths.

Function

Choose When

Avoid When

COSINE

Using text embedding models (most common). Vectors may have varying magnitudes.

Vectors represent absolute quantities (distances, counts).

DOT_PRODUCT

Vectors are already L2-normalized. You need maximum query speed.

Vectors are not normalized (results will be incorrect).

EUCLIDEAN

Working with spatial data, sensor readings, or continuous measurements.

Comparing text embeddings of different lengths.

Quantization Trade-offs

Use INT8 quantization for most use cases. It provides 4x memory savings with minimal accuracy loss and significantly faster ingestion and search:

< 10K vectors: NONE is fine, but INT8 works well too
10K - 1M vectors: Use INT8 (4x memory savings, < 2% accuracy loss) — recommended
> 1M vectors: Use INT8 for general use, or PRODUCT for zero-disk-I/O graph construction on very large datasets
Extreme compression: Use BINARY for first-pass filtering, then rerank with full vectors

-- INT8: recommended for most workloads
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 768,
  similarity: 'COSINE',
  quantization: 'INT8'
}

-- PRODUCT: for very large datasets, enables in-memory graph build
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 1024,
  similarity: 'COSINE',
  quantization: 'PRODUCT'
}

INT8 Pre-Quantized Ingest

Available since ArcadeDB v26.5.1. See Vector Encoding for the underlying concept.

When your embedding provider already emits signed-int8 vectors (Cohere int8 endpoints, OpenAI text-embedding-3-large reduced precision, Sentence Transformers with int8 quantization), use encoding: 'INT8' to keep the bytes byte-shaped end-to-end:

CREATE PROPERTY Doc.embedding BINARY;
CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  "dimensions": 1024,
  "similarity": "COSINE",
  "encoding": "INT8"
};

What this saves:

HTTP payload: 4x smaller (1 byte/dim vs 4 bytes/dim) when sending vectors with the typed-marker convention — see HTTP wire convention.
Document bucket storage: 4x smaller, since the property column is BINARY instead of ARRAY_OF_FLOATS.
No client-side int8 → float32 → server round trip; the precision the provider already discarded does not get padded back out for the wire.

What this does not change today:

HNSW graph and search internally still run on float32 (JVector 4.0.0-rc.8 contract). The engine dequantizes once on the read path. Native int8 HNSW is tracked upstream at datastax/jvector#665.
encoding is independent of quantization. Combining encoding: 'INT8' with quantization: 'INT8' is rejected at index creation — pick one, not both.

When not to use INT8 encoding:

Your provider emits float32 (or float64) and you do not have an int8 quantizer client-side. The default FLOAT32 encoding skips the dequantize hop on every search and keeps full precision in the documents.
You want the index-internal compression benefit (search-time memory footprint). That is quantization: 'INT8', not encoding: 'INT8'. They are orthogonal.

Tuning for Recall vs Speed

Adjust maxConnections and beamWidth based on your priorities:

Profile	maxConnections	beamWidth	Trade-off
Default	32	100	Balanced for most workloads; matches hnswlib `M=16` density
High recall	48	200	Better accuracy, slower builds, more memory
Fast indexing	24	80	Faster builds, slightly lower recall
Memory constrained	16	60	Minimal memory footprint

maxConnections is the Vamana per-layer graph degree; unlike hnswlib’s M it is not doubled at the base layer, so to match an hnswlib M set maxConnections = 2 * M.

For datasets over 100K vectors or with 1024+ dimensions, enable hierarchical mode:

CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 1536,
  similarity: 'COSINE',
  quantization: 'INT8',
  addHierarchy: true,
  maxConnections: 32,
  beamWidth: 200
}

Tuning efSearch

The efSearch parameter controls how many candidates the search explores at query time. By default, ArcadeDB uses an adaptive strategy that works well for most workloads. You only need to tune efSearch if you have specific recall or latency requirements.

Profile efSearch Trade-off

Profile	efSearch	Trade-off
Adaptive (default)	auto	Two-pass: fast first pass (`2×k`), wider retry (`10×k`) if needed
High recall	200-500	Consistent high accuracy, higher latency
Low latency	20-50	Fast responses, lower recall on hard queries

Adaptive (default)

auto

Two-pass: fast first pass (2×k), wider retry (10×k) if needed

High recall

200-500

Consistent high accuracy, higher latency

Low latency

20-50

Fast responses, lower recall on hard queries

You can override efSearch per-query without changing the index:

-- High recall for a critical search
SELECT expand(vector.neighbors('Doc[embedding]', $queryVector, 10, 500))

-- Low latency for autocomplete/typeahead
SELECT expand(vector.neighbors('Doc[embedding]', $queryVector, 5, 30))

Or set a default on the index:

CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
  dimensions: 768,
  similarity: 'COSINE',
  quantization: 'INT8',
  efSearch: 200
}

Store multiple embeddings per record for different search modalities:

CREATE VERTEX TYPE Product
CREATE PROPERTY Product.imageEmbedding ARRAY_OF_FLOATS
CREATE PROPERTY Product.textEmbedding  ARRAY_OF_FLOATS

CREATE INDEX ON Product (imageEmbedding) LSM_VECTOR METADATA {dimensions: 512, similarity: 'COSINE'}
CREATE INDEX ON Product (textEmbedding)  LSM_VECTOR METADATA {dimensions: 768, similarity: 'COSINE'}

Query each index independently:

-- Search by image similarity
SELECT name, distance FROM (
  SELECT expand(vector.neighbors('Product[imageEmbedding]', $imageVector, 10))
)

-- Search by text similarity
SELECT name, distance FROM (
  SELECT expand(vector.neighbors('Product[textEmbedding]', $textVector, 10))
)

Hybrid Search: Dense + Sparse + Full-Text

The vector.fuse operator described below is available since ArcadeDB v26.5.1. Earlier versions required two queries plus client-side fusion via vector.rrfScore.

Combine dense vector similarity with sparse retrieval and/or keyword matching in a single server-side query. vector.fuse accepts any number of ranked sub-pipelines plus a fusion strategy (RRF, DBSF, LINEAR) and returns one ranked top-K:

-- Schema: dense + sparse properties + indexes (sparse needs LSM_SPARSE_VECTOR).
CREATE PROPERTY Document.dense   ARRAY_OF_FLOATS;
CREATE PROPERTY Document.tokens  ARRAY_OF_INTEGERS;
CREATE PROPERTY Document.weights ARRAY_OF_FLOATS;

CREATE INDEX ON Document (dense) LSM_VECTOR
  METADATA { dimensions: 384, similarity: 'COSINE' }

CREATE INDEX ON Document (tokens, weights) LSM_SPARSE_VECTOR
  METADATA { dimensions: 30000, modifier: 'IDF' }

-- Hybrid retrieval in one statement.
SELECT expand(`vector.fuse`(
    `vector.neighbors`('Document[dense]', :denseVec, 50),
    `vector.sparseNeighbors`('Document[tokens,weights]', :qIdx, :qVal, 50),
    { fusion: 'RRF', groupBy: 'source_file', groupSize: 1 }
)) LIMIT 10

vector.neighbors exposes distance (lower = better); vector.fuse auto-flips it so dense and sparse sources compose without manual rescaling.
groupBy + groupSize collapse same-source duplicates server-side. Drop the option to return chunk-level results.
Pre-fusion grouping is also possible by attaching { groupBy: 'source_file', groupSize: 1 } to each individual source.

To include full-text alongside dense/sparse, add a third source built from SEARCH_INDEX:

SELECT expand(`vector.fuse`(
    `vector.neighbors`('Document[dense]', :denseVec, 100),
    `vector.sparseNeighbors`('Document[tokens,weights]', :qIdx, :qVal, 100),
    (SELECT @rid, $score FROM Document
     WHERE SEARCH_INDEX('Document[content]', 'machine learning') = true),
    { fusion: 'RRF' }
)) LIMIT 10

Pick the strategy that matches your scoring shape:

RRF — rank-only, indifferent to score scales. Default, safest with mixed source types.
DBSF — mean +/- 3sigma normalisation per source then weighted sum. Use when scores are roughly Gaussian on each side.
LINEAR — per-source min-max normalisation then weighted sum. Use with already-tuned offline weights.

For the legacy two-query workaround (still supported via vector.rrfScore and vector.hybridScore on already-computed scores), see the SQL Vector Functions reference.

Batch Ingestion

For bulk loading vectors, batch your inserts within transactions:

BEGIN

CREATE VERTEX Document SET content = 'First document',  embedding = [0.1, 0.2, ...]
CREATE VERTEX Document SET content = 'Second document', embedding = [0.3, 0.4, ...]
-- ... more inserts ...

COMMIT

For large bulk loads, increase mutationsBeforeRebuild to delay index rebuilds until after the load completes, then trigger a rebuild.

When vectors are inserted below the rebuild threshold, an inactivity timer ensures the graph is still rebuilt after a period of no new mutations (default: 15 seconds). On a small graph (under 1,000 vectors) this rebuild is cheap and fires for any number of pending mutations, so buffered vectors never sit in the brute-force delta buffer for long during low-volume ingestion. On a larger graph a rebuild re-indexes the whole graph, so the timer only fires once pending mutations reach at least 10% of the effective rebuild threshold - a single stray insert into a large, otherwise-settled index no longer costs a full graph rebuild on the next quiet period. "How large" is measured against the vectors the index actually holds, not against the part of the graph the current session has loaded: the graph loads lazily on the first query, so a process that reopens a database, writes and then goes idle without ever querying it is gated exactly like one that queried first. Configure the timer via inactivityRebuildTimeoutMs (per-index metadata or arcadedb.vectorIndex.inactivityRebuildTimeoutMs globally); set to 0 to disable it.

Searching Vectors Not Yet in the Graph

Between rebuilds, everything written since the last one lives in an in-memory delta buffer rather than in the HNSW graph. Those vectors are still searchable: every query scores the buffer and merges the result into the graph’s own, in distance order, so a search never returns a corpus older than the last write. The cost is linear in the size of the buffer, which is what mutationsBeforeRebuild, rebuildGraphRatio and the inactivity timer between them decide.

Before v26.9.1 this was true of every query except a grouped one. Adding groupBy routed the search down a path that read the graph only, so it answered from the corpus as of the last rebuild while the same query without groupBy, on the same index at the same instant, returned the newer rows - with no error, no warning and nothing in the statistics to say so. On default settings the invisible window was as wide as rebuildGraphRatio allows the buffer to grow, and stayed open for as long as writes kept arriving. If you worked around this by dropping groupBy and grouping in the application, that workaround is no longer needed.

The groupedSearchesMergingDelta counter reports how many grouped searches actually took rows out of the buffer:

-- Rising in step with your grouped query rate means the graph is persistently behind the write rate,
-- and every grouped query is paying a linear scan of the buffer; deltaVectorsCount is how long that scan is
SELECT groupedSearchesMergingDelta, deltaVectorsCount FROM (SELECT expand(stats) FROM schema:indexes WHERE name = 'Document[embedding]')

A high value is not itself a fault - it is the index doing what it should under ingestion. It is the signal to shorten the scan by lowering mutationsBeforeRebuild or rebuildGraphRatio, if query latency matters more than the CPU the extra rebuilds cost.

Rebuild Memory and Deferred Rebuilds

A graph rebuild is the most memory-hungry thing a vector index does, and an online rebuild - one triggered by the mutation threshold or the inactivity timer while the index keeps serving queries - is the most expensive kind. The graph being replaced stays resident so searches keep working, and the new build pays for its own working set on top of it (the build cache, the graph under construction, the ordinal map). Measured on a 50,000 x 128 corpus, that is roughly 1.7x the peak of building the same index from nothing.

Two behaviours follow from that, both new in v26.9.1:

The auto-sized caches budget against available heap, not total heap. graphBuildCacheMaxHeapPercent and searchCacheMaxHeapPercent are shares of the heap actually free at the time, so a rebuild holding the old graph resident asks for a smaller cache instead of the same one. Previously they were shares of -Xmx, which meant a larger heap grew the caches proportionally and a rebuild that did not fit still did not fit after raising it.
A rebuild that will not fit is deferred instead of attempted. Before starting, the estimated peak is compared against arcadedb.vectorIndex.rebuildMaxHeapPercent (default 90) of the available heap. If it does not fit, the cycle is skipped rather than run into an OutOfMemoryError. Nothing is lost: the pending vectors remain fully searchable through the in-memory delta buffer, so the cost is a longer brute-force scan per query until a later rebuild succeeds.

A deferral is deliberately visible. It logs a warning naming what to change, and increments the rebuildsDeferredForMemory counter in the index statistics:

-- Non-zero and climbing means rebuilds keep being declined for lack of heap:
-- the graph is going stale and every query is paying a longer delta scan for it
SELECT rebuildsDeferredForMemory FROM (SELECT expand(stats) FROM schema:indexes WHERE name = 'Document[embedding]')

If it keeps climbing, give the JVM more heap, lower arcadedb.vectorIndex.graphBuildCacheMaxHeapPercent so the build asks for less, or split the index. Setting arcadedb.vectorIndex.rebuildMaxHeapPercent to 0 disables the check and restores the previous attempt-regardless behaviour.

Because a deferral does not consume the mutations that triggered it, the trigger would otherwise fire again on the very next query. arcadedb.vectorIndex.rebuildDeferralCooldownMs (default 30 seconds) is the minimum gap before another attempt; a rebuild that completes clears it immediately.

Only online rebuilds are subject to this check. A first build, the rebuild performed when a database closes, an explicit REBUILD INDEX and COMPACT INDEX are never declined - nothing would retry them, so declining one would mean it never happens at all.

If you create the index before inserting data (e.g., during schema setup), set buildGraphNow: false to skip the initial (empty) graph build. The graph will be built lazily on the first search:

-- Schema setup phase: defer graph build since no data exists yet
CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
  dimensions: 384,
  similarity: 'COSINE',
  quantization: 'INT8',
  buildGraphNow: false
}

-- Bulk load data...
-- Graph is built automatically on first vector.neighbors() query

If you create the index after data is already loaded, leave buildGraphNow at its default (true) so the index is immediately ready to query.

Global Configuration

Set database-wide defaults for vector index parameters:

ALTER DATABASE `arcadedb.vectorIndex.locationCacheSize` 100000
ALTER DATABASE `arcadedb.vectorIndex.graphBuildCacheSize` 10000
ALTER DATABASE `arcadedb.vectorIndex.mutationsBeforeRebuild` 100
ALTER DATABASE `arcadedb.vectorIndex.inactivityRebuildTimeoutMs` 15000
ALTER DATABASE `arcadedb.vectorIndex.storeVectorsInGraph` false

Per-index metadata overrides these global settings.

Vector Embeddings

Choosing an Embedding Model

Creating a Vector Index

Choosing a Similarity Function

Quantization Trade-offs

INT8 Pre-Quantized Ingest

Tuning for Recall vs Speed

Tuning efSearch

Multi-Modal Embeddings

Hybrid Search: Dense + Sparse + Full-Text

Batch Ingestion

Searching Vectors Not Yet in the Graph

Rebuild Memory and Deferred Rebuilds

Global Configuration

Further Reading