Graph OLAP Engine

ArcadeDB’s OLTP engine is optimized for point lookups and ACID transactions. Analytical workloads — PageRank, community detection, multi-hop traversals — access millions of edges in tight loops. The row-oriented, pointer-chasing nature of OLTP storage causes cache misses, object overhead, and GC pressure.

The OLAP engine solves this by encoding graph topology as flat int[] arrays and properties as typed columns:

A Graph Analytical View is a named, schema-persisted OLAP snapshot of selected vertex types, edge types, and properties.

Named views are persisted in schema.json and automatically restored on database restart. As of ArcadeDB v26.9.1, a READY view’s CSR snapshot itself can also be persisted to disk at a clean database close and reused on the next open without rescanning the graph; see CSR Persistence.

Creating a view

All clauses after the view name are optional. A minimal view covering the entire graph:

Use IF NOT EXISTS to avoid errors if the view already exists:

You can also materialize edge properties (e.g., weights):

Add CCH (…​) to keep a Customizable Contraction Hierarchy per weight property, for fast point-to-point shortest paths. A weight listed there is materialized as an edge property even when EDGE PROPERTIES does not name it:

Edge sub-types

Listing an edge type in EDGE TYPES also materializes its sub-types, and a hop on that type answers with the sub-types' edges too, as it does without a view. The view keeps one slice per concrete type, so the order the types are listed in does not matter. Views persisted by an earlier release are rebuilt once.

Altering a view

Change the update mode or compaction threshold of an existing view:

Rebuilding a view

Force a full rebuild of the CSR snapshot:

Dropping a view

Listing views

For large graphs, use buildAsync() to avoid blocking the calling thread:

The GAV supports three synchronization modes between OLTP and OLAP:

In SYNCHRONOUS mode, the engine captures transaction deltas (new/deleted vertices, added/removed edges, property changes) and merges them into an immutable overlay on top of the base CSR. Readers always see a consistent snapshot via an atomic volatile reference swap.

Writes during a build or rebuild. A view starts tracking changes before it scans the graph, so a transaction that commits while the view is being built or rebuilt is never lost: in SYNCHRONOUS mode it is applied on top of the finished view (checked against what the scan already read, so nothing is counted twice), in OFF mode the view is published STALE, and in ASYNCHRONOUS mode it rebuilds once more. Writers are not blocked by a running build, including REBUILD GRAPH ANALYTICAL VIEW.

In SYNCHRONOUS mode the view being rebuilt keeps serving queries, and keeps absorbing commits, until the rebuild publishes its new CSR: only the caller of REBUILD GRAPH ANALYTICAL VIEW (or of build()) waits for the scan. Since v26.10.1 the scan no longer runs while holding the view’s lock, so neither a committing transaction nor a status check waits behind it (issue #8403); before, the view served during a blocking rebuild lagged the commits until the rebuild finished. A rebuild superseded by a newer one while it scans waits for the newer one and reports its outcome (success or error); a rebuild interrupted by the database closing fails with an error rather than reporting success.

Lightweight edges and bulk loads. A lightweight edge is tracked like any other edge in every mode: its creation and its deletion reach the overlay in SYNCHRONOUS mode, mark an OFF view STALE and rebuild an ASYNCHRONOUS one. Each copy of a duplicated lightweight edge counts as an edge of its own, and a delete removes one copy. Copies have no identity to tell an old one from a new one, so when lightweight edges of a pair change while the view is being built or rebuilt, the view checks that pair against the graph before serving it; if a duplicated copy made them disagree, it rebuilds rather than serve the pair one copy off. Edges loaded with GraphBatch (and so by the graph importer, the HTTP batch endpoint and the gRPC batch load) are written in bulk, without an event per edge, so the view learns of them when the batch closes: OFF goes STALE, ASYNCHRONOUS rebuilds, and SYNCHRONOUS rebuilds too rather than absorbing millions of edges into the overlay. While that rebuild runs the view is not READY, so queries take the ordinary path and never answer without the load. During the batch itself the view still serves what it held before the batch started.

Before v26.9.1, a named view’s CSR was never itself persisted: only its definition (types, properties, update mode) was, in schema.json. Every database open therefore rebuilt the CSR from a full graph scan, and because shutdown() waits for any in-flight rebuild, the cost of that scan landed on close() rather than open() - at 1M vertices, closing a database with a view took over 4 seconds even for a session that ran no query at all.

A READY view (with no pending overlay changes) now writes its CSR to disk at a clean database close, alongside a freshness certificate: the database’s last committed transaction id at the time the CSR was built. On the next open, if a persisted file plausibly applies (see arcadedb.gavPersistCsr below), the view is marked READY immediately, but the file itself is not read yet - see Lazy Restore. That deferred read either:

This means the speedup only applies to reopening a database that was not written to since its last clean close. A database that receives writes between every session still pays the rebuild on every open, exactly as before v26.9.1.

Persistence is controlled by the arcadedb.gavPersistCsr database-level setting (default true). Set it to false to skip writing the CSR file, for example to avoid its disk footprint or the extra write at close.

Serving queries against a stale CSR while a delta is applied in the background (the way SYNCHRONOUS mode’s overlay already does for live updates) is not part of this: a CSR’s contiguous array layout makes partial coverage expensive to retrofit, so an invalidated certificate always means a full rebuild, not a partial one.

The graph topology is stored as two pairs of arrays (forward for outgoing edges, backward for incoming):

This layout enables sequential memory access (cache-line friendly) and O(1) degree lookups.

Properties are stored as typed flat arrays — int[], long[], double[], or dictionary-encoded int[] for strings. Each column has a compact null bitmap (1 bit per vertex). Dictionary encoding maps unique string values to integer codes, achieving near-100% compression for low-cardinality fields.

The OLAP representation is significantly more compact than the OLTP equivalent:

Example: for a graph with 500K vertices and 8M edges, the GAV uses 134.6 MB compared to an estimated ~1.2 GB for the OLTP representation — 9.3x more compact.

The module includes parallelized graph algorithms that operate directly on CSR arrays with zero GC pressure:

A view can keep a Customizable Contraction Hierarchy for a weight property: a precomputed structure that answers point-to-point shortest paths in a time that depends on the depth of the hierarchy rather than on how far apart the two vertices are. It is meant for road, logistics, utility and other infrastructure networks that serve many routing queries.

Queries go through algo.cch.shortestPath (Cypher) or cchShortestPath() (SQL). A query is answered by the hierarchy when its weight property and relationship types are exactly those of a hierarchy on a ready view; otherwise, and whenever the hierarchy cannot answer for the view’s current state, it is answered by bidirectional Dijkstra. Answers are exact either way, and a transaction holding uncommitted changes always sees them.

How it works

On grids, the hardest case for this technique, a query on 360,000 vertices takes about 0.5 ms for a route across the whole grid against about 50 ms for Dijkstra, and the gap grows with the size of the graph. Road networks have much smaller separators than grids and benefit more.

Keeping up with changes

The hierarchy follows the view: every change the view absorbs is followed by a background preparation, on the view’s build executor. In SYNCHRONOUS mode most changes are caught up in milliseconds:

Measured on the California road network (DIMACS USA-road-d.CAL: 1.9M vertices, 4.7M arcs) through a database: a query takes 0.04 to 0.1 ms at any distance, against 18 to 230 ms for Dijkstra; updating 1, 10 and 100 weights takes about 0.7, 8 and 65 ms from commit until the hierarchy answers for it; closing a road about 4 ms; adding a road the hierarchy does not join about 0.9 s; the first build about 13 s. Each query needs a few tens of KB of scratch, sized to the depth of the hierarchy rather than to the graph.

The order is persisted next to the view’s CSR at a clean close (see CSR Persistence) and reused when the database is reopened with nothing committed in between, so a reopen only contracts and customizes.

Suitability

The technique relies on small separators. Graphs without them, such as social graphs or graphs with supernodes, would need a supergraph many times larger than themselves. A hierarchy that would exceed arcadedb.gavCchMaxArcsPerEdge supergraph arcs per edge (default 16) is not built: it reports UNSUITABLE, and queries keep using bidirectional Dijkstra. A REBUILD GRAPH ANALYTICAL VIEW tries again.

Monitoring

SELECT FROM schema:graphAnalyticalViews and GraphAnalyticalView.getStats() list each hierarchy with its status (PREPARING, READY, UNSUITABLE, UNAVAILABLE), size in arcs and memory, how many topologies were built or restored, how many customizations ran, and how many queries it answered or handed back to Dijkstra.

The weight of an edge is the numeric value of its weight property; an edge without one weighs 1, and an edge whose value is negative is not walked.

The Cypher query planner automatically detects ready GAVs and substitutes OLTP traversal operators with CSR-based operators when:

No query changes are needed — the optimizer transparently accelerates matching traversal patterns.

Relationship uniqueness. Cypher binds a relationship at most once per MATCH clause, so MATCH (a)-[:KNOWS]-(b)-[:KNOWS]-(c) never walks back to a over the edge it just took. A view holds adjacency, not edge records, so since v26.10.1 a hop that could reuse a relationship of another hop in its clause - they share an edge type, or either is untyped - is walked one edge type and direction at a time and remembers which relationship it took, and EXPLAIN marks it unique relationships. Such a clause is served from the view when every one of those hops is anonymous and has a fixed length; a named relationship variable or a variable-length relationship among them sends all of them to the records instead. Hops that cannot collide - (a)-[:KNOWS]→(b)-[:WORKS_AT]→(c) - are unaffected. Before v26.10.1 a view answered such patterns without the rule, counting for example every friend-of-a-friend path that returns to its start over the same edge, and every path that took a self-loop twice (issue #8394). An undirected chain of several hops also counted each self-loop once per adjacency list it sits in, even across separate MATCH clauses.

Views over some of the vertex types. A view built with VERTEX TYPES (…​) holds only the vertices of those types, so the edges that leave them are missing from its adjacency. Edge types do not declare which vertex types they connect, so the planner cannot know that a traversal stays inside the view: since v26.11.1 a view is used for SQL out(), in(), both(), MATCH, TRAVERSE, shortest path, the count push-downs and the algo.* procedures only when it covers every vertex type of the database that holds a vertex, and these statements otherwise read the records. Before that they answered with the covered part of the graph, for example 0 rows for MATCH (b:B)-[:E]→(c:C) on a view over (A, B) (issue #9301). A type with no vertex of its own does not count, because no traversal can reach it: a parent type whose records all live in the sub-types the view lists (Message over Post and Comment), or a type not used yet. The first vertex created in such a type makes the view partial again, without any schema change. A view over a subset of the types still serves a one-hop Cypher scan whose labels it holds. To accelerate the other statements, list every vertex type that holds vertices.

Stale views. A view that is STALE (in OFF mode after a commit, or after a failed rebuild) is not used by the planner unless arcadedb.gavUseWhenStale is true, in which case queries read the snapshot as of its last build. The setting is per database, and applies to every view of that database immediately, including views already built and views restored at open (since v26.10.1; before, only the JVM-wide -Darcadedb.gavUseWhenStale had any effect, issue #7875):

The builder’s withUseWhenStale(boolean) overrides it for one view, and that override is saved with the view’s definition.

A one-hop pattern feeding an aggregation, such as

is read entirely from the view since v26.10.1: the sources are the view’s nodes carrying the source label, both endpoints read their properties from the view’s columns, and every edge reaches the aggregation as a lightweight row rather than a copied record. EXPLAIN shows it as GAV ONE-HOP SCAN. This applies when the statement is a single MATCH of one fixed-length relationship feeding an aggregating RETURN, with no relationship or path variable and no inline property map, the optimizer would otherwise scan the source label (not seek it through an index), and the view is not stale and covers every vertex type the two labels can match. A count per endpoint alone (RETURN p.city, count(*)) keeps its own, even cheaper, edge-count plan, and an order-sensitive aggregate such as collect(), or a grouped LIMIT/SKIP without ORDER BY, keeps the regular plan.

Counts that build no row. A statement whose only output is count(*) - and a COUNT { } subquery - is answered by a count push-down when its pattern is a chain, a star around one node, or one of the LSQB shapes; EXPLAIN shows it under Using Count Push-Down. Since v26.11.1 the push-downs also take:

A transaction that holds uncommitted changes - any change, even to types the view does not cover - does not use the view: the view serves the committed graph only, and a query inside the transaction must see the transaction’s own writes. Such a query, Cypher, Gremlin or a graph algorithm, runs on the records until the transaction commits, which PROFILE shows as a plan without the view. Run analytical queries outside write transactions, or after committing, to keep the acceleration.

On a graph with 500K vertices and ~8M edges: