Graph OLAP Engine

ArcadeDB’s OLTP engine is optimized for point lookups and ACID transactions. Analytical workloads — PageRank, community detection, multi-hop traversals — access millions of edges in tight loops. The row-oriented, pointer-chasing nature of OLTP storage causes cache misses, object overhead, and GC pressure.

The OLAP engine solves this by encoding graph topology as flat int[] arrays and properties as typed columns:

A Graph Analytical View is a named, schema-persisted OLAP snapshot of selected vertex types, edge types, and properties.

Named views are persisted in schema.json and automatically restored on database restart. As of ArcadeDB v26.9.1, a READY view’s CSR snapshot itself can also be persisted to disk at a clean database close and reused on the next open without rescanning the graph; see CSR Persistence.

Creating a view

All clauses after the view name are optional. A minimal view covering the entire graph:

Choosing the vertex properties

Without a PROPERTIES clause the view copies every vertex property. PROPERTIES (a, b) copies only the listed ones, and a list of names prefixed with !, quoted with backticks, copies every property except those: PROPERTIES (`!Body`, `!Text`). Use one form or the other: when a list mixes them, the exclusions apply and the plain names are ignored.

(Since v26.10.2) an empty list, PROPERTIES (), copies no vertex property: the view holds only the graph’s structure, which is all that algorithms such as algo.pagerank() or algo.wcc() need, and it builds faster and takes less memory. In v26.10.1 and earlier PROPERTIES () was a parsing error.

Use IF NOT EXISTS to avoid errors if the view already exists:

You can also materialize edge properties (e.g., weights):

Add CCH (…​) to keep a Customizable Contraction Hierarchy per weight property, for fast point-to-point shortest paths. A weight listed there is materialized as an edge property even when EDGE PROPERTIES does not name it:

Edge sub-types

Listing an edge type in EDGE TYPES also materializes its sub-types, and a hop on that type answers with the sub-types' edges too, as it does without a view. The view keeps one slice per concrete type, so the order the types are listed in does not matter. Views persisted by an earlier release are rebuilt once.

Vertex sub-types

Since v26.11.1 listing a vertex type in VERTEX TYPES also holds the vertices of its sub-types, including sub-types created after the view was built, as a query on that type reads them without a view. A view over V with W EXTENDS V therefore covers both, and serves queries once W holds vertices. Before, a view held the listed types only, and once a sub-type held a vertex the view no longer covered every vertex, so the planner never used it (issue #10208). A CSR persisted by an earlier release is rebuilt once on the first open.

A type that already holds vertices and is attached to a listed type later (ALTER TYPE Y SUPERTYPE +V) holds vertices the view never read: the view does not use them until it is rebuilt, which it does by itself in SYNCHRONOUS and ASYNCHRONOUS mode, while an OFF view turns STALE.

The same holds for an edge type attached to a listed edge type later (ALTER TYPE G SUPERTYPE +E) when it already held edges between the view’s vertices, lightweight ones included: a hop on E reads the edges of its sub-types, so the view is not used for any query until it has rebuilt (an OFF view turns STALE and is not used even with useWhenStale). An edge type that already existed when the view was built or restored is held only once a scan has read it, even when it looks empty, because a transaction open at that moment may still commit edges of it: attaching it rebuilds the view once. An edge type created after that, and not written while it was not covered, is used at once, as is a sub-type created after the build. Before v26.11.1 such a view answered hops on E without the edges of G (issue #10214).

Altering a view

Change the update mode or compaction threshold of an existing view:

Rebuilding a view

Force a full rebuild of the CSR snapshot:

A view whose build failed when the database opened is no longer loaded, but its definition is kept: REBUILD builds it again from the definition, and reports the error to the caller if the build fails again. Before v26.11.1 REBUILD refused such a view as "not loaded", and the only way back was DROP and CREATE (issue #10368).

Dropping a view

Types a view lists

A view stores the vertex and edge types it lists, and the edge types of its contraction hierarchies, by name. Like a materialized view does with its source types, it protects them:

Before v26.11.1 both were accepted. The view kept serving a CSR that no longer matched the schema, and the next database open failed to build it and left it unbuilt (issue #10368).

Listing views

For large graphs, use buildAsync() to avoid blocking the calling thread:

The GAV supports three synchronization modes between OLTP and OLAP:

In SYNCHRONOUS mode, the engine captures transaction deltas (new/deleted vertices, added/removed edges, property changes) and merges them into an immutable overlay on top of the base CSR. Readers always see a consistent snapshot via an atomic volatile reference swap.

In ASYNCHRONOUS mode the same deltas are captured, but merged into the overlay by a background task instead of by the committing thread, so the commit does not wait for the merge; commits that arrive together are merged by one task. Until a commit’s delta is merged the view reports STALE: by default it is then not used, so a query (including the next one of the committing thread) reads the records and sees the commit, while with arcadedb.gavUseWhenStale the view keeps serving the snapshot as of the previous merge. awaitReady() waits for the pending merges. Since v26.11.1; before, every relevant commit started a full rebuild of the CSR whatever the size of the change, and the view served no query until it finished, even with arcadedb.gavUseWhenStale, so under a steady stream of small commits almost no query used it (issue #10189). If the merge falls more than 100,000 commits behind, the pending changes are dropped and the view rebuilds once, STALE in the meantime.

Commit order. In both modes the changes reach the overlay in the order their transactions committed, however many threads commit at once. A thread whose commit follows another one on the same records returns from commit() once its change is in the view, which is after the change it follows; this is why a commit can wait a moment on a slower one. arcadedb.gavCommitOrderMaxWait (30 seconds by default, 0 to not wait) bounds that wait; a transaction that never reports costs the commits behind it one such wait, logged as a WARNING, and is then skipped. Since v26.11.1; before, two commits that conflicted could reach the overlay in the opposite order, and when an edge was created in the record slot a deleted edge had just freed, the deletion arriving second withdrew the new edge from the view for good, so the counts it served were a few edges short of the same queries without the view (issue #10249).

Writes during a build or rebuild. A view starts tracking changes before it scans the graph, so a transaction that commits while the view is being built or rebuilt is never lost: in SYNCHRONOUS mode it is applied on top of the finished view (checked against what the scan already read, so nothing is counted twice) in SYNCHRONOUS and ASYNCHRONOUS modes, and in OFF mode the view is published STALE. Writers are not blocked by a running build, including REBUILD GRAPH ANALYTICAL VIEW.

A committed change that the view cannot merge, or that a build cannot replay on top of the CSR it just scanned, is not dropped: the view logs a warning, reports itself STALE and rebuilds, so it never keeps answering from a graph that never existed (issue #10338). Before 26.11.1 such a change was dropped with a warning, and a first build that hit one never published.

In SYNCHRONOUS and ASYNCHRONOUS modes the view being rebuilt keeps serving queries, and keeps absorbing commits, until the rebuild publishes its new CSR: only the caller of REBUILD GRAPH ANALYTICAL VIEW (or of build()) waits for the scan. Since v26.10.1 the scan no longer runs while holding the view’s lock, so neither a committing transaction nor a status check waits behind it (issue #8403); before, the view served during a blocking rebuild lagged the commits until the rebuild finished. A rebuild superseded by a newer one while it scans waits for the newer one and reports its outcome (success or error); a rebuild interrupted by the database closing fails with an error rather than reporting success.

Until then the overlay costs whole-graph reads (the Cypher count operators, the algo.* procedures) one copy of the adjacency per committed change, made on the first read after the change: the overlay applied to the base CSR, kept with that snapshot for every later read. Before v26.11.1 a view with any overlay sent those reads down the per-vertex path instead (a 2-hop count measured about a quarter slower), from the first committed edge until the compaction (issue #10411).

The copy is made for the reads that pay for it. A read that starts from a handful of vertices - a 2-hop count from one indexed vertex visits some 64 adjacency entries - does not copy the adjacency of the whole graph for itself: a commit replaces the snapshot, so the copy would serve no second read. Such reads walk the vertices one by one, as they did before the overlay, and the copy is made by the read that brings the entries the reads of the snapshot have visited to what the copy costs (a whole-graph read does it at once; point reads do it only when no commit comes first). Since v26.11.1 point reads while writes are committed take about as long as before the copy existed, where they had become three to six times slower at 20 to 100 commits per second (issue #10471).

Lightweight edges and bulk loads. A lightweight edge is tracked like any other edge in every mode: its creation and its deletion reach the overlay in SYNCHRONOUS and ASYNCHRONOUS modes, and mark an OFF view STALE. Each copy of a duplicated lightweight edge counts as an edge of its own, and a delete removes one copy. Copies have no identity to tell an old one from a new one, so when lightweight edges of a pair change while the view is being built or rebuilt, the view checks that pair against the graph before serving it; if a duplicated copy made them disagree, it rebuilds rather than serve the pair one copy off. Edges loaded with GraphBatch (and so by the graph importer, the HTTP batch endpoint and the gRPC batch load) are written in bulk, without an event per edge, so the view learns of them when the batch closes: OFF goes STALE, and SYNCHRONOUS and ASYNCHRONOUS rebuild rather than absorbing millions of edges into the overlay. While that rebuild runs the view is not READY, so queries take the ordinary path and never answer without the load. During the batch itself the view still serves what it held before the batch started.

Before v26.9.1, a named view’s CSR was never itself persisted: only its definition (types, properties, update mode) was, in schema.json. Every database open therefore rebuilt the CSR from a full graph scan, and because shutdown() waits for any in-flight rebuild, the cost of that scan landed on close() rather than open() - at 1M vertices, closing a database with a view took over 4 seconds even for a session that ran no query at all.

A READY view (with no pending overlay changes) now writes its CSR to disk at a clean database close, alongside a freshness certificate: the database’s last committed transaction id at the time the CSR was built. On the next open, if a persisted file plausibly applies (see arcadedb.gavPersistCsr below), the view is marked READY immediately, but the file itself is not read yet - see Lazy Restore. That deferred read either:

This means the speedup only applies to reopening a database that was not written to since its last clean close. A database that receives writes between every session still pays the rebuild on every open, exactly as before v26.9.1.

Persistence is controlled by the arcadedb.gavPersistCsr database-level setting (default true). Set it to false to skip writing the CSR file, for example to avoid its disk footprint or the extra write at close.

Serving queries against a stale CSR while a delta is applied in the background (the way SYNCHRONOUS mode’s overlay already does for live updates) is not part of this: a CSR’s contiguous array layout makes partial coverage expensive to retrofit, so an invalidated certificate always means a full rebuild, not a partial one.

The graph topology is stored as two pairs of arrays (forward for outgoing edges, backward for incoming):

This layout enables sequential memory access (cache-line friendly) and O(1) degree lookups.

Properties are stored as typed flat arrays — int[], long[], double[], or dictionary-encoded int[] for strings. Each column has a compact null bitmap (1 bit per vertex). Dictionary encoding maps unique string values to integer codes, achieving near-100% compression for low-cardinality fields.

A column takes its type from the schema, or for an undeclared property from the first values the build samples. Since v26.11.1 a value the column cannot give back as it is (a 2.5 or a NaN in an integer column, a FLOAT in a double one, a number in a string one) is not stored in it: a read of that vertex’s property through the view falls back to the record, so queries answer with the value the record holds. Until v26.10.2 it was converted into the column’s type, and a query over the view could read 2 where the record held 2.5 (issue #10255).

The OLAP representation is significantly more compact than the OLTP equivalent:

Example: for a graph with 500K vertices and 8M edges, the GAV uses 134.6 MB compared to an estimated ~1.2 GB for the OLTP representation — 9.3x more compact.

The module includes parallelized graph algorithms that operate directly on CSR arrays with zero GC pressure:

A view can keep a Customizable Contraction Hierarchy for a weight property: a precomputed structure that answers point-to-point shortest paths in a time that depends on the depth of the hierarchy rather than on how far apart the two vertices are. It is meant for road, logistics, utility and other infrastructure networks that serve many routing queries.

Queries go through algo.cch.shortestPath (Cypher) or cchShortestPath() (SQL). A query is answered by the hierarchy when its weight property and relationship types are exactly those of a hierarchy on a ready view; otherwise, and whenever the hierarchy cannot answer for the view’s current state, it is answered by bidirectional Dijkstra. Answers are exact either way, and a transaction holding uncommitted changes always sees them.

How it works

On grids, the hardest case for this technique, a query on 360,000 vertices takes about 0.5 ms for a route across the whole grid against about 50 ms for Dijkstra, and the gap grows with the size of the graph. Road networks have much smaller separators than grids and benefit more.

Keeping up with changes

The hierarchy follows the view: every change the view absorbs is followed by a background preparation, on the view’s build executor. In SYNCHRONOUS mode most changes are caught up in milliseconds:

Measured on the California road network (DIMACS USA-road-d.CAL: 1.9M vertices, 4.7M arcs) through a database: a query takes 0.04 to 0.1 ms at any distance, against 18 to 230 ms for Dijkstra; updating 1, 10 and 100 weights takes about 0.7, 8 and 65 ms from commit until the hierarchy answers for it; closing a road about 4 ms; adding a road the hierarchy does not join about 0.9 s; the first build about 13 s. Each query needs a few tens of KB of scratch, sized to the depth of the hierarchy rather than to the graph.

The order is persisted next to the view’s CSR at a clean close (see CSR Persistence) and reused when the database is reopened with nothing committed in between, so a reopen only contracts and customizes.

Suitability

The technique relies on small separators. Graphs without them, such as social graphs or graphs with supernodes, would need a supergraph many times larger than themselves. A hierarchy that would exceed arcadedb.gavCchMaxArcsPerEdge supergraph arcs per edge (default 16) is not built: it reports UNSUITABLE, and queries keep using bidirectional Dijkstra. A REBUILD GRAPH ANALYTICAL VIEW tries again.

Monitoring

SELECT FROM schema:graphAnalyticalViews and GraphAnalyticalView.getStats() list each hierarchy with its status (PREPARING, READY, UNSUITABLE, UNAVAILABLE), size in arcs and memory, how many topologies were built or restored, how many customizations ran, and how many queries it answered or handed back to Dijkstra.

The weight of an edge is the numeric value of its weight property; an edge without one weighs 1, and an edge whose value is negative is not walked.

The Cypher query planner automatically detects ready GAVs and substitutes OLTP traversal operators with CSR-based operators when:

No query changes are needed — the optimizer transparently accelerates matching traversal patterns.

Relationship uniqueness. Cypher binds a relationship at most once per MATCH clause, so MATCH (a)-[:KNOWS]-(b)-[:KNOWS]-(c) never walks back to a over the edge it just took. A view holds adjacency, not edge records, so since v26.10.1 a hop that could reuse a relationship of another hop in its clause - they share an edge type, or either is untyped - is walked one edge type and direction at a time and remembers which relationship it took, and EXPLAIN marks it unique relationships. Such a clause is served from the view when every one of those hops is anonymous and has a fixed length; a named relationship variable or a variable-length relationship among them sends all of them to the records instead. Hops that cannot collide - (a)-[:KNOWS]→(b)-[:WORKS_AT]→(c) - are unaffected. Since v26.11.1 this also holds for an edge type with sub-types: a hop on KNOWS with KNOWS_WELL EXTENDS KNOWS walks the slice of each concrete type on its own and names a relationship after the type it actually has, so a hop on the parent and a hop on the sub-type recognize the same edge. Before, such a clause walked the edge records (issue #10213). Before v26.10.1 a view answered such patterns without the rule, counting for example every friend-of-a-friend path that returns to its start over the same edge, and every path that took a self-loop twice (issue #8394). An undirected chain of several hops also counted each self-loop once per adjacency list it sits in, even across separate MATCH clauses.

Views over some of the vertex types. A view built with VERTEX TYPES (…​) holds only the vertices of those types, so the edges that leave them are missing from its adjacency. Edge types do not declare which vertex types they connect, so the planner cannot know that a traversal stays inside the view: since v26.10.2 a view is used for SQL out(), in(), both(), MATCH, TRAVERSE, shortest path, the count push-downs and the algo.* procedures only when it covers every vertex type of the database that holds a vertex, and these statements otherwise read the records. Before that they answered with the covered part of the graph, for example 0 rows for MATCH (b:B)-[:E]→(c:C) on a view over (A, B) (issue #9301). A type with no vertex of its own does not count, because no traversal can reach it: a parent type whose records all live in the sub-types the view lists (Message over Post and Comment), or a type not used yet. The first vertex created in such a type makes the view partial again, without any schema change. A view over a subset of the types still serves a one-hop Cypher scan whose labels it holds. To accelerate the other statements, list every vertex type that holds vertices.

Stale views. A view that is STALE (in OFF mode after a commit, in ASYNCHRONOUS mode until a commit is merged, or after a failed rebuild) is not used by the planner unless arcadedb.gavUseWhenStale is true, in which case queries read the snapshot as of its last build. The setting is per database, and applies to every view of that database immediately, including views already built and views restored at open (since v26.10.1; before, only the JVM-wide -Darcadedb.gavUseWhenStale had any effect, issue #7875):

The builder’s withUseWhenStale(boolean) overrides it for one view, and that override is saved with the view’s definition.

A one-hop pattern feeding an aggregation, such as

is read entirely from the view since v26.10.1: the sources are the view’s nodes carrying the source label, both endpoints read their properties from the view’s columns, and every edge reaches the aggregation as a lightweight row rather than a copied record. EXPLAIN shows it as GAV ONE-HOP SCAN. This applies when the statement is a single MATCH of one fixed-length relationship feeding an aggregating RETURN, with no relationship or path variable and no inline property map, the optimizer would otherwise scan the source label (not seek it through an index), and the view is not stale and covers every vertex type the two labels can match. A count per endpoint alone (RETURN p.city, count(*)) keeps its own, even cheaper, edge-count plan, and an order-sensitive aggregate such as collect(), or a grouped LIMIT/SKIP without ORDER BY, keeps the regular plan.

Counts that build no row. A statement whose only output is count(*) - and a COUNT { } subquery - is answered by a count push-down when its pattern is a chain, a star around one node, or one of the LSQB shapes; EXPLAIN shows it under Using Count Push-Down. Since v26.10.2 the push-downs also take:

Since v26.11.1 a triangle written as one MATCH, MATCH (a:Person)-[:KNOWS]-(b:Person)-[:KNOWS]-(c:Person)-[:KNOWS]-(a) RETURN count(*), is a count push-down too, with no WHERE or with one that orders the three variables strictly on one property (WHERE a.id < b.id AND b.id < c.id, in any spelling of < and > that orders all three). Each triangle is found once by the sorted intersection algo.triangleCount uses and weighed by the matches it stands for, so on a 10,000-person, 180,000-edge graph the ordered count takes milliseconds instead of seconds (issue #10255). The answer is the pattern’s own, not the algorithm’s simple graph: parallel relationships are matches of their own, the comparison is on the property values (two equal values, a null, or values of two types such as a number and a string drop the triangle), and without a WHERE the six walks of every triangle and the walks over self loops are counted. The hops must be undirected, anonymous and of one type, and the three variables carry one label or none; EXPLAIN shows COUNT TRIANGLES (CSR sorted intersection) with the order it counts.

Multi-hop chains on the view. A chain of labeled hops such as MATCH (a:P)-[:K]→(b:P)-[:K]→(c:P) is walked on the view (EXPLAIN shows GAVFusedChain). Since v26.10.2 a count(), or count(x) of a variable of the chain, is counted during the walk with up to two grouping keys that are chain variables, in a RETURN or a WITH (RETURN a, count() AS n); EXPLAIN shows it on the chain as fused count(*) AS n. When the statement returns the paths' rows instead, they are produced in bounded batches, so memory no longer grows with the number of paths and a LIMIT stops the walk early (issue #9738). If the view is rebuilt while such a result is still being read, the query fails with a retryable error rather than mixing rows from two builds: run it again.

WHERE on a chain walked on the view. Since v26.10.2 each AND-ed condition of the WHERE is checked on the hop that binds the last variable it reads, so a path that fails it is not walked any further (issue #9938). In MATCH (a:P)-[:K]→(b:P)-[:K]→(c:P) WHERE a.id < b.id AND b.id < c.id, the first condition is checked as soon as b is reached and the second one once c is: EXPLAIN shows them on the chain as filter on (b)=a.id < b.id and filter on (c)=b.id < c.id. A pattern the view walks one hop at a time instead of as one chain - a cycle such as a triangle, closed by a GAVExpandInto hop, or a pattern that branches from one of its nodes - gets one Filter per condition right above the hop that binds its variables, as a pattern without a view does. A triangle counted with ordered ids (WHERE a.id < b.id AND b.id < c.id) so prunes its wedges before closing them, about three times faster on the LDBC SNB SF1 KNOWS graph. A condition that reads no node of the pattern is still checked on the complete path.

A transaction that holds uncommitted changes - any change, even to types the view does not cover - does not use the view: the view serves the committed graph only, and a query inside the transaction must see the transaction’s own writes. Such a query, Cypher, Gremlin or a graph algorithm, runs on the records until the transaction commits, which PROFILE shows as a plan without the view. Run analytical queries outside write transactions, or after committing, to keep the acceleration.

On a graph with 500K vertices and ~8M edges: