Graph OLAP Engine

Available since ArcadeDB v26.4.1.

The Graph OLAP Engine maintains a read-optimized, columnar representation of your graph alongside the live OLTP data. It uses Compressed Sparse Row (CSR) encoding and flat primitive arrays to deliver 5x–400x speedups on analytical workloads — multi-hop traversals, graph algorithms, and property aggregations — without sacrificing transactional safety.

Why Graph OLAP?

ArcadeDB’s OLTP engine is optimized for point lookups and ACID transactions. Analytical workloads — PageRank, community detection, multi-hop traversals — access millions of edges in tight loops. The row-oriented, pointer-chasing nature of OLTP storage causes cache misses, object overhead, and GC pressure.

The OLAP engine solves this by encoding graph topology as flat int[] arrays and properties as typed columns:

  • Sequential memory access — cache-line friendly, no pointer chasing

  • Zero object allocation — no GC pressure during traversal

  • SIMD-friendly — enables JVM vectorized operations

  • 9x more compact — flat arrays vs. Java object overhead

Graph Analytical View (GAV)

A Graph Analytical View is a named, schema-persisted OLAP snapshot of selected vertex types, edge types, and properties.

GraphAnalyticalView gav = GraphAnalyticalView.builder(database)
    .withName("social")
    .withVertexTypes("Person", "Company")
    .withEdgeTypes("FOLLOWS", "WORKS_AT")
    .withProperties("name", "age", "status")
    .withUpdateMode(UpdateMode.SYNCHRONOUS)
    .build();

Named views are persisted in schema.json and automatically restored on database restart. As of ArcadeDB v26.9.1, a READY view’s CSR snapshot itself can also be persisted to disk at a clean database close and reused on the next open without rescanning the graph; see CSR Persistence.

SQL

Creating a view

CREATE GRAPH ANALYTICAL VIEW social
  VERTEX TYPES (Person, Company)
  EDGE TYPES (FOLLOWS, WORKS_AT)
  PROPERTIES (name, age, status)
  UPDATE MODE SYNCHRONOUS

All clauses after the view name are optional. A minimal view covering the entire graph:

CREATE GRAPH ANALYTICAL VIEW fullGraph

Use IF NOT EXISTS to avoid errors if the view already exists:

CREATE GRAPH ANALYTICAL VIEW social IF NOT EXISTS
  VERTEX TYPES (Person)
  EDGE TYPES (FOLLOWS)
  UPDATE MODE SYNCHRONOUS

You can also materialize edge properties (e.g., weights):

CREATE GRAPH ANALYTICAL VIEW weighted
  VERTEX TYPES (City)
  EDGE TYPES (ROAD)
  EDGE PROPERTIES (distance, toll)
  UPDATE MODE SYNCHRONOUS
  COMPACTION THRESHOLD 50000

Altering a view

Change the update mode or compaction threshold of an existing view:

ALTER GRAPH ANALYTICAL VIEW social UPDATE MODE ASYNCHRONOUS
ALTER GRAPH ANALYTICAL VIEW social COMPACTION THRESHOLD 20000

Rebuilding a view

Force a full rebuild of the CSR snapshot:

REBUILD GRAPH ANALYTICAL VIEW social

Dropping a view

DROP GRAPH ANALYTICAL VIEW social
DROP GRAPH ANALYTICAL VIEW IF EXISTS social

Listing views

SELECT FROM schema:graphAnalyticalViews

Builder Options

Method Description Default

withName(String)

Named registration + schema persistence

anonymous

withVertexTypes(String…​)

Filter to specific vertex types

all

withEdgeTypes(String…​)

Filter to specific edge types

all

withProperties(String…​)

Materialize specific vertex properties

all

withEdgeProperties(String…​)

Materialize edge properties (e.g., weights)

none

withUpdateMode(UpdateMode)

OFF, SYNCHRONOUS, or ASYNCHRONOUS

OFF

withCompactionThreshold(int)

Rebuild CSR after N accumulated delta edges

10,000

Async Build for Large Graphs

For large graphs, use buildAsync() to avoid blocking the calling thread:

GraphAnalyticalView gav = GraphAnalyticalView.builder(database)
    .withName("large-graph")
    .withUpdateMode(UpdateMode.ASYNCHRONOUS)
    .buildAsync();

// Wait for build completion
boolean ready = gav.awaitReady(30, TimeUnit.SECONDS);

Update Modes

The GAV supports three synchronization modes between OLTP and OLAP:

Mode Behavior Staleness Use Case

OFF

Marks view STALE on commit; requires manual rebuild

Until rebuild

Batch analytics, static snapshots

SYNCHRONOUS

Applies an overlay on each commit

Zero

Real-time analytics, consistent reads

ASYNCHRONOUS

Triggers background rebuild on commit

Brief BUILDING window

Large graphs, tolerable brief inconsistency

In SYNCHRONOUS mode, the engine captures transaction deltas (new/deleted vertices, added/removed edges, property changes) and merges them into an immutable overlay on top of the base CSR. Readers always see a consistent snapshot via an atomic volatile reference swap. When the overlay accumulates too many changes (configurable threshold, default 10,000 edges), a background compaction rebuilds the full CSR.

CSR Persistence

Available since ArcadeDB v26.9.1.

Before v26.9.1, a named view’s CSR was never itself persisted: only its definition (types, properties, update mode) was, in schema.json. Every database open therefore rebuilt the CSR from a full graph scan, and because shutdown() waits for any in-flight rebuild, the cost of that scan landed on close() rather than open() - at 1M vertices, closing a database with a view took over 4 seconds even for a session that ran no query at all.

A READY view (with no pending overlay changes) now writes its CSR to disk at a clean database close, alongside a freshness certificate: the database’s last committed transaction id at the time the CSR was built. On the next open, if a persisted file plausibly applies (see arcadedb.gavPersistCsr below), the view is marked READY immediately, but the file itself is not read yet - see Lazy Restore. That deferred read either:

  • finds the database’s current last committed transaction id still matches the certificate - nothing was committed to the database in between, so the persisted CSR is loaded from disk as-is instead of being rebuilt; or

  • finds a mismatch - anything was committed in between, to any type, not only the ones the view covers - and falls back to the previous behavior: an async rebuild.

This means the speedup only applies to reopening a database that was not written to since its last clean close. A database that receives writes between every session still pays the rebuild on every open, exactly as before v26.9.1.

Persistence is controlled by the arcadedb.gavPersistCsr database-level setting (default true). Set it to false to skip writing the CSR file, for example to avoid its disk footprint or the extra write at close.

Serving queries against a stale CSR while a delta is applied in the background (the way SYNCHRONOUS mode’s overlay already does for live updates) is not part of this: a CSR’s contiguous array layout makes partial coverage expensive to retrofit, so an invalidated certificate always means a full rebuild, not a partial one.

Lazy Restore

Available since ArcadeDB v26.9.1.

A persisted CSR that plausibly applies is not read from disk during open(). Reading and deserializing it is a real, if bounded, cost - up to roughly a second at 10M vertices - and a session that opens a database and never queries the view got nothing for paying it. Instead, open() marks the view READY immediately without touching the file, and the actual read is deferred to whichever of these happens first:

  • a query that actually needs the view’s data; or

  • an explicit call to the view’s awaitReady() method.

Either trigger re-verifies the freshness certificate from scratch at that point (not the one implied at open() time), so a commit that lands after open() but before the first query is still caught correctly and falls back to a rebuild rather than serving stale data. A session that opens and closes a database without ever touching the view now costs what the no-view baseline costs, not what a restore costs - close() does not wait for a restore that was never triggered either.

The arcadedb.gavRestoreAwaitTimeout database-level setting (default 0) forces the wait at open() time instead, for either the deferred restore or a full rebuild: set it to a positive number of milliseconds to trade a slower open() for the view being immediately usable by the query that triggered the reopen, rather than the first query racing (and possibly missing) it.

How CSR Works

The graph topology is stored as two pairs of arrays (forward for outgoing edges, backward for incoming):

Forward CSR (outgoing edges):
  offsets:   [0, 3, 5, 8, ...]     -- one entry per vertex + sentinel
  neighbors: [1, 5, 7, 2, 6, ...]  -- dense neighbor IDs, contiguous per source

  Outgoing neighbors of vertex v = neighbors[offsets[v] .. offsets[v+1])
  Out-degree of vertex v         = offsets[v+1] - offsets[v]   -- O(1)

This layout enables sequential memory access (cache-line friendly) and O(1) degree lookups.

Columnar Property Storage

Properties are stored as typed flat arrays — int[], long[], double[], or dictionary-encoded int[] for strings. Each column has a compact null bitmap (1 bit per vertex). Dictionary encoding maps unique string values to integer codes, achieving near-100% compression for low-cardinality fields.

Edge Properties and Pending Changes

Behavior changed in ArcadeDB v26.9.1.

Edge property columns (EDGE PROPERTIES (…​), used as weights by the shortest-path and spanning-tree algorithms) are laid out in the order the graph had when the view was built. In SYNCHRONOUS mode a commit is absorbed into a pending-changes overlay rather than rebuilding, so from that moment the neighbour list and the columns describe two different orderings.

Before v26.9.1 the view resolved this by reporting no edge properties at all as soon as an overlay existed, and every weighted algorithm silently went back to reading edge records - after every single commit. The view now reconciles the two instead: an edge already in the base graph keeps answering from its column, and an edge the overlay added answers from the value recorded when it was committed. Weighted algorithms therefore keep the fast columnar path while a view is being updated.

Two cases remain that the view cannot answer, and for those the algorithm reads the edge records directly - exact, only slower:

  • several parallel edges join the same two vertices and only some of them were deleted, since the overlay counts deletions per vertex pair and cannot say which of the parallel edges went;

  • an existing edge’s property value was changed, until the rebuild that refreshes the columns completes. Adding and deleting edges does not have this effect; only changing a property of an edge already in the base graph does.

Memory Usage

The OLAP representation is significantly more compact than the OLTP equivalent:

  • CSR topology: ~8 bytes per edge (bidirectional)

  • Node ID mapping: ~8 bytes per vertex

  • Columnar properties: 4–8 bytes per vertex per column

  • Null bitmaps: 1 bit per vertex per column

Example: for a graph with 500K vertices and 8M edges, the GAV uses 134.6 MB compared to an estimated ~1.2 GB for the OLTP representation — 9.3x more compact.

long bytes = gav.getMemoryUsageBytes();

Graph Algorithms

The module includes parallelized graph algorithms that operate directly on CSR arrays with zero GC pressure:

Algorithm Description

PageRank

Pull-based, parallel, configurable damping factor and iterations

Connected Components

Min-label propagation for weakly connected components

BFS

Breadth-first search with distance arrays

SSSP (Dijkstra)

Single-source shortest path for weighted graphs

Label Propagation

Community detection

Triangle Counting

Count 3-cliques in the graph

Local Clustering Coefficient

Per-vertex clustering coefficients

 
GraphAlgorithms algos = new GraphAlgorithms();

// PageRank (20 iterations, damping 0.85)
double[] ranks = algos.pageRank(gav, 20, 0.85);

// Connected Components
int[] components = algos.connectedComponents(gav);

// BFS from a source vertex
int[] distances = algos.bfs(gav, sourceNodeId);

Algorithms and Pending Changes

Behavior changed in ArcadeDB v26.9.1.

In SYNCHRONOUS mode a committed change is absorbed into a pending-changes overlay rather than rebuilding the view. The overlay keeps the slot of a deleted vertex and numbers an added vertex above the base graph, so from that moment the view’s vertex count and the range of its internal vertex numbers stop being the same figure.

Before v26.9.1 the algo.* procedures took them for one and the same, with two consequences:

  • a vertex added after the view was built could be missing from the answer of every procedure, with no error to say so;

  • once the additions outnumbered the deletions, the CSR-accelerated procedures (algo.wcc, algo.pagerank, algo.labelpropagation, algo.localClusteringCoefficient, algo.bfs) failed with an ArrayIndexOutOfBoundsException - one added vertex was enough.

Both are fixed. The vertices a view holds are renumbered onto a gapless range before any algorithm sees them, so every algo.* procedure answers for exactly the vertices that are live at the time of the call - the added ones included, the deleted ones excluded.

While a view is holding pending changes the whole-graph procedures also stop using the parallel kernels listed above, which read the base CSR arrays directly and know nothing of the overlay: they would otherwise answer for the graph as it stood at the last build. Those calls still run off the view’s adjacency, single-threaded rather than parallel, and the compaction that folds the overlay back into the base graph (by default after 10,000 pending edges) restores the parallel path.

Query Planner Integration

The Cypher query planner automatically detects ready GAVs and substitutes OLTP traversal operators with CSR-based operators when:

  • A named GAV is registered and in READY state

  • The GAV covers the required vertex and edge types

  • The query does not return edge variables as first-class records (edges in CSR have no RID; edge properties are fully supported)

No query changes are needed — the optimizer transparently accelerates matching traversal patterns.

Lifecycle

// Check status
if (gav.isReady()) { /* safe to query */ }

// Status values: NOT_BUILT, BUILDING, READY, STALE
Status status = gav.getStatus();

// Drop (removes from registry + schema)
gav.drop();

// Shutdown (release resources, schema definition persists)
gav.shutdown();

Benchmark Results

On a graph with 500K vertices and ~8M edges:

Benchmark OLTP OLAP Speedup

1-hop count

6.9 µs

1.2 µs

5.7x

2-hop

101.4 µs

5.1 µs

19.8x

3-hop

1,037 µs

56.4 µs

18.4x

5-hop

194,046 µs

5,141 µs

37.7x

Shortest Path

394 ms/pair

7.5 ms/pair

52.8x

PageRank (20 iter)

124,563 ms

316 ms

394.2x

Connected Components

5,591 ms

197 ms

28.4x

Label Propagation

62,619 ms

645 ms

97.1x

Limitations

  • CSR uses int[] arrays — maximum ~2.1 billion vertices per bucket and ~2.1 billion edges per direction

  • Edges in CSR do not carry their own RID; the Cypher query planner falls back to OLTP only when the query returns an edge variable as a first-class record (e.g., RETURN r). Edge properties are fully supported via withEdgeProperties()

  • Dictionary encoding applies only to string properties

  • Initial build requires a full scan of selected vertex/edge types. A rebuild triggered by an invalidated persisted CSR certificate (or by OFF mode, or by exceeding the SYNCHRONOUS/ASYNCHRONOUS compaction threshold) also requires a full scan - only a clean-close-to-unchanged-reopen cycle skips it