Concurrency, processes and crashes

Every number on this page comes from probe scripts that were run on 2026-10-24 on one Linux x86_64 machine. The setup of each probe is next to its numbers. The scripts and their raw logs are not part of this repository yet.

Share one Database object between the threads of a process.

The probe: one writer thread and 1, 4 or 16 reader threads in one process, all using one Database, for 10 minutes (3 minutes on Python 3.14t). The writer commits batches of 5 rows and rolls back every seventh batch. On every read, a reader checks that the row count is not below the rows the writer had already committed, that the count is a multiple of 5 (no batch half applied), that no row of a rolled-back batch is visible, and that a random batch reads back whole with matching checksums. At the end the probe reopens the database and counts again.

All 7 runs ended with no error, no bad checksum and the same number of rows before and after the reopen. This probe has one writer. For several writers, read the next section.

Two transactions that change the same record conflict at commit. The one that loses raises ArcadeDBError, and the name ConcurrentModificationException is in the message or in the cause chain. The engine treats it as retryable: see Isolation for when it is raised. db.run_in_transaction(fn, retries=…​) runs fn in a transaction again after such a conflict.

The probe: 8 threads, one shared Database, 4 counter records. Each operation is one transaction that changes one counter, chosen by a fixed pseudo-random sequence, and inserts one row with a unique key into a ledger type. Each thread makes 1,500 operations (500 on Python 3.14t), so 12,000 in total (4,000 on Python 3.14t). The probe counts a lost update as an operation whose transaction committed but whose increment is not in the final counters. Each cell below is the number of lost updates out of the number of operations, from one run.

The first two rows change the counter in the statement itself: UPDATE Counter SET n = n + 1 WHERE id = ?. The last three read the counter with SELECT n, compute n + 1 in Python, and write it back with UPDATE Counter SET n = ? WHERE id = ?. "Own retry loop" is the loop in A retry loop for REPEATABLE_READ: it retries only on the three Java exception names ConcurrentModificationException, NeedRetryException and LockTimeoutException, up to 60 attempts, with a random backoff of up to 100 ms. The last row starts each transaction with BEGIN ISOLATION REPEATABLE_READ.

Without any retry, the atomic update raised ConcurrentModificationException for 6,602 of 12,000 operations on the JVM build, 7,613 of 12,000 on the native build and 2,552 of 4,000 on Python 3.14t. The other operations committed (5,398, 4,387 and 1,448), and the counters added up to exactly those numbers. No write was lost without an error.

What the runs show:

Based on that, do the arithmetic in the statement and retry the transaction. If the new value needs a computation that SQL cannot do, start the transaction with BEGIN ISOLATION REPEATABLE_READ and retry as in A retry loop for REPEATABLE_READ. The evidence for each is one run per build, as in the table. Do not read a value, compute in Python and write it back under the default isolation level.

Call the bindings from asyncio through asyncio.to_thread or loop.run_in_executor. A call made directly on the event loop blocks the loop for as long as the call takes.

The probe: 100 tasks, each making 20 operations (10 on Python 3.14t), so 2,000 operations (1,000 on Python 3.14t) per call style. One operation is db.run_in_transaction(fn, retries=60) with an atomic counter update and one unique ledger insert, then a count. A heartbeat task sleeps 10 ms in a loop and records how late it wakes up. Each cell below is that lateness in milliseconds, as the 99th percentile and the maximum. In the short phases, where the 99th percentile equals the maximum, there were too few heartbeats for them to differ. The two async_executor() rows use the bindings' AsyncExecutor with set_parallel_level(1) and set_parallel_level(4): each task inserts one row with command(…​, callback=…​, error_callback=…​), and the callbacks hand the result to the loop with loop.call_soon_threadsafe.

In all 15 phases (5 call styles, 3 builds) every operation landed (2,000 of 2,000 or 1,000 of 1,000), with no deadlock and no error. That is one run per cell, so no recommendation beyond "this works" follows from it. The phase that called the bindings directly on the loop stalled it for up to 653.5 ms on the JVM build, 928.9 ms on the native build and 382.3 ms on Python 3.14t. The lateness in the other phases differs between the builds. Measure it on your own workload if a bound matters to you.

One process owns an embedded database at a time. Start worker processes with spawn and give each its own database. If several processes must share data, run a server and use a driver: see Packages that talk to a server.

The probe has these cases. Each ran once per build.

Based on that:

A write that has returned from commit() survived every kill in the probe. The probe cannot say anything about a power loss or an operating system crash, which a process kill does not simulate. For how the engine flushes its write-ahead log, see WAL flush and durability.

The kill probe: a child process writes in a loop, in transactions of 5 or 200 rows, and prints the number of rows after every commit that returned. The parent sends SIGKILL (kill -9) at a random time between 0.2 and 2.5 s after the first commit, then reopens the same database. The reopen must recover, CHECK DATABASE must report nothing, every acknowledged row must be present with a matching checksum, no transaction may be half applied, and the database must accept a write. The same database was used for all runs of a build.

A killed process lost no acknowledged commit in any of the 45 runs, and every check passed.

The signal probe: a child opens a transaction and inserts rows without committing. The parent sends the signal 0.3 s after the first insert. After the child ended, or after 60 s if it did not, the parent reopens the database and runs the same checks. For the rows with a handler, the child must also report that it caught KeyboardInterrupt. Each build ran every row 5 times (2 times on Python 3.14t).

If a process must stop and it does not react to a signal, SIGKILL cost no acknowledged commit in 45 runs (20 on the JVM build, 20 on the native build, 5 on Python 3.14t).

The native wheel is a py3-none wheel, and it installed and ran on free-threaded Python 3.14.5 with the GIL off. The Python 3.14t rows in the tables above are those runs: threads, several writers, asyncio and the signal and kill probes. The probe on several processes was not run there. The runs are shorter than those on Python 3.12, as listed at the top of this page.

The JVM package was not run on Python 3.14t. Its wheels up to 26.11.1.dev0 were tagged for 3.11 to 3.14 (cp314), so uv found no wheel for 3.14t (it reported no wheel with a matching Python version tag for arcadedb-jvm 26.10.2.post1). The newer wheels are py3-none, and JPype has a 3.14t wheel on Linux, so pip can now install the package on 3.14t. No JVM test run has been made there, so treat it as untested.