Databases and queries
This guide covers the life of a database directory and the two calls that talk to it. Prefer SQL or openCypher through db.command(…) and db.query(…) for schema, inserts and graph work. The bindings also expose wrapper objects for the rest of the Java API; they are not covered here.
Every script on this page ran against the published arcadedb-jvm wheel. Run each one in an empty directory.
The life of a database
A database is a directory on your filesystem. It is created, opened, used, closed and, if you want, dropped.
import arcadedb
# Create (raises if the path already holds a database)
with arcadedb.create_database("./mydb") as db:
print(f"Created at: {db.get_database_path()}")
db.command("sql", "CREATE VERTEX TYPE User")
with db.transaction():
db.command("sql", "INSERT INTO User SET name = ?", "Alice")
print("Exists:", arcadedb.database_exists("./mydb"))
# Creating it again fails
try:
with arcadedb.create_database("./mydb"):
pass
except Exception as e:
print(type(e).__name__, e, sep=": ")
# Open, and read a value back
with arcadedb.open_database("./mydb") as db:
print("Name:", db.get_name())
for user in db.query("sql", "SELECT FROM User"):
print(user.get("name"))
# Open an existing database or create it
path = "./myapp"
if arcadedb.database_exists(path):
db = arcadedb.open_database(path)
else:
db = arcadedb.create_database(path)
with db:
print("Transaction active:", db.is_transaction_active())
with db.transaction():
print("Transaction active:", db.is_transaction_active())
print("Transaction active:", db.is_transaction_active())
# Drop: deletes all files, irreversible
with arcadedb.open_database("./myapp") as db:
db.drop()
print("Exists after drop:", arcadedb.database_exists("./myapp"))
Created at: ./mydb
Exists: True
ArcadeDBError: Failed to create database: com.arcadedb.exception.DatabaseOperationException: Database './mydb' already exists
Name: mydb
Alice
Transaction active: False
Transaction active: True
Transaction active: False
Exists after drop: False
Notes on the script:
-
create_database(path)creates the directory and opens the database for reading and writing. It raisesArcadeDBErrorif the path already holds a database, as the third output line shows. -
open_database(path)opens an existing database.database_exists(path)tells the two cases apart. -
withcloses the database when the block ends, also when an exception leaves it. On close, active transactions roll back. You can calldb.close()yourself, insidetry/finally, but the context manager is the recommended way. -
db.drop()deletes all files of the database. It cannot be undone. -
db.is_transaction_active()isTrueonly insidewith db.transaction():.
Locking
The open database holds an operating-system lock on database.lck in its directory. A second process cannot open the same directory while the first holds it; open it in one process and share the handle between that process’s threads.
After a crash the operating system releases the lock and leaves the file behind. A leftover file blocks nothing: the next open sees it and replays the write-ahead log to recover. Do not delete it by hand, because that skips the recovery.
Several databases, one JVM
Each Python process starts one JVM, on the first database or server it creates. All databases of the process share it. To set JVM options, call start_jvm(…) before the first database; see JVM options.
Write and query
import arcadedb
with arcadedb.create_database("./qdb") as db:
db.command("sql", "CREATE VERTEX TYPE Person")
db.command("sql", "CREATE PROPERTY Person.name STRING")
db.command("sql", "CREATE PROPERTY Person.age INTEGER")
db.command("sql", "CREATE EDGE TYPE Knows")
with db.transaction():
db.command("sql", "INSERT INTO Person SET name = ?, age = ?", "Alice", 30)
db.command("sql", "INSERT INTO Person SET name = ?, age = ?", "Bob", 25)
db.command("sql", "INSERT INTO Person SET name = ?, age = ?", "Charlie", 35)
# Positional parameters
for p in db.query("sql", "SELECT FROM Person WHERE name = ? AND age > ?", "Alice", 25):
print("positional:", p.get("name"), p.get("age"))
# Named parameters
rows = db.query(
"sql",
"SELECT FROM Person WHERE name = :name AND age > :min_age",
{"name": "Alice", "min_age": 25},
)
print("named:", [r.get("name") for r in rows])
# Named collection parameter
rows = db.query(
"sql",
"SELECT FROM Person WHERE age IN :ages ORDER BY age",
{"ages": [25, 30]},
)
print("collection:", [r.get("name") for r in rows])
# first() and to_list()
first_person = db.query("sql", "SELECT FROM Person ORDER BY name").first()
print("first:", first_person.get("name"))
people = db.query("sql", "SELECT FROM Person ORDER BY name").to_list()
print("to_list:", len(people), people[1]["name"])
missing = db.query("sql", "SELECT FROM Person WHERE name = ?", "Unknown").first()
print("no match:", missing)
# Count a type
print("count_type:", db.count_type("Person"))
# Update and delete in a transaction
with db.transaction():
db.command("sql", "UPDATE Person SET age = ? WHERE name = ?", 26, "Bob")
db.command("sql", "DELETE FROM Person WHERE name = ?", "Charlie")
print("after write:", [(r.get("name"), r.get("age")) for r in db.query("sql", "SELECT FROM Person ORDER BY name")])
# SQLScript: several statements in one call
script = """
CREATE VERTEX TYPE ScriptVertex;
INSERT INTO ScriptVertex SET name = 'test';
ALTER TYPE ScriptVertex ALIASES sv;
"""
with db.transaction():
result = db.command("sqlscript", script)
last = result.first()
print("sqlscript:", last.get("operation"), last.get("typeName"))
# @rid-based pagination
last_rid = "#-1:-1"
batch_size = 1
page_no = 0
while True:
chunk = db.query(
"sql",
"SELECT @rid AS rid, name FROM Person WHERE @rid > ? LIMIT ?",
last_rid,
batch_size,
).to_list()
if not chunk:
break
page_no += 1
print("page", page_no, [c["name"] for c in chunk])
last_rid = str(chunk[-1]["rid"])
# openCypher through the same two calls
with db.transaction():
db.command(
"opencypher",
"MATCH (a:Person {name: 'Alice'}), (b:Person {name: 'Bob'}) CREATE (a)-[:Knows]->(b)",
)
for row in db.query(
"opencypher",
"MATCH (a:Person)-[:Knows]->(b:Person) RETURN a.name AS a, b.name AS b",
):
print("cypher:", row.get("a"), "knows", row.get("b"))
# Aggregation
total = db.query("sql", "SELECT count(*) AS total FROM Person").first().get("total")
print("total:", total)
positional: Alice 30
named: ['Alice']
collection: ['Bob', 'Alice']
first: Alice
to_list: 3 Bob
no match: None
count_type: 3
after write: [('Alice', 30), ('Bob', 26)]
sqlscript: ALTER TYPE ScriptVertex
page 1 ['Alice']
page 2 ['Bob']
cypher: Alice knows Bob
total: 2
Parameters
Bind values as parameters instead of pasting them into the statement text. Identifiers such as type and property names cannot be bound and belong in the text.
-
Positional: one
?per value, from the extra arguments (db.query("sql", q, "Alice", 25)) or from one list or tuple (db.query("sql", q, ["Alice", 25])). -
Named:
:namein the statement and a dict as the one extra argument. A named parameter can hold a collection, as inage IN :ages. -
Nonebinds as null.
A list or tuple on its own is the positional list, so it never binds as one parameter. db.query("sql", "SELECT FROM V WHERE @rid IN ?", [rid_a, rid_b]) has two values for one ?. The bindings raise TypeError before the statement runs when a SQL or SQLScript call has more positional values than the statement can bind, and since 26.11.1 the engine refuses a mismatched count too (ArcadeData/arcadedb#10721). Older engines bound rid_a, dropped rid_b without a word and returned one row where two were meant. To bind a list as one parameter, bind it by name or wrap it so it is the one positional value:
db.query("sql", "SELECT FROM V WHERE @rid IN :ids", {"ids": [rid_a, rid_b]})
db.query("sql", "SELECT FROM V WHERE @rid IN ?", [[rid_a, rid_b]])
The bindings' check counts the ?, $ and : characters of the SQL text, so its count is an upper bound: a ? inside a string literal counts as a placeholder, and a call that binds is never refused. The engine counts exactly: from 26.11.1 it also refuses fewer values than placeholders, with an error that names both counts (an ArcadeDBError wrapping the engine’s CommandSQLParsingException), where older engines bound null for the missing ones. Pass None for a placeholder you mean as null. db.query and db.command run the bindings' check; the async executor does not, so there the engine’s error reaches the error callback.
A query vector
A query vector for vectorNeighbors can be a numpy array or a list. The conversion before the statement runs is the cost to watch: a query takes about 0.3 to 0.75 ms, and a vector of 1536 numbers took 0.5 to 0.6 ms to convert when it was a list.
-
A
float32array crosses as one buffer on both builds: about 5 to 15 us. This is the fast path.np.asarray(v, dtype=np.float32)makes it from a list or from another array. -
A list that holds only Python floats (no
int,bool,None,stror numpy scalar) also crosses as one buffer, and so does afloat64array on the JVM build. The statement reads the same list of doubles it always read: no value changes, and anintin a list stays anint. A list shorter than 32 numbers (JVM build) or 64 numbers (native build) is not worth a buffer and goes number by number. On the native build the list crosses as a buffer only whenfloat32holds every number exactly, which is true of a list from a float32 model (array.tolist()); a list of other doubles,NaNor an infinity is sent as text, because a buffer would round it. -
A
float64array is afloat32array to the engine on the native build and when it is the one positional parameter on the JVM build, so its numbers are rounded tofloat32(a vector index storesfloat32). As a named parameter on the JVM build it keeps its doubles.
Results
db.query returns a result set you can iterate. Two methods cover the usual needs:
-
first()returns the first row, orNonewhen there is none, soif db.query(…).first():tests whether anything matched. -
to_list()materialises the whole result as a list of dicts. Use it when you need the whole result in memory; iterate when you stop early or the result is large.
Record ids in rows
A record id inside a row value (SELECT @rid AS r, a LINK property, a list of links such as out('Knows').@rid) comes back as an RID object, not as text. str(rid) is the text ("#10:5"), so is an f-string, and get_rid() always returns text or None. Two RIDs of the same record are equal and hash alike, so they work in sets and as dict keys, and RIDs sort by bucket, then position. getBucketId(), getPosition(), toString() and isValid() work on the object. It goes back in as a query parameter (WHERE @rid = ?, or a list passed by name) and to db.lookup_by_rid(), as itself or as its text.
row = db.query("sql", "SELECT @rid AS r, name FROM User LIMIT 1").first()
rid = row.get("r")
print(str(rid), rid.getBucketId(), rid.getPosition()) # #1:0 1 0
print(db.lookup_by_rid(rid).get("name")) # the same record, from the object or from str(rid)
Compare record ids as str(rid). A row also has an identity of its own: result.get_rid() returns it for a record row, an expand() row and a projection of @rid that does not alias it, and None for any other projection.
Pagination
Page a large type by @rid instead of SKIP: each page is WHERE @rid > ? LIMIT ?, and the cursor is the @rid of the last row of the previous page. Start with "#-1:-1". The script above runs the loop with a page size of one so that two short pages print.
SQLScript
db.command("sqlscript", script) runs several statements in one call, and the result set holds the last executed statement, which is why the script above reports the ALTER TYPE statement. The script syntax is in the SQLScript reference.
openCypher
The same two calls run openCypher: pass "opencypher" as the language. The script creates an edge with a CREATE pattern and reads it back with MATCH. The language is documented in the openCypher reference.
ArcadeDB SQL has no JOINs, so SQLAlchemy and other ORMs, and tools that generate JOIN queries, do not work with these two calls. Write the statements yourself, following links as in No JOINs, or use openCypher for relationships.
|
Database settings
An engine setting of scope DATABASE, which includes every arcadedb.vectorIndex.* setting, is changed on one open database with SQL, which db.command runs like any other statement:
db.command("sql", "ALTER DATABASE `arcadedb.vectorIndex.rebuildGraphRatio` 0.05")
The key is backquoted because it contains dots. The change applies to that database only. A setting of scope SERVER is refused there, with a message that names SET SERVER SETTING. For the vector settings, the index statistics show the value in force: index.get_stats()["effectiveMutationsThreshold"] is the number of changes after which the graph is rebuilt, computed from mutations_before_rebuild, rebuildGraphRatio and maxPendingMutations (see the create_vector_index docstring).