MongoDB Query Language
ArcadeDB provides support for both MongoDB Query Language and MongoDB protocol.
If you’re using ArcadeDB as embedded, please add the dependency to the arcadedb-mongodbw library.
If you’re using Maven include this dependency in your pom.xml file.
<dependency>
<groupId>com.arcadedb</groupId>
<artifactId>arcadedb-mongodbw</artifactId>
<version>26.11.1</version>
</dependency>
MongoDB Query Language
If you want to use MongoDB Query Language from Java API, you can simply keep the relevant jars in your classpath and execute a query or a command with "mongo" as language.
Example:
// CREATE A NEW DATABASE
Database database = new DatabaseFactory("heroes").create();
// CREATE THE DOCUMENT TYPE 'HEROES'
database.getSchema().createDocumentType("Heroes");
// CREATE A NEW DOCUMENT
database.transaction((tx) -> {
database.newDocument("Heroes").set("name", "Jay").set("lastName", "Miner").set("id", i).save();
});
// EXECUTE A QUERY USING MONGO AS QUERY LANGUAGE
for (ResultSet resultset = database.query("mongo", // <-- USE 'mongo' INSTEAD OF 'sql'
"{ collection: 'Heroes', query: { $and: [ { name: { $eq: 'Jay' } }, { lastName: { $exists: true } }, { lastName: { $eq: 'Miner' } }, { lastName: { $ne: 'Miner22' } } ], $orderBy: { id: 1 } } }"); resultset.hasNext(); ++i) {
Result doc = resultset.next();
...
}
For more information on the MongoDB query language see: MongoDB CRUD Operations
Mongo queries through Postgres Driver
You can execute a Mongo query against ArcadeDB server by using the Postgres driver and prefixing the query with {mongo}.
Example:
"{mongo} { collection: 'Heroes', query: { $and: [ { name: { $eq: 'Jay' } }, { lastName: { $exists: true } }, { lastName: { $eq: 'Miner' } } ] } }"
ArcadeDB server will execute the query { collection: 'Heroes', query: { $and: [ { name: { $eq: 'Jay' } }, { lastName: { $exists: true } }, { lastName: { $eq: 'Miner' } } ] } } using the Mongo query language.
Mongo queries through HTTP/JSON
You can execute a Mongo query against ArcadeDB server by using HTTP/JSON API. Example of executing an idempotent query with HTTP GET command:
curl "http://localhost:2480/query/graph/mongo/{ collection: 'Heroes', query: { $and: [ { name: { $eq: 'Jay' } }, { lastName: { $exists: true } }, { lastName: { $eq: 'Miner' } } ]} }"
You can also execute the same query in HTTP POST, passing the language and query in payload:
curl -X POST "http://localhost:2480/query/graph" -d "{'language': 'mongo', 'command': '{ collection: \"Heroes\", query: { $and: [ { name: { $eq: \"Jay\" } }, { lastName: { $exists: true } }, { lastName: { $eq: \"Miner\" } } ] } }\"}"
MongoDB Protocol Plugin
If your application is written for MongoDB and you’d like to run it with ArcadeDB instead, you can simply replace the MongoDB server with ArcadeDB server with the MongoDB Plugin installed. This plugin supports MongoDB BSON Network protocol. In this way you can use any MongoDB driver for any supported programming language.
ArcadeDB Server supports a subset of the MongoDB protocol, like CRUD operations and queries.
To start the MongoDB plugin, enlist it in the server.plugins settings.
To specify multiple plugins, use the comma , as separator.
Example to start ArcadeDB with the MongoDB Plugin:
~/arcadedb $ bin/server.sh -Darcadedb.server.plugins="MongoDB:com.arcadedb.mongo.MongoDBProtocolPlugin"
If you’re using MS Windows OS, replace server.sh with server.bat.
In case you’re running ArcadeDB with Docker, use -e to pass settings (Port 27017 is the default MongoDB binary port):
docker run --rm -p 2480:2480 -p27017:27017 \
--env ARCADEDB_SETTINGS="-Darcadedb.server.rootPassword=playwithdata \
-Darcadedb.server.plugins=MongoDB:com.arcadedb.mongo.MongoDBProtocolPlugin " \
arcadedata/arcadedb:latest
The Server output will contain this line:
2018-10-09 18:47:01:692 INFO <ArcadeDB_0> - MongoDB Protocol plugin started [ArcadeDBServer]
Supported commands
The plugin implements a subset of the MongoDB protocol, focused on CRUD operations, queries and index creation:
| Command | Driver helpers |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Any other command is refused with error 59 (CommandNotFound), what MongoDB answers for a command it does not have, so a driver can tell the command is missing (since v26.12.1).
Updating and deleting documents
The update and delete commands accept the same filter operators as find ($eq, $ne, $lt, $lte, $gt, $gte, $in, $nin, $and, $or, $not, $exists, …).
// Delete the first matching document
db.doc.deleteOne({ test: { $eq: "abcde" } })
// Delete all matching documents
db.doc.deleteMany({ counter: { $gte: 5 } })
// Replace the whole document (replaceOne / replaceMany)
db.doc.replaceOne({ test: { $eq: "abcde" } }, { test: "vwxyz" })
// Update with operators ($set, $unset, $inc)
db.doc.updateOne({ test: "v2" }, { $set: { name: "Jay" } })
db.doc.updateMany({ counter: { $gte: 8 } }, { $inc: { counter: 100 } })
db.doc.updateOne({ test: "v1" }, { $unset: { counter: "" } })
// Upsert: insert when nothing matches
db.doc.updateOne({ test: "missing" }, { $set: { counter: 999 } }, { upsert: true })
Notes:
-
deleteOne(),updateOne()andreplaceOne()affect a single matching document;deleteMany()andupdateMany()affect all matches. -
A replacement document (no
$-prefixed keys) fully replaces the matched document’s fields, but keeps its_id. The_idis immutable: an update or replacement that changes or removes it is refused (error 66). -
The supported update operators are
$set,$unsetand$inc. They accept dotted paths ({ $set: { "addr.city": "Rome" } }): the embedded document is updated and missing levels are created. Writing below a value that is not a document (for example a string) is refused. Another MongoDB update operator (such as$push,$addToSetor$setOnInsert) or a pipeline-style update is refused with error238(NotImplemented) naming it, and an operator that MongoDB does not have with error9(FailedToParse), whether or not a document matches (since v26.12.1). -
An update is refused, and changes nothing, in the same cases as MongoDB, with or without a filter (since v26.12.1):
$incon a field that is not a number (a string, boolean, array, date, document ornull) or by an amount that is not a number (TypeMismatch, 14),$incpast the range of a 64-bit integer (BadValue, 2), and two operators on the same field or on a field and one below it, such as{ $set: { s: 2 }, $unset: { s: "" } }(ConflictingUpdateOperators, 40). -
With
upsert: true, when no document matches a new one is inserted, seeded from the filter’s equality conditions plus the update. If the filter includes an_id, that_idis used for the new document; otherwise one is generated. Either way, the resulting_idis returned. -
The counts in the reply are what the database actually wrote. A write refused by a
BeforeRecordCreate/Update/DeleteListener(see Events) is not counted:deleteandinsertcount only the documents removed or stored,updatereports it inn(matched) but not innModified(changed), and an upsert whose match was refused does not insert a new document (since v26.10.2). -
nModifiedcounts only the documents an update changed, as in MongoDB: an update that leaves a document as it was ($setto the current value,$incby 0,$unsetof a missing field, an identical replacement) counts innbut not innModified, and the document is not rewritten. Reordering the fields of an embedded document is a change (since v26.12.1). -
A document refused on its own is reported as a write error of that document, and the other documents of the command are kept, as in MongoDB (since v26.12.1). This covers a duplicate key (
11000), a value a schema constraint refuses (121,DocumentValidationFailure, for example amaxdeclared on a property), and an update the document cannot take (such as writing below a string). Withordered: true(the default) the documents or update statements after the refused one are not applied, and withordered: falsethey are. The Java driver throwsMongoWriteExceptionorMongoBulkWriteExceptionfor these. In production mode the error text does not include the stored values.
Null values, projections, regular expressions and duplicate keys
-
$ne,$nin,$in(withnull) and$notalso match documents whose field isnullor missing, like MongoDB. For example{ age: { $ne: 30 } }returns documents with noage. -
Filters follow MongoDB’s rules for types and arrays, in
find,count,updateanddelete. A number never equals a string or a boolean ({ n: 5 }does not match"5"), and$gt/$ltcompare within one type only. A filter on an array field tests its elements:{ tags: "a" }matches["a", "b"],{ tags: { $ne: "a" } }does not, and$all,$elemMatch,$sizeandtags.0work.$existsworks on dotted paths ({ "a.b": { $exists: false } }), and collection names with dots (fs.files) can be read like any other. -
A filter on
_id(equality or$in) uses the unique_idindex. A filter on any other field is tested on the documents with MongoDB’s own rules, and an index narrows which documents are tested only for a field that an index starts with and whose type is declared in the schema as a scalar (STRING,BOOLEAN,BYTE,SHORT,INTEGER,LONGorDOUBLE; for exampleCREATE PROPERTY products.sku STRINGfollowed byCREATE INDEX ON products (sku) UNIQUE). Equality and$inuse the index, and so does a range ($gt,$gte,$lt,$lte) on an integer field, when the operand is of the kind of the field (a string for aSTRING, an integer for anINTEGER). Lookups, updates, deletes and bulk upserts by such a field do not read the whole collection. A field the schema does not declare is never narrowed by its index, because an array can be stored in it and an index does not see the elements of an array: filters on it, and on dotted paths, negations, regular expressions and temporal values, read every document of the collection and are slower on large ones. The declared type is relied upon: when you declare a property on a collection that already holds data, records stored before the declaration are not converted, so rewrite them if they hold an array or a value of another kind in that field. -
A top-level
$not,$exprand a$regexdirectly inside$elemMatchare refused with an error. Regular expressions are time-bounded byarcadedb.command.regexTimeout, shared by all the documents one command scans: raise it if a big scan with a regular expression times out. The same bound applies to the$matchstages of an aggregation, includingcountDocuments()with a filter, and to the$matchstages nested in$facet,$lookupand$unionWith. A non-regex$exprworks inside an aggregation$match. -
Aggregating or counting a collection that does not exist returns an empty result (
count()andestimatedDocumentCount()return0(since v26.12.1)), and the first insert creates the collection. Acountwith askiporlimitthat is not a number is refused with error14(TypeMismatch). -
find()honors the projection ({ name: 1 },{ id: 0 },{ age: 0 }, dotted paths). Inclusion and exclusion cannot be mixed, except for_id. A dotted path goes into arrays like in MongoDB:{ "items.name": 1 }returns thenameof every element ofitems, and{ "items.price": 0 }removespricefrom every element. An inclusion returns the fields in the order of the stored document. Overlapping paths ({ a: 1, "a.b": 1 }) are refused _(since v26.12.1). Projection operators ($slice,$elemMatch,$meta, the positional$) are not supported and return an error. Documents never contain ArcadeDB’s@rid,@typeor@cat. -
find().sort()follows MongoDB’s order (since v26.12.1). Values of different types are ordered by type (null, numbers, strings, documents, arrays, ObjectId, booleans, dates), numbers by value whatever their type, and strings by character code. An array sorts by its smallest element in an ascending sort and by its largest in a descending one, and an empty array sorts beforenull.skipandlimitpage through that order. Sorting is done in memory: with alimitonly the documents needed are kept, without one every matching document is. -
A
MAPproperty whose keys are not strings (written through SQL or the HTTP API, for example{1: "one"}) is returned with each key as a field named by its text ("1", and"null"for anullkey), as the HTTP API shows it. A dotted path such as{ $set: { "m.1": "uno" } }updates that same key and keeps its type (since v26.10.2). -
Regular expressions work as literals (
/^a/i) and with$regexand$options(i,m,s,x), also under$not. An invalid pattern returns an error. -
_idis unique: inserting an existing_idfails withE11000. In a batch,ordered: true(the default) stops at the first duplicate,ordered: falseinserts the others. The first insert creates the collection and a unique index on_id; on a big existing collection that first insert builds the index and other writes to it wait until it is done. -
The
_idindex keys follow the_idtype (integers, doubles, strings or ObjectIds), so range queries and sorts on numeric ids are numeric. If a collection mixes kinds of_id, the index uses string keys:1and"1"are then the same key, and ordering is textual.
|
An ObjectId and a String with the same 24 hex digits are two different |
BSON value types
Besides strings, numbers, booleans, dates, embedded documents and arrays, the plugin stores and returns these BSON types with their type kept: binary, regular expression, timestamp, MinKey, MaxKey, JavaScript code, Decimal128 and ObjectId. A value the server cannot store is refused with an error instead of being dropped.
-
Decimal128 is stored as an exact decimal. A negative zero (
-0,-0.00),NaNand the infinities are kept as they are (since v26.12.1). A decimal with more than 34 digits, written through SQL, is returned as a double. -
An ObjectId outside
idis stored as the text$oid:<24 hex digits>and returned as an ObjectId. An ObjectId filter matches only ObjectIds: plain hex text, including the one stored by versions before v26.10.1, is a String and is matched by a String filter _(since v26.12.1)._idkeeps the plain hex form. -
Binary, timestamp, regular expression, MinKey, MaxKey, JavaScript and the special Decimal128 values are stored as a small document with the reserved field
$bson. MongoDB filters and sorts compare them in BSON order; a SQL query sees them as documents. -
Reserved names: a field called
$bsonis refused, and a text starting with$oid:or$str:is escaped on write, so it comes back unchanged. -
The Symbol type and binary subtypes other than 0 and 4 are not supported by the bundled MongoDB library.
Authentication
The MongoDB Protocol Plugin supports authentication through the SASL PLAIN mechanism (RFC 4616).
Credentials are validated against the ArcadeDB server users, exactly like the PostgreSQL protocol.
Because ArcadeDB stores passwords as one-way hashes, the challenge-response mechanisms SCRAM-SHA-1 and SCRAM-SHA-256 (the default negotiated by most MongoDB drivers and by mongosh) cannot be supported: they would require the clear-text password to be available on the server side.
For this reason the client must explicitly request the PLAIN mechanism.
Example using a connection string:
mongosh "mongodb://root:playwithdata@localhost:27017/mydb?authMechanism=PLAIN"
Or, from an already opened shell:
db.auth({ user: "root", pwd: "playwithdata", mechanism: "PLAIN" })
The user and pwd are the ArcadeDB server credentials, and the authentication database is checked against the databases the user is authorized to access.
The PLAIN mechanism transmits the credentials in clear text. In production always use it over a TLS-encrypted connection, the same recommendation that applies to the PostgreSQL wire protocol.
|
Index management
The MongoDB Protocol Plugin supports the createIndexes command, used by the driver helpers createIndex() and createIndexes().
Each requested index is mapped to an ArcadeDB LSM-Tree index over the same property list.
// Single-field index
db.doc.createIndex({ test: 1 })
// Unique index
db.doc.createIndex({ email: 1 }, { unique: true })
// Compound index
db.doc.createIndex({ lastName: 1, firstName: -1 })
Notes:
-
If the target collection does not exist yet, it is created automatically (the response reports
createdCollectionAutomatically: true). -
Because MongoDB collections are schemaless, properties that are not yet declared on the type are kept free-form and the index serializes their keys as strings, preserving the stored value type.
-
The sort direction (
1ascending,-1descending) is accepted for compatibility but ignored: ArcadeDB LSM-Tree indexes can be scanned in both directions. -
Re-creating an existing index is a no-op, matching MongoDB semantics.
-
Specialized index types (
text,2dsphere,hashed, …) are created as standard LSM-Tree indexes on the indexed fields.
Listing and terminating running commands
Since v26.10.2 the commands a MongoDB client sends are listed, while they run, by the HTTP
list queries server command and by Cypher SHOW TRANSACTIONS,
with protocol mongodb, the connection’s channel id as connectionId and the client’s address. terminate query,
TERMINATE TRANSACTIONS or the Studio Running Queries tab stop one: the client
receives error 11601 (Interrupted), the code MongoDB answers for an operation that killOp interrupted. root
sees and stops every command, any other user only their own.
Writes refused by an HA replica
Since v26.10.2 a write that an HA replica refuses and that names the leader fails with error 10107
(NotWritablePrimary), the code MongoDB answers a write sent to a secondary with. The errmsg contains not primary,
which older drivers match on, followed by The leader is server '<address>'. A driver answers the code by marking the
server unknown and rescanning the topology. During an election, when no leader is known yet, the refusal names nobody
and keeps the retryable 112 (WriteConflict). See how-to/operations/ha.adoc#ha-leader-refusal.