Time Series

(Since v26.11.1) ALTER PROPERTY declares MANDATORY, NOTNULL, MIN, MAX and REGEXP on a column of a TIMESERIES type as on any property, and every write path enforces them: SQL INSERT, the InfluxDB line protocol, gRPC, Prometheus remote write and the JSONL import. The value checked is the one the column stores, read the way SELECT returns it:

A SQL INSERT that violates a constraint fails with a ValidationException, as for a document. Unlike a document type, a multi-row INSERT (CONTENT […​], multi-row VALUES, FROM SELECT) refused at one row keeps the rows appended before it, because each sample is appended in its own shard transaction (see Appends and Transactions). On the line protocol, gRPC and Prometheus remote write the sample that violates it is dropped alone and named in unreadableValues (<measurement>.<column> (tag|field|timestamp): <reason>, the reason naming the constraint and never the value); the JSONL import applies its row-error policy. As for a document type, a constraint is refused when a sample already stored violates it (Cannot set MAX 10: the sample at timestamp …​ violates it). A sample cannot be updated, but it can be deleted by its time and its tags (see Deleting Samples), so the remedy is to delete the sample the message names and declare the constraint again:

Or, to keep the old type as it is, a new TIMESERIES type that declares the constraint, filled with the conforming samples:

A stored STRING column reads an absent value back as the empty text, so MANDATORY/NOTNULL cannot be checked against the samples already there, only against the new ones.

DEFAULT and HIDDEN are refused on a TIMESERIES column: no write path fills in an absent column and every read returns all of them, so either would be stored without any effect. READONLY is accepted, a sample being immutable anyway.

The fastest remote ingestion path. ArcadeDB exposes an InfluxDB Line Protocol-compatible HTTP endpoint that skips SQL parsing entirely.

Line Protocol format: <measurement>[,<tag>=<value>…​] <field>=<value>[,…​] [<timestamp>]

A malformed line is not ingested; the rest of the payload is. When any line was malformed the endpoint answers 400 with a partial-write body: written and dropped counts (a malformed line counts as one dropped sample) and malformedLines, the 1-based numbers of the rejected lines in the payload (the first 100). A field with no value (temp, temp= or a line cut short after a field key, such as weather temp=21.5,hum) makes the whole line malformed. So does a , separator with nothing after it, in the tags or in the fields (weather temp=21.5, or weather,city=rome, temp=1).

(Since v26.10.2) A tag with an empty value (cpu,host= value=1) makes the line malformed too, as InfluxDB rejects it with "missing tag value"; to write a sample without a tag, leave the tag out. Earlier versions stored it as host="". The gRPC write RPCs refuse an empty-string tag value with INVALID_ARGUMENT (an unset value still means the tag is absent), and the Java remote clients refuse it before sending.

(Since v26.10.2) A sample carrying a tag or field key its type does not declare in that role (a misspelled tag such as weather,citty=rome temp=21.5, or a field sent as a tag) is dropped, because storing it would file the point under a different series than the one sent, with the declared tag left null. The other samples of the request are stored, and the 400 partial-write body names the key in undeclaredKeys as <measurement>.<key> (tag|field) (the first 100; dropped counts every sample), with a message naming the database setting below. The OpenAPI TimeSeriesWriteError schema carries the new property, and the gRPC TimeSeriesWriteSummary reports the same list as undeclared_keys (field 8) on the unary and streaming write RPCs. A sample that simply omits a declared key is still accepted. Earlier versions stored the point without the undeclared key and answered 204.

The database setting arcadedb.timeSeriesUndeclaredKeys decides this for every time-series write: line protocol, gRPC, Prometheus remote write and SQL INSERT. reject (the default) drops and reports the sample as above (a failed statement for INSERT); ignore stores the sample with the undeclared keys discarded, which is what a schema-less producer such as Telegraf, sending extra tags the type was never meant to hold, needs:

A value its column cannot store drops only the sample that carries it, which the 400 partial-write body names in unreadableValues as <measurement>.<column> (tag|field): <reason> (the reason names the column and its type, never the value; dropped is the count of samples, the list holds one entry per distinct reason and the first 100). The gRPC TimeSeriesWriteSummary carries the same list as unreadable_values. (Since v26.10.2) the line protocol and gRPC read a value as INSERT does, instead of failing the whole request with a server error:

Precision parameter: ns (nanoseconds, default), us (microseconds), ms (milliseconds), s (seconds).

Example with curl:

Example with Python:

If the type does not exist and auto-creation is enabled (arcadedb.tsAutoCreateType=true), the schema is inferred from the first line: measurement name becomes the type, tags become TAG columns, fields become FIELD columns with inferred types.

ArcadeDB acts as a drop-in Prometheus remote storage backend. Configure Prometheus to write to ArcadeDB:

The endpoint accepts the standard Prometheus remote_write Protobuf payload (snappy-compressed). Each time series is mapped to an ArcadeDB TimeSeries type named after the name label. Types are auto-created if they do not exist.

(Since v26.10.2) A label with an empty value is the same as an absent label, as in Prometheus: it is stored as absent and {label=""} selects it. A series carrying a label that the existing type does not declare as a tag is dropped and the request answers 400 naming the label (the other series of the request are stored); Prometheus does not retry a 400. Set arcadedb.timeSeriesUndeclaredKeys=ignore to discard the extra label instead.

(Since v26.10.2) The endpoint implements remote write 1.0 (prometheus.WriteRequest) and answers as the remote-write specification asks of such a receiver:

Standard ArcadeDB SQL syntax works for TimeSeries types:

The same literal, per column type and surface (since v26.11.1, pinned by a test that runs every cell). 1700000000000 is 2023-11-14 22:13:20 UTC in epoch milliseconds, and 1700000000 the same instant in seconds:

An operator who writes one measurement from both places should write it the same way from both: an ISO date in text, or a number in the unit the surface reads (milliseconds in SQL, the column’s unit on the wire). The refusal of SQL names the rule.

The fastest path — bypasses all protocol and SQL overhead:

TimeSeriesEngine.appendSamples() and appendBatch() are the engine’s own API: they store the values as given, and are for data the application has already checked. database.async().appendSamples() is the checked one. (Since v26.11.1) It converts every value as INSERT does and holds it to the constraints of the type’s columns (MANDATORY, NOTNULL, MIN, MAX, REGEXP) before it returns, on the calling thread: a value no column can store, or a sample a constraint refuses, raises a ValidationException that names the column and the reason, and the whole batch is refused, with nothing appended. Before, such a sample was stored (and then could not be removed, or declared against, when the constraint was declared again), or the batch was lost inside the async worker with waitCompletion() returning normally.

The form that takes a TimeSeriesBatch (database.async().appendSamples(type, batch)) is checked the same way. (Since v26.11.1) A TAG or STRING text of more than 256 UTF-8 bytes, which a row cannot hold, is refused on the calling thread with a ValidationException that names the column and the limit, and the whole batch is refused; before, the call returned and the whole batch, the conforming rows included, was lost inside the async worker. The typed setters of a batch (setLong(), setDouble(), …​) store the primitive they are given. TimeSeriesBatch.setValue(), for a value you hold as an Object, converts it as INSERT does and raises an IllegalArgumentException naming the column for a value the column cannot store: a number past the range of an INTEGER, SHORT or BYTE column, text that is not true or false for a BOOLEAN column, text that is not a number for a numeric column, a text over the limit. Before, a Long past the INTEGER range was stored modulo 2^32, any text for a BOOLEAN column was stored as false, and numeric text failed with a bare ClassCastException.

A TimeSeries append is not part of the transaction that contains it. This is true of every ingestion method above - SQL INSERT, the REST write endpoint, the line protocol, Prometheus remote write and the Java embedded API alike:

Use a session or a transaction around an append for what it does do - it holds that session’s lock and principal and keeps its idle timer from reaping it while you ingest - and not for atomicity with the samples.

Time series types support standard SQL SELECT with WHERE, GROUP BY, and ORDER BY. Time range conditions (BETWEEN, >, >=, <, <=, =) on the timestamp column are pushed down to the storage engine for efficient range scans. A time range on one branch of an OR never narrows the other branch, and two equalities on the same tag in one AND match nothing when the values differ, exactly as on a document type.

An aggregation that groups by the time bucket, by up to four TAG columns, or by both is answered by the storage engine in one pass (AGGREGATE FROM TIMESERIES in EXPLAIN), the shape of a per-host hourly report. Every projected tag must also be in the GROUP BY, and the aggregates are avg, min, max, sum and count(*) over numeric fields. A query outside that shape (a field in the GROUP BY, a projected tag that is not grouped by, count(field), DISTINCT) gives the same answer through the generic row-at-a-time plan, which is slower.

An aggregation with no GROUP BY at all is answered by the storage engine as well, as one row over the queried range (AGGREGATE FROM TIMESERIES …​ ungrouped in EXPLAIN). Every sealed block lying wholly inside the range is answered from the sample count, minimum, maximum and sum it records, so only the blocks that cross a bound of the range are decompressed, and the cost of the query follows the number of blocks rather than the number of samples. Over a range holding no sample it answers one row, 0 for count(*) and NULL for every other aggregate, as SQL does.

Range conditions on a numeric FIELD are pushed down too: >, >=, <, <=, = and BETWEEN against a literal or a parameter, combined with AND (FIELDS in EXPLAIN). A sealed block whose recorded minimum and maximum rule a condition out is skipped without being read, a block whose whole range satisfies every condition is answered as if there were no condition (from its statistics, in an aggregation), and only the blocks in between are decompressed, with the condition tested on the decoded columns before any row is built. That applies to the aggregations above, so SELECT count(*) …​ WHERE ts …​ AND temperature > 30 is answered by the storage engine, and to a plain row query, which then builds rows only for the samples that pass.

When the storage engine answers the whole WHERE of a row query - a time range, tag equalities and the field conditions above, combined with AND - the plan has no FILTER ITEMS step, and a projection made only of column names (aliases allowed) builds each row once, in the fetch (PROJECTED in EXPLAIN), instead of building a row of every column and copying it, as for SELECT sensor_id, ts, temperature FROM SensorReading WHERE ts >= ? AND ts < ? AND temperature > 30.

The comparisons pushed down are those SQL makes as plain numbers: a DOUBLE field against any number, and a LONG, INTEGER, SHORT or BYTE field against an integer. A missing measurement never satisfies one, as NULL > 30 does not. Any other condition - on a FLOAT field, <>, an OR, a comparison between two columns, a function of the field, a decimal against an integer field - gives the same answer through the generic filter.

ArcadeDB provides a comprehensive set of ts.* SQL functions for time series analytics.

A simplified REST endpoint is available for programmatic access and Grafana integration:

For raw (non-aggregated) queries, omit the aggregation and bucketInterval fields.

The optional bucketOrigin (epoch milliseconds, also available on the Grafana endpoint, on gRPC as bucket_origin_ms and in the Java client as TimeSeriesQuery.bucketOrigin()) moves the bucket grid like the origin option of ts.timeBucket, see Moving the bucket grid.

To retrieve the most recent data point:

Repeat tag once per tag column to narrow to a single series; every occurrence must match:

A sample can be absent, which means "no measurement here" rather than a value of zero. You write one either as NaN or as null, and both are stored the same way on a DOUBLE or FLOAT column. Aggregations treat it that way:

This is the semantics of SQL’s NULL under SUM, AVG and count(*). The PromQL endpoint is the one exception: its sum, avg, sum_over_time and avg_over_time propagate NaN, because that is what Prometheus does and a PromQL query is expected to answer what Prometheus would.

Over the REST and Grafana endpoints an absent value is serialized as JSON null, since JSON has no NaN literal. A Grafana panel therefore draws a gap for such a bucket rather than a dip to zero. Over SQL it reads back as NULL, both as a projected column and as the result of an aggregate with nothing to aggregate:

A NaN that arithmetic itself produces over real samples - the sum of +Infinity and -Infinity, for instance - is an undefined total rather than an absence, and is returned as NaN.

SUM, AVG, MIN and MAX read the column’s values, so they need a column that is stored as a number. That means a numeric FIELD: DOUBLE, FLOAT, LONG, INTEGER, SHORT, BYTE, BOOLEAN (as 1 and 0), and the datetime types.

A column that is not stored as a number is refused, and the message names it:

Tags are always stored as text, whatever type they are declared with, so a tag cannot be aggregated even when it is declared LONG. The same applies to a STRING field and to the timestamp column itself.

COUNT is the exception and works on any column. The SQL push-down also behaves differently: rather than refusing the query it simply falls back to the generic aggregation path, so SELECT …​ max(host) …​ GROUP BY and count(field) …​ GROUP BY (which skips null values) still answer. Only count(*) is answered by the push-down.

Aggregation from embedded code goes through TimeSeriesEngine.aggregateMulti, which reads any number of columns in a single pass:

A bucket interval of 0 aggregates the whole range as one bucket, answering every sealed block wholly inside it from its statistics. The overloads of aggregateMulti, aggregateGrouped and iterateQuery that take a FieldFilter read only the samples that pass range conditions on numeric fields, judged on each block’s statistics before it is read. A condition is built with FieldFilter.range, which returns null for a comparison it cannot evaluate exactly as SQL does (see the SQL Queries section above):

The PromQL endpoints follow the Prometheus HTTP API format and return standard {status: "success", data: {…​}} JSON responses.

Instant query:

The time parameter is Unix seconds (float). Defaults to current time if omitted.

Range query:

All timestamps in Unix seconds. The step parameter accepts a duration string (60s, 1m) or seconds as a float.

Label discovery:

No other PromQL function is implemented, among them quantile, delta(), idelta(), stddev_over_time(), sqrt(), clamp_min(), label_replace(), label_join(), absent(), histogram_quantile(), vector(), scalar() and time(). A call to one is refused with Unknown function: <name> (HTTP 400 bad_data).

The promql() SQL function calls the PromQL evaluator from within SQL queries:

The function accepts 1-2 arguments: the PromQL expression (required) and an optional evaluation timestamp in milliseconds.

Configure Prometheus to read from ArcadeDB for long-term storage:

The endpoint accepts the standard Prometheus remote_read Protobuf query (snappy-compressed) and supports =, !=, =~, !~ label matchers, applied with the same rules as the PromQL /query and /series endpoints (including the typed-tag rule in the Vector selectors row of the table above). A name matcher that is not an equality (for example {name=~"cpu.*"}) selects every metric it matches among those the user can read. An invalid or unsafe regular expression is refused with HTTP 400 rather than answered with an empty result (since v26.10.2).

(Since v26.10.2) Like remote write, the endpoint reads the proto= parameter of the Content-Type header and answers 415 when it names a message other than prometheus.ReadRequest, and a malformed protobuf body is answered 400 instead of 500.

Downsampling replaces samples rather than deleting them, so a read already in flight when a pass runs cannot finish coherently: the rows it has already returned are the original samples, and the rows still to come are their coarser replacements. Joining the two would report the bucket at the crossing point at two different resolutions.

Such a read is therefore refused rather than allowed to return a short answer:

Re-running the read succeeds and returns a complete answer at a single resolution, because downsampling is a scheduled maintenance event rather than something that happens per request.

A retention pass is different. It deletes old data outright, so a read that crosses one still completes and simply returns the rows that remain.

(Since v26.10.2) An unbounded read streams too: a SQL SELECT from a TIMESERIES type with no LIMIT in ascending order, the gRPC time-series stream without a row limit, and TimeSeriesEngine.iterateQuery in Java decode one sealed block per shard at a time instead of loading the whole range before returning the first row, so their memory no longer grows with the range (a 50-million-point range used to exhaust a 4 GB heap). They therefore follow the rules above: a block a retention pass deletes mid-read is skipped, and a downsampling pass mid-read refuses the read with the retryable error. Earlier versions held the type’s compaction lock while loading the range, so neither pass could change what such a read returned.

The EXPORT DATABASE summary carries two counters for TIMESERIES data that changed underneath it:

CHECK DATABASE DEEP adds the checks that have to decompress the blocks. A CRC proves the bytes are the bytes that were written; it proves nothing about whether they mean what the block claims. Three things the read paths answer queries from, without ever looking at a value, are verified only here:

Each of those produces a wrong answer rather than an error, which is why nothing else in the engine reports them.

FIX repairs only what is derived from the data, and never a sample:

None of those is cosmetic. The global bounds are read straight out of the header when the file is opened rather than recomputed, so a range query pruned against a wrong bound silently misses data the file holds; and because new blocks are appended at the end of the file, a tail no reader can see also hides every block appended after it.

A sealed block that fails its CRC, or that DEEP finds inconsistent, is reported and left untouched. It is the only copy of the samples in it, so discarding it is an operator’s decision - and in an HA cluster the sealed store is derived, so a node can rebuild one by recompacting from its replicated mutable pages.

The report carries totalTimeSeriesTypes, totalTimeSeriesShards, totalTimeSeriesSamples and totalTimeSeriesSealedBlocks alongside the findings, so a clean result can be told apart from a pass that never ran. Findings appear in warnings with the offending type named in corruptedTimeSeries; repairs are counted in repairedTimeSeries and listed in timeSeriesRepairs.