Basic Concepts
Basic Concepts
1. Common Concepts in Time-Series Databases
This section introduces common basic concepts in the time-series database field, including time-series data, time series, devices, timeseries, data points, sampling frequency, TTL, metadata, encoding, and compression.
1.1 Time-Series Data
In scenarios such as IoT, industrial production, energy and power, connected vehicles, and infrastructure monitoring, devices usually use sensors to continuously collect status data about themselves or their environment. For example, a motor collects voltage and current, a wind turbine collects blade rotation speed, angular velocity, and generated power, a vehicle collects longitude, latitude, speed, and fuel consumption, and a bridge collects vibration frequency, deflection, and displacement.
The common feature of this type of data is that it is time-related: the same collection object continuously produces new records as time passes. Data that is continuously produced and recorded in chronological order is called time-series data.
1.2 Time Series
In time-series data scenarios, a collection point continuously produces data points over time. After these data points are arranged in ascending timestamp order, they form a time series. In tabular form, a time series can be represented as a data table composed of time and value. In graphical form, a time series can be represented as a trend curve that changes over time, and can also be described figuratively as a device's "electrocardiogram".
1.3 Device
A device, also called an entity or equipment, is a device or apparatus that has physical quantities in a real-world scenario. It can be a physical device, a measurement apparatus, or a set of sensors.
Common examples are as follows:
| Scenario | Device Example | Identifier Example |
|---|---|---|
| Energy | Wind turbine | Region, station, line, model, instance, etc. |
| Factory | Robotic arm | Unique ID generated by an IoT platform |
| Connected vehicle | Vehicle | Vehicle Identification Number (VIN) |
| Monitoring | CPU | Machine room, rack, hostname, device type, etc. |
1.4 Timeseries
A timeseries can also be called a physical quantity, time series, timeline, signal, metric, point, or measured value. It is measurement information recorded by a detection device in a real-world scenario. Usually, a physical quantity represents a collection point that can periodically collect the physical quantity of the environment or device where it is located. When the data points generated by a timeseries over time are arranged in ascending timestamp order, they form a time series.
Common examples are as follows:
| Scenario | Timeseries Example |
|---|---|
| Energy and power | Current, voltage, wind speed, rotation speed |
| Connected vehicle | Fuel level, vehicle speed, longitude, latitude |
| Factory | Temperature, humidity |
In the IoTDB tree model, the number of timeseries equals the number of leaf nodes under the entire path pattern. For the detailed counting method, see Count Timeseries.
1.5 Data Point
A data point consists of a timestamp and a value. The timestamp indicates when the data was generated, and the value indicates the collection result of the timeseries at that time. The value can be of multiple types, such as BOOLEAN, FLOAT, and INT32.
A row in a tabular time series, or a point in a trend chart, can both be understood as a data point.
1.6 Sampling Frequency
Sampling frequency refers to the number of times a physical quantity generates data within a certain period of time. For example, if a temperature sensor collects temperature data once per second, the sampling frequency is 1 Hz, that is, once per second.
The higher the sampling frequency, the more data points are generated per unit of time, and the higher the requirements for writing, storage, and query capabilities.
1.7 Data Retention Time (TTL)
TTL is used to specify the data retention time. Data beyond the TTL will be automatically deleted.
Using TTL properly can control disk space usage, avoid issues such as disks becoming full, and help maintain query performance and reduce memory resource usage.
1.8 Metadata (Schema)
Metadata is the data model information of a database and is used to describe the structure and definition of data. For the tree model, metadata usually includes information such as path hierarchy, devices, timeseries, data types, encoding methods, and compression methods.
1.9 Encoding and Compression
Encoding is a compression technique used to represent data in binary form, thereby improving storage efficiency. Compression further compresses binary data after encoding to improve storage efficiency.
For details about the encoding and compression methods supported by IoTDB, see Compression and Encoding.
2. Common IoTDB Concepts
This section introduces common concepts in the IoTDB tree model, distributed architecture, and deployment. These concepts explain how IoTDB organizes, manages, and deploys time-series data by using hierarchical paths.
2.1 Data Model Concepts
2.1.1 Data Model (sql_dialect)
IoTDB supports two data models: the tree model and the table model. Both models manage devices and timeseries as their core objects, but they organize data differently and use different syntax.
Tree model: Manages data by using hierarchical paths. One path corresponds to one timeseries of one device.
Table model: Manages data by using relational tables. One table is recommended to correspond to one type of device.
Two model spaces can exist in the same cluster instance. Different models use different syntax and database naming methods, and are not visible to each other by default.
2.1.2 Database
In the tree model, a database is a path segment prefixed with root., and can be understood as the upper management boundary for tree model data. During modeling, it is usually recommended to use only the next-level node under root as the database, such as root.db.
The parent node or child node of a database cannot be set as another database. A single database can also make full use of machine resources, so it is usually unnecessary to create multiple databases for performance reasons.
2.1.3 Time Series and Device
A time series (timeseries) is a complete path prefixed with the database path and separated by periods (.). It can contain any number of levels. Each time series can have its own data type, encoding method, and compression method.
In the tree model, the penultimate level is usually regarded as the device. For example, in root.db.turbine.device1.metric1, the device1 level is the device, and metric1 is the timeseries. A device cannot be created separately; it usually exists along with the creation of a time series.
During modeling, it is recommended to put only the labels that can uniquely locate a time series into the path, generally no more than 10 levels. Low-cardinality labels should be placed as early as possible to help the system compress common prefixes.
If there are few devices but many timeseries for each device, you can add levels such as
.valueat the end so that the penultimate-level nodes are more sufficient, for example,root.db.device01.metric.value.
2.1.4 Alias, Tag, and Attribute
When creating a time series, you can add an alias, tags, and attributes to the timeseries. An alias is bound to the timeseries and can be used equivalently in scenarios where the original timeseries name is used. A temporary alias in an SQL query only replaces the name in the current query result and is not bound to the time series.
| Concept | Purpose |
|---|---|
| Alias | Bound to a timeseries and used to access it instead of the original timeseries name |
| Tag | Can be used to query time-series paths. The system maintains an index from "tag" to "time-series path" |
| Attribute | Used to describe a time series. Attribute information can only be queried from the time-series path |
2.2 Distributed Concepts
IoTDB supports running in cluster mode. Common concepts in a cluster include nodes, Regions, and multiple replicas. A common cluster deployment pattern is 3C3D, that is, 3 ConfigNodes and 3 DataNodes.
2.2.1 Node
An IoTDB cluster includes three types of nodes: ConfigNode, DataNode, and AINode.
ConfigNode: Manages cluster node information, configuration information, user permissions, metadata, partition information, and more. It is responsible for scheduling distributed operations and load balancing. All ConfigNodes are full backups of each other.
DataNode: Serves client requests and is responsible for data storage and computation.
AINode: Provides machine learning capabilities, supports registering trained machine learning models, and invokes models for inference through SQL.
2.2.2 Data Partition (Region)
In IoTDB, metadata and data are divided into smaller partitions, called Regions, and are managed by DataNodes in the cluster.
SchemaRegion: A metadata partition used to manage the metadata of some devices and timeseries.
DataRegion: A data partition used to manage the data of some devices within a time range.
Regions with the same RegionID on different DataNodes are replicas of each other.
2.2.3 Multiple Replicas
The number of replicas for data and metadata is configurable. Multiple replicas can provide high availability.
| Category | Configuration Item | Recommended Standalone Configuration | Recommended Cluster Configuration |
|---|---|---|---|
| Metadata | schema_replication_factor | 1 | 3 |
| Data | data_replication_factor | 1 | 2 |
2.3 Deployment Concepts
IoTDB has three running modes: standalone mode, cluster mode, and dual-active mode.
2.3.1 Standalone Mode
An IoTDB standalone instance includes 1 ConfigNode and 1 DataNode, that is, 1C1D.
Feature: Easy for developers to install and deploy, with low deployment and maintenance costs and convenient operations.
Applicable scenarios: Scenarios with limited resources or low high-availability requirements, such as edge servers.
Deployment method: Standalone Deployment.
2.3.2 Dual-Active Mode
Dual-active deployment is a feature of TimechoDB Enterprise Edition. It means two independent instances synchronize with each other bidirectionally and can provide services externally at the same time. After one instance stops and restarts, the other instance resumes transferring the missing data from the breakpoint.
An IoTDB dual-active instance usually consists of two standalone nodes, that is, two sets of 1C1D. Each instance can also be a cluster.
Feature: A high-availability solution with the lowest resource usage.
Applicable scenarios: Scenarios with limited resources (only two servers) but a need for high availability.
Deployment method: Dual-Active Deployment.
2.3.3 Cluster Mode
An IoTDB cluster instance has 3 ConfigNodes and no fewer than 3 DataNodes, usually 3 DataNodes, that is, 3C3D. When some nodes fail, the remaining nodes can still provide services externally, ensuring high availability of database services. Database performance can also be improved as nodes are added.
Feature: High availability and high scalability. System performance can be improved by adding DataNodes.
Applicable scenarios: Enterprise-level application scenarios that require high availability and reliability.
Deployment method: Cluster Deployment.
2.3.4 Feature Summary
| Dimension | Standalone Mode | Dual-Active Mode | Cluster Mode |
|---|---|---|---|
| Applicable scenarios | Edge deployment and scenarios with low high-availability requirements | High-availability services, disaster recovery scenarios, etc. | High-availability services, disaster recovery scenarios, etc. |
| Number of required machines | 1 | 2 | >=3 |
| Safety and reliability | Cannot tolerate a single point of failure | High, can tolerate a single point of failure | High, can tolerate a single point of failure |
| Scalability | DataNodes can be expanded to improve performance | Each instance can be expanded as needed | DataNodes can be expanded to improve performance |
| Performance | Can scale with the number of DataNodes | Same as one of the instances | Can scale with the number of DataNodes |
The deployment steps for standalone mode and cluster mode are similar (ConfigNodes and DataNodes are added one by one). The only differences are the number of replicas and the minimum number of nodes that can provide services.