In the industrial field, a large amount of time-stamped sensor data may be generated in production, testing, and operation stages, which are typical time-series data. Time-series data is mainly collected or generated by various types of real-time monitoring, inspection and analysis equipment, involving manufacturing, electric power, chemical industry, engineering operations and other industries, with typical characteristics such as more writes and less reads, and very large volumes. Databases commonly used by Internet companies such as Apache HBase and MySQL have exposed many problems in writing, storage, query, operation and maintenance, etc. In this case, from the perspective of business development, data architecture transformation has become a top priority.
This article summarizes the structural transformation cases of four representative industrial enterprises including Siemens, Midea, Topstar, and Hollysys. Let’s take a look at how they did it and whether the transformation effect has reached expectations.
Siemens x TDengine
"Starting from the goals of high performance, high availability, low cost, and high integration, we found that TDengine just meets all the requirements for product reconstruction, especially the two points of low cost and high integration, which are currently most Neither the data platform nor the time series database is available. After deciding to choose TDengine as the system database, we removed Flink, Kafka and Redis in SIMICAS ® OEM 2.0, and the system architecture was greatly simplified."
business background
SIMICAS® OEM equipment remote operation and maintenance suite is a set of digital solutions for equipment manufacturers developed by SIEMENS DE&DS DSM team. In its version 1.0, the team used the architecture of Flink + Kafka + PostgreSQL + Redis. Because of the introduction of Flink and Kafka, the system deployment is very cumbersome and the server overhead is huge; at the same time, in order to meet the storage problem of a large amount of data, PostgreSQL has to The application program is more complicated for sub-database and sub-table operations. In this case, how to reduce system complexity, reduce hardware resource overhead, and help customers reduce costs has become the core task of the R&D team. During the research, TDengine stood out.
architecture diagram
Click on the case to view more technical details
Midea x TDengine
"Currently, TDengine is mainly used in the monitoring business of central air-conditioning and refrigeration equipment. As a pilot project, this scenario has achieved good results. In terms of building intelligence, we also have a lot of work to do, from edge monitoring, We will continue to explore and tap the potential of TDengine in these scenarios, from command control to edge-cloud collaboration integrated services.”
business background
At the 2021 Building Technology TRUE Conference, Midea's HVAC and Building Division released the digital platform iBuilding for the first time, empowering the construction industry with a "soft drive and hard core" approach. As a brand new project, iBuilding is more cautious in database selection, comparing relational databases (Relational Database) and mainstream time series databases (Time Series Database), including InfluxDB, TDengine, MySQL, etc., because they are more biased in terms of requirements For efficient storage and data fetching in a wide range of time, iBuilding finally chose TDengine after comprehensively evaluating the comprehensive capabilities of adaptation, query, writing, and storage.
architecture diagram
Click on the case to view more technical details
Topstar x TDengine
"After running for a period of time, the query and writing speed of TDengine can fully meet the needs of our current customers. The slowest is at the minute level, and the fastest can reach 1 second; a device can write up to nearly 100,000 pieces of data a day. There is no problem with nearly a thousand devices writing at the same time. Compared with before, the writing speed has increased by dozens of times. There is no obvious delay in the query data within the time range of months. The overall data compression ratio is about It is 1/10, and the amount of data generated every day is about several gigabytes."
business background
在拓斯达的业务中，传统的关系型数据库已经无法高效处理时序数据，在加载、存储和查询等多个方面都遇到了挑战，主要问题包括写入吞吐低、存储成本大、维护成本高、查询性能差。为了更好地满足时序数据的处理需求，拓斯达开始进行数据库选型调研，他们发现，TDengine 专为时序数据库所打造和优化的写入、存储、查询等功能，非常匹配工业传感器数据的应用分析场景，最终其使用 TDengine 搭建了新的数据处理架构。
架构实现思路
通过网关采集设备数据推送到 MQTT，Java 后端监听到后会写入 TDengine，在后端按需求查询处理后再把数据返回给前端。具体来说，网关会先读取后台发布的上行规则，在采集到设备数据后，使用上行规则对数据进行处理计算后再将结果返回给下行规则模块，后台监听到后，会连接 TDengine 进行数据库表的创建修改和数据写入。之前在云平台拓斯达使用过 Kafka 进行数据的发布订阅，现在所有环境都改为 MQTT 了。
点击案例查看更多技术细节
和利时 x TDengine
“在测试阶段，我们发现，同等条件下，TDengine 的压缩率最高，数据占用的存储空间最小；在原始数据查询上，OpenTSDB 最慢，TDengine 与 HolliTSDB 在伯仲之间；在聚合查询操作上，TDengine 最快，HolliTSDB 的速度和 InfluxDB 相当，OpenTSDB 最慢。同时，InfluxDB 只能单机部署，集群版本并未开源，且查询性能存在瓶颈，其 QPS 约为 30-50。”
业务背景
在物联网场景下，面对庞大的时序数据处理需求，Oracle、PostgreSQL 等传统关系型数据库越来越吃力，因此和利时开始进行时序数据库的选型，对包括 InfluxDB、OpenTSDB、HolliTSDB（和利时自研时序数据库）和 TDengine 在内的四款时序数据库进行了选型调研及相关测试。测试结果显示，在同等条件下，TDengine 在查询、存储等方面均优于其他几款数据库，最终和利时决定接入 TDengine，以享受更多元的本地化支持和响应。
架构图
点击案例查看更多技术细节
结语
从以上案例中不难看出，在工业互联网场景下，面对庞大的时序数据处理需求，专业的时序数据库显然比传统的关系型数据库效果更加明显，上述企业案例在架构改造之后，确实达到了更高程度的降本增效。如果你有同样的困扰，欢迎添加小T微信，可以邀请你进入TDengine 用户交流群，和专业的解决方案架构师点对点沟通。
想了解更多TDengine Database的具体细节，欢迎大家在GitHub上查看相关源代码。