DRDS Application Development Guide

Recently, according to the situation of using Alibaba Cloud DRDS ( https://www.aliyun.com/product/drds ) in multiple applications, I have made a relatively introductory guide here. Because there are many documents, it is unrealistic to read them all, but using Well, there are several points in this product that still need to be very clear, otherwise it may be pitted.

At the beginning of everything, you need to understand what this thing does. It is recommended to read the overview of this product https://help.aliyun.com/document_detail/29659.html and the basic principle of the product https://help.aliyun.com/ document_detail/29679.html

confirms that this product can solve your core problems, and the next important point is how to use this product well. I personally think that database products all have pits, but the pits are big and small, and whether they can be used from The problem of crawling out of the pit.

No matter how you drop it, just get started and take a look. The Quick Start series of documents are more appropriate https://help.aliyun.com/document_detail/50067.html

After you connect to DRDS according to the steps and do some operations, in fact, you It may not be clear what happened. If you go to test the business table according to your understanding at this time, you may be buried in a hole. There is a very important difference between the table structure design of the DRDS product and the stand-alone database: how to disassemble it. So at this time, I recommend everyone to read the document https://help.aliyun.com/document_detail/51309.html and the video https://yq.aliyun.com/video/play/981 . I think this time will be spent is worth it.

After you understand the most important points of this product, the next step is actually a process of initializing the test environment, importing data or creating data. You must be very concerned about resources and how to match DRDS and RDS. According to personal experience, DRDS can Start with the minimum specification, because I understand that the product itself has no status, and it is more convenient to upgrade like an application expansion machine. For RDS, one or two test environments can be used. For production, please refer to https://help.aliyun.com/ document_detail/51307.html This document, but there is one point that may not be mentioned in the document, you need to consider the problem of data capacity, because a single RDS currently has a maximum of 2T, in fact, it is not a hardware problem, but you need to make a backup, make a Read-only instance, do a disaster recovery, the time is measured in days, so I personally recommend that unless you want to live in an old RDS instance, if the program can be changed to split, the amount of data in a single RDS does not exceed 1T as well. As a very simple example, you have 10T data, I personally recommend using 10-15 RDS to solve the problem, not 5-6.

In addition, the most important point is that if you want to avoid some production troubles in the future, it is recommended to choose an RDS node of 8c16g (inclusive) or above, and a distributed database built with a bunch of diaosi instances is far less stable than a larger size. instance. When you have planned the environment, in the process of functional development, it is recommended to read most of the documentation of this product, and it is recommended to read this post https://bbs.aliyun.com/read/313990.html

, at first glance It's shocking to see, but there are not many products that can tell you about the pit, and understanding these contents will help the team to solve problems encountered in development.

After the function development is completed, many applications may be launched directly. I personally recommend adding performance tests, especially for core scenarios. The distributed database currently available on the market, even if the functional compatibility is good, without performance testing, it is very unreliable, and it is likely to lead to the failure of the release, because some physical problems cannot be overcome (according to the current large The architecture and hardware of some products), so you will import some data into DRDS at this time, you can refer to https://help.aliyun.com/document_detail/52131.html .

Before the actual production use, you should also be very concerned about the alarm problem. Although this product provides monitoring alarms, I personally think that the monitoring and alarm indicators are not rich enough, only CPU, and the aggregation granularity of monitoring indicators is relatively large (5 minutes). Introduction You can take a look at https://help.aliyun.com/document_detail/52009.html , but it is very interesting that he provides a second-level statistics command so that we can play, see the description of this command https://help .aliyun.com/document_detail/52139.html , I also wrote an article for this purpose using grafana to monitor this product in seconds, see the document http://bucketli.iteye.com/blog/2397401 , is generally valid, including ERROR_PER_SECOND, FULLGCCOUNT, CPU are all indicators worthy of attention, if ERROR_PER_SECOND has a value, your application is abnormal (excluding VIOLATION_PER_SECOND), if FULLGCCOUNT is constantly increasing (changes in a few seconds) Once), it is very likely that there is a problem with SQL. When the CPU is always above 80%, one may be that there is a large amount of data import and export or data processing, and the other may be that the application traffic is too large, etc. I will write an article in the future. After analyzing these indicators, this command is very good as a whole.

When you pass the performance test or actually launch the function, you will find that the applicable scenarios of DRDS may not cover all your scenarios. From the perspective of the Yunqi Conference, this product is more suitable for the front-end data business (of course, if you can put the This kind of business can be solved even better if the global secondary index is done), and the background business may encounter some problems. Therefore, you will need to synchronize data to other databases (including stand-alone databases) to do some business such as report analysis. Here we recommend the cloud data integration tool (CDP, the open source product is called datax), see their use for DRDS Documentation https://github.com/alibaba/DataX/blob/master/drdsreader/doc/drdsreader.md   and https://github.com/alibaba/DataX/blob/master/drdswriter/doc/drdswriter.md , the former It extracts data from DRDS, and the latter writes data to DRDS. However, this tool does not have incremental pain. Because DRDS is based on RDS, it is actually possible to open multiple binlog synchronization tasks for data synchronization, which is recommended here. https://www.aliyun.com/product/dts on the cloud Products, of course, if you don't want to pay and want to toss yourself, you can also use https://github.com/alibaba/canal products. For data import and export, and synchronization, I personally think that it is not good enough. When can a product that kills everything be produced? It is worth looking forward to.

The overall user guide is introduced here. If you have a successful use of DRDS products, I believe that it will be relatively cool, at least personally think so.