1. Scenario requirements

After online encryption covers all users, the next one The important step is to encrypt all historical data.

We summarized the RDS migration plan and performance data based on internal stress test data and case experience for reference.

2 Project Introduction

##2.1 Data migration, preparation:

1) Please confirm that the code logic is compatible with plaintext and ciphertext.

2) Confirm that the encryption switch is turned on.

3) Confirm that the newly incoming data from RDS and TOP api is already ciphertext.

2.2Data migration recommended plan

After confirming fault tolerance Migrate after the logic has been completed. After first conducting a small number of customer migration tests, you can then conduct a full migration test.

1) First, single customer data migration test:

Take the opportunity to migrate one customer to test the migration plan. Recommended migration amount <1 million. You can start with a lower concurrency number. For example: 5-10 threads concurrently. (You can choose based on the above stress test data)

Record usual performance in advance;

Record migration During the period, record the migration time, IOPS/connection/CPU/TPS/QPS, etc. during the migration process, and compare it with the usual performance.

2)

Migration plan

Calculate the migration time and plan based on the previously calculated migration speed.

First determine the minimum granularity of migration: for example, for an independently deployed ISV, you can decide a deployment as a minimum granularity. Take into account the independence of functions and the total duration of migration (control it within 3 hours).

When migrating on the first day, continue to record the data volume, time...etc.

If the migration process goes smoothly without problems and the performance load is bearable, you can gradually increase the number of threads later.

After that, you can gradually increase the number of threads and the number of concurrent migrations based on the first day. Pay attention to continuous monitoring of database performance. until the migration is complete.

3) Note on code logic:

• Please note: only when the user Session is valid, or expires no more than 90 days, and the user The key can be retrieved only if the status has not expired.

During the migration process, Please exclude users that have expired for more than 90 days and are invalid ( That is: freezing, clearing, etc.).

Otherwise it will cause a large number of failed key requests and encryption will fail.

• When pulling user data from the database and encrypting it, please pay attention to controlling the paging size.

2.3 Data migration, key data reference:

Goal: Recommend an appropriate data migration plan for ISV based on database size, database and application architecture, and selected database performance.

Information and data:

RDS push database performance: http://cloud.tmall.com/rdsSelection .htm

The above picture shows the database configuration list. For our internal stress test, we selected the four RDS configurations in the red box for testing. See the next section for test results.

Stress test results:

1 ) Medium-sized database (IOPS = 1200) stress test: 4 machines were used and three tests were conducted. 10, 15 and 25 threads were enabled respectively. 2 million data were migrated. The migration time and performance during the migration process are as follows: ：

Data migration time:

##31min##Test 3

	#Number of machines	Number of threads	Data volume	Time
Test 1	##4	10	200w
Test 2 ##4	15	200w	19 min
4	25	##200w	13 min	Migration performance: ##CPU# #Test1##30004500##Test 34 ##Number of machines Number of threads ##IOPS ##Active Number of connections Peak ##Average transactions per second (TPS) ( Peak) Every SQL seconds (QPS) ( Peak) ##4 ##10 27 20 12% Test 2 ##4 15 40 45 20% ##5500 7500 25 ##40 75 32% 7500 10000 2) Large database (IOPS = 3000) stress test: 4 machines were used, three tests were conducted, and 10, 15 and 25 threads were enabled respectively. , migrating 2 million data, the migration time and performance during the migration process are as follows: Data migration time ： 4Test 3 ##Number of machines Number of threads Amount of data Time Test 1 4 10 ##200w 31 min ##Test 2 ##15 200w ##19 min 4 ##25 200w 13 min Migration performance: ##CPU# #Test1##33004500##Test 34 ##Number of machines Number of threads ##IOPS ##Active Number of connections Peak ##Average transactions per second (TPS) ( Peak) Every SQL seconds (QPS) ( Peak) ##4 ##10 30 20 10% Test 2 ##4 15 40 45 18% ##6000 7500 25 ##112 60 27% 8000 ##11000 Note: Database IOPS performance consumption is low because the database itself contains cache buffering logic. Therefore, the final migration time of databases with different IOPS performance is not much different. FAQ There is no FAQ about this document ← Cipher text field retrieval scheme API field encryption documentation → submit php.cn HomeCourseArticleQ&ADictionaryManualDownloadsearch APPDownload