Home>Article>Database> About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

藏色散人
藏色散人 forward
2020-04-22 13:18:01 5756browse

Foreword:

I have also been playing with databases recently, and I feel that ordinary machines are still a little behind when it comes to databases. Full-text search can take more than a minute if it is slow.

But the cmd5 library is very good, a billion-level database, millisecond level.

Okay, let’s start. First, you need a mysql database.

Environment:

apmserv5.2.6 php+mysql Navicat for MySQL

Recommended these two software, the installation is very simple, both are in Chinese, so it is easy for novices to operate.

Other things needed are a computer and about 10GB of hard drive space. A larger dictionary.

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

# Let’s start the first part, open the Navicat for MySQL connection, it’s very simple, I won’t take a screenshot here.

Recommendation: "mysql video tutorial"

Start creating the table, hash_cmd5, field, plaintext, cmd5_16, cmd5_32

are all set to not empty , the first is 255, the second is 16, and the third is 32 bits.

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

#Start importing data. It’s very simple. Just refer to the picture and figure it out yourself.

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

#Refer to your own database source and set the encoding

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

Refer to your own database content.

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

The fourth and fifth parts can be skipped directly.

Start the sixth step, you only need to set plaintext, other fields do not need to be set, but the prerequisite must be to select the plaintext found in the database, otherwise it will be meaningless.

About building billion-level cmd5 database with mysql and millisecond-level query (complete process)

#Go directly to step 8 to import data.

Let’s try it yourself. The next step is to teach you how to batch generate cmd16 32-bit.

High-speed single table import

If: your text file is in d:\aa.txt

Table name: t

Field name: c

is the following command

mysql> load data local infile 'd:/aa.txt' into table cmd5 lines terminated by ',' (cmd5_txt);

Create 32 16-bit md5 with one click

update `hash` set cmd5_16 = substr(md5(plaintext), 1, 16) where 1=1 update `hash` set cmd5_32 = md5(plaintext) where 1=1

As for some experts, this The method is unstable and so on. When tested on a single machine, there is no pressure on a single table of 600 million.

The above is the detailed content of About building billion-level cmd5 database with mysql and millisecond-level query (complete process). For more information, please follow other related articles on the PHP Chinese website!

Statement:
This article is reproduced at:cnblogs.com. If there is any infringement, please contact admin@php.cn delete