AI master Li Mu's installation video is here! You can also practice 10 billion large models

Before the installation video was released, Teacher Li Mu once launched a small questionnaire survey. Taking advantage of the price reduction of graphics cards, let’s see what children think about the installation video. How interested are you in running Transformer?

At that time, even Huawei’s talented young man "Zhihui Jun" came to like it, which shows that everyone is still looking forward to it.

No, Mu Shen is here with his installation video. How to train a 10 billion model at the lowest cost?

Just recently, the currency circle has been cooling down, and GPU prices have also dropped significantly. For example, the Nvidia 3090TI is now priced at US$1,600 (original price USD 2,000).

At the beginning of this project, 2 prototype machines were installed, each machine was a dual-card RTX 3090TI , using a water cooling system to reduce noise.

#The cost of installing a machine is more than 5,000 US dollars, about 35,000 yuan.

Without further ado, let’s take a look at how Mu God installed the machine~

Installation list

The requirements for installation are first to be quiet enough. Otherwise, it will be too noisy and you won’t be able to work.

#The second requirement is good heat dissipation. Otherwise, if the temperature is too high, it will cause the GPU to underclock.

The third point is, because we need to run a relatively large Transformer model, the bandwidth of the GPU must be good enough.

If you have installed a GPU server before to run CNN, the requirements for running the Transformer model will be different. Because the Transformer model is larger than the CNN model, the memory usage will be higher. So the memory size of the GPU is very important.

Mu Shen also said before that such a large Transformer model should be trained on multiple GPUs. From Engineers from Google, Microsoft, etc. all use machines like DGA X100 to run. Even on such machines, the bandwidth of the GPU remains a bottleneck.

The difference between buying this kind of server GPU and game GPU is that the former is not about how fast a single card can run, but how many cards can run between cards. Connect quickly.

Therefore, the focus of the installation concept is: try to increase the GPU memory and the bandwidth of the interconnection between GPUs,

If you want to put a lot of cards in a machine, you need to buy a turbine cooling system.

#If you want quietness, buy a water-cooled heat sink. Mu Shen bought 4 yuan of 3090 TI. The advantage of using water cooling is that it is relatively quiet, but the disadvantage is that it takes up a lot of space.

#So, if you want to put four cards in the chassis, don’t buy the water-cooled version, but buy the version with only one turbo fan.

##And the direction of the wind in the chassis is a particularly important issue. If you buy a card with three fans, the air will enter the chassis from the front and then dissipate heat from all directions. If the cards are too close together, the temperature inside the chassis will be very high.

Mu Shen also said that many years ago, he bought four cards with two large fans and put them together. As a result, the temperature of one card was too high. Burned.

After selecting the GPU (ASUS ROG), the remaining configuration is relatively simple. The CPU is a 12-core AMD CPU, the motherboard is a brand called PCIE 4.0 16, the hard drive is a 2 TB M.2 hard drive, the fan is a 120mm water-cooled fan, and a full-size chassis is added.

Installation Steps

After the installation list is completed, the next step is the specific installation process. The steps are as follows:

#First put the GPU. Note that you must not touch metal places with your hands during the placement process. If there is static electricity, it will easily cause the GPU to conduct electricity.

After putting the GPU in, tighten the screws. Then put the fan in.

# After plugging in the power, tie the power cord and water pipe together. Then connect the NVLink bridge.

Finally connect the power supply and the machine is ready to run.

Stress Test

After the installation is completed, the next task is to continue to install the operating system .

Mu Shen installed ubuntu22. After installing it, he connected remotely.

Of course, Mu Shen also explained various situations in detail. In addition to ubuntu22, windows and linux are also available under different needs.

#Here Mu Shen uses SSH for remote connection.

##Mu Shen’s system has already installed the driver. At the same time, he also pointed out that if there is no driver yet, you can Use apt-get to install nvidia-driver-515.

After installation, you can run nvidia-smi and see the system.

You can see various information from it. Such as the number of GPUs, temperature, wattage, memory usage, etc.

Next, you can also see whether the nv-link is normal through the topo-m matrix of nvidia-smi.

You can see that the two GPUs are connected by NV4. 4 means 4 channels, which means the connection is normal.

#The next question is to test the temperature of the system under full load.

Mu Shen said that the GPU is tested with a small program called gpu-burn, which can be downloaded from github.

Here Mu Shen simulated running for ten minutes, and also saw the temperatures of the two GPUs. Mu Shen also joked that you can feel the hot air blowing from the GPU.

Similarly, the CPU can also use this method to test the temperature, using cpu-burn.

In the end, the temperatures of the two GPUs stayed at 58 degrees and 55 degrees, and the power consumption reached more than 440 watts (full power consumption 480 watts), which is pretty good. of.

#The last parameter is the power consumption of the machine. Mu Shen's test used about 1240 watts, which means 1.5 kilowatt hours of electricity per hour.

Judging from the current data, the stability is OK.

#As for the performance of running Transformer on this machine, we have to wait for the next video.

Hot comments from netizens

After the video was released, netizens at Station B also expressed great interest.

There is a representative student from the perfect score class who appears and lists the complete configuration list mentioned in the video.

Some netizens rushed to watch, "Learn to install the machine from Li Mu."

## Mu Shen himself expressed that he felt that the 3090ti card was not very good. A netizen immediately commented, "If it doesn't work, just smoke it and give it away."

Of course, this kind of hard-core installation There is definitely an element of humor in the comments under the video.

I can only say that it is too true.

The above is the detailed content of AI master Li Mu's installation video is here! You can also practice 10 billion large models. For more information, please follow other related articles on the PHP Chinese website!