Do you want to install ChatGPT on your computer? The domestic open source large language model ChatGLM helps you realize it!

Hello, everyone.

Today I would like to share with you an open source large language model ChatGLM-6B.

# Within ten days, I gained nearly 10,000 stars.

ChatGLM-6B is an open source conversational language model that supports Chinese and English bilinguals. It is based on the General Language Model (GLM) architecture and has 6.2 billion parameters. Combined with model quantization technology, users can deploy it locally on consumer-grade graphics cards (a minimum of 6GB of video memory is required at the INT4 quantization level). ChatGLM-6B uses technology similar to ChatGPT and is optimized for Chinese question and answer and dialogue. After bilingual training in Chinese and English with about 1T identifiers, supplemented by supervised fine-tuning, feedback self-service, human feedback reinforcement learning and other technologies, the 6.2 billion parameter ChatGLM-6B has been able to generate answers that are quite consistent with human preferences.

Everyone can install it on their own computers and try it out. The minimum video memory of independent graphics is 6G, and a CPU computer can also run it, but it is very slow.

The project currently only open-sources the model and inference code, but does not open-source the training of the model.

To run the project, you only need two steps

First step, download the source code

git clone https://github.com/THUDM/ChatGLM-6B.git

Execute pip install -r requirements.txt to install dependencies

Second step Step, run the project

python web_demo.py

After execution, the model file will be automatically downloaded, about 4G.

If it is running on GPU, by default, the model running video memory is at least 13G. If the video memory is not enough, you can modify web_demo.py

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()

Change the THUDM/chatglm-6b of the above code It is THUDM/chatglm-6b-int4​, that is, using the int4 quantized model. As long as the video memory is greater than 6g, it can run smoothly.

If it reports insufficient CPU memory, turn off other software, especially the browser.

After successful operation, it will automatically jump to the browser page, and then you can use it like ChatGPT.

The following is the result of my local operation. You can see the difference with ChatGPT

Self-awareness

Write an outline

Write an email

The above is the detailed content of Do you want to install ChatGPT on your computer? The domestic open source large language model ChatGLM helps you realize it!. For more information, please follow other related articles on the PHP Chinese website!