- chatGLM-6B official
代码: https://github.com/THUDM/ChatGLM-6B - Official
博客: https://chatglm.cn/blog - paper:
- 2210.GLM-130B: An Open Bilingual Pre-trained Model (Open source bilingual pre-trained model)
- 2103.GLM: General Language Model Pretraining with Autoregressive Blank Infilling (General Language Model Pretraining with Autoregressive Cover Filling)
- In the autumn leaves version UI, the model may lag:
ChatGLM-6B, combined with model quantization technology, users can deploy locally on consumer-grade graphics cards (only 6GB of video memory is required at the INT4 quantization level). After about 1T of Chinese and English bilingual training of identifiers, supplemented by the blessing of technologies such as supervision and fine-tuning, feedback self-help, and human feedback reinforcement learning, although the scale of62 亿ChatGLM-6B with parameters is not as large as the 100 billion model, it greatly reduces the threshold for user deployment, and It has been able to generate answers that are quite in line with human preferences.
Environment installation
Mainly combined with the official code and Qiuye github to see each other
Create a virtual environment
conda create -n chatglm python=3.10 -y
Add domestic source
conda activate chatglm
pip config set global.index-url https://mirror.baidu.com/pypi/simple
pip config set global.trusted-host mirror.baidu.com
python -m pip install --upgrade pip
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
requirementsam arranges dependencies
It is recommended 删除that torch>=1.10 in the official dependencies, otherwise torch2.0 will be automatically installed, resulting in unsuccessful environment installation
official
git clone https://github.com/THUDM/ChatGLM-6B
cd ChatGLM-6B
pip install -r requirements.txt
autumn leaves
git clonehttps://github.com/Akegarasu/ChatGLM-webui
cd ChatGLM-webui
pip install -r requirements.txt
Start (default 12G video memory)
The official website code starts directly (by default, it can only be opened locally)
https://github.com/THUDM/ChatGLM-6B/blob/main/web_demo.py
python web_demo.py # 可自动模型
It took about 50 minutes to download.
Modify the code, you can access any ip
The right side of the picture above is the source code
demo.queue().launch(
server_name="0.0.0.0",
share=False,
inbrowser=True)
Find the automatic model download location
The linux command, the first parameter is the search path, and -name is followed by a suffix.
find ~/.cache/huggingface -name pytorch_model-00001-of-00008.bin
Find the location of the model to facilitate subsequent management
~/.cache/huggingface/hub/models–THUDM–chatglm-6b/snapshots/cde457b39fe0670b10dd293909aab17387ea2c80/pytorch_model-00001-of-00008.bin
Autumn Leaf launch (ui may be better, but the model will be a bit older and needs to be updated by itself)
The precision can be changed to int8,
int4 and int8 will sometimes report an error
–listen means non-local access, enter the server ip
python webui.py --precision fp16 --model-path "./model/chatglm-6b" --listen
It will be a little stuck, there is no chatgpt typewriter effect, maybe it will be updated after the update
use
Here are a few questions you can ask me in different areas: 1. Artificial intelligence: what are the application areas of artificial intelligence? How can it change our life and work? 2. Health and fitness: how to maintain a healthy
lifestyle? What are some fitness tips that can help us better cope with stress and stay fit?
3. Computer Science: How to write efficient code? In which areas are machine learning and deep learning most widely used?
4. Business and Marketing: How To succeed in a highly competitive market? What marketing strategies can help us attract more customers? 5.
History and culture: How to understand the history and traditions of different cultures? What historical events and cultural heritage are worthy of our deep understanding and appreciation?
请写一个介绍chatglm的博客提纲
output result
Addendum: LLM Ragnarok
from : A Survey of Large Language Models
memory size
