怎么实现LLM并行训练和推理:device_map=‘auto‘

怎么实现LLM并行训练和推理:device_map=‘auto’

model = AutoModelForCausalLM.from_pretrained(model_path, device_map=‘auto’)
在这里插入图片描述

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer


# 1. 加载分词器与模型(确保路径正确)
# model_path =