torchtune 端到端工作流Llama-3.2-3B 的 LoRA 微调、评测、量化与部署实战【免费下载链接】torchtunePyTorch native post-training library项目地址: https://gitcode.com/GitHub_Trending/to/torchtune本文基于 torchtune 官方教程End-to-End Workflow with torchtunedocs/source/tutorials/e2e_flow.rst完整串联下载模型 → LoRA 微调 → EleutherAI 评测 → 生成推理 → torchao 量化 → 接入 Hugging Face / vLLM 生态的全链路操作并结合仓库源码解释tuneCLI 子命令、checkpoint 目录结构与 state-dict 不变性设计帮助你掌握一套可直接复制运行的 LLM 后训练闭环方案。一、工作流总览torchtune 中的 Recipe 生态torchtune 是一个 PyTorch 原生的后训练库其核心抽象是recipe配方 config配置每个 recipe 是一个可复现的训练/推理脚本每个 config 是驱动它的 YAML 参数文件。这套体系由一个静态注册表维护所有 CLI 可见的 recipe 及其配置都登记在 torchtune/_recipe_registry.py 中。从注册表可以确认除lora_finetune_single_device、full_finetune_distributed等微调 recipe 外torchtune 还内置了eleuther_eval基于 EleutherAI 评测框架的模型评测recipes/eleuther_eval.pygenerate文本生成recipes/generate.pyquantize模型量化recipes/quantize.pyknowledge_distillation_single_device/knowledge_distillation_distributed知识蒸馏qat_single_device、qat_distributed量化感知训练dev/grpo_full_finetune_distributed等 RL 类 recipe。本教程以Llama-3.2-3B-Instruct为例演示如何把上述 recipe 串成一条完整流水线。前置要求是熟悉 torchtune 的整体概览、完成安装并理解 configs 配置机制 与 checkpointer 检查点设计。二、第一步用tune download下载模型先用tuneCLI 从 Hugging Face Hub 下载 Llama-3.2-3B-Instruct 到本地文件系统。Hugging Face 仓库里同时上传了原始权重consolidated.00.pth和兼容from_pretrained()API 的*.safetensors权重两者只需其一因此用--ignore-patterns跳过原始权重$ tune download meta-llama/Llama-3.2-3B-Instruct --ignore-patterns original/consolidated.00.pth Successfully downloaded model repo and wrote to the following locations: /tmp/Llama-3.2-3B-Instruct/.cache /tmp/Llama-3.2-3B-Instruct/.gitattributes /tmp/Llama-3.2-3B-Instruct/LICENSE.txt /tmp/Llama-3.2-3B-Instruct/README.md /tmp/Llama-3.2-3B-Instruct/USE_POLICY.md /tmp/Llama-3.2-3B-Instruct/config.json /tmp/Llama-3.2-3B-Instruct/generation_config.json /tmp/Llama-3.2-3B-Instruct/model-00001-of-00002.safetensors ...从源码看torchtune/_cli/download.py 中的Download子命令实现了该逻辑Hugging Face 路径下调用huggingface_hub.snapshot_download并透传ignore_patterns支持--output-dir默认为/tmp/model_name、--hf-token读取HF_TOKEN环境变量访问门控仓库必需它也支持--source kaggle从 Kaggle Model Hub 拉取需要--kaggle-username与--kaggle-api-key。一个容易被忽略的细节是下载完成后CLI 会把repo_id写入original_repo_id.json见 download.py 中的REPO_ID_FNAME逻辑以便保存 LoRA adapter 时把基座仓库标识写入 adapter 配置——这正是后续用 PEFT 加载 adapter 时的必要元数据。三、第二步LoRA 微调3.1 为什么选 LoRA本例使用 LoRA 微调。LoRA 是一种参数高效微调技术冻结基座 LLM只引入极少量可学习的低秩参数从而把梯度和优化器状态相关的显存压到很低。使用 torchtune在 RTX 3090/4090 上以 bfloat16 微调 Llama-3.2-3B-Instruct显存可控制在 16GB 以内。LoRA 的深入用法可参考官方 LoRA 教程。3.2 用tune ls找到合适的 configtune ls会打印所有内置 recipe 与配置的对照表。其实现非常直接torchtune/_cli/ls.py 遍历get_all_recipes()即 torchtune/_recipe_registry.py 中的_ALL_RECIPES逐行打印 recipe 名与配置名例如$ tune ls RECIPE CONFIG full_finetune_single_device llama2/7B_full_low_memory llama3/8B_full_single_device llama3_1/8B_full_single_device llama3_2/1B_full_single_device llama3_2/3B_full_single_device mistral/7B_full_low_memory phi3/mini_full_low_memory qwen2/7B_full_single_device ... full_finetune_distributed llama2/7B_full llama2/13B_full llama3/8B_full ... lora_finetune_single_device llama2/7B_lora_single_device llama2/7B_qlora_single_device llama3/8B_lora_single_device ...在注册表中lora_finetune_single_device对应 recipes/lora_finetune_single_device.py其配置列表中包含llama3_2/3B_lora_single_device——即本教程使用的配置。3.3 启动训练使用默认配置 recipes/configs/llama3_2/3B_lora_single_device.yaml 启动$ tune run lora_finetune_single_device --config llama3_2/3B_lora_single_device Setting manual seed to local seed 3977464327. Local seed is seed rank 3977464327 0 Hint: enable_activation_checkpointing is True, but enable_activation_offloading isnt. Enabling activation offloading should reduce memory further. Writing logs to /tmp/torchtune/llama3_2_3B/lora_single_device/logs/log_1734708879.txt Model is initialized with precision torch.bfloat16. Memory stats after model init: GPU peak memory allocation: 6.21 GiB GPU peak memory reserved: 6.27 GiB GPU peak memory active: 6.21 GiB Tokenizer is initialized from file. Optimizer and loss are initialized. Loss is initialized. Dataset and Sampler are initialized. Learning rate scheduler is initialized. Profiling disabled. Profiler config after instantiation: {enabled: False} 1|3|Loss: 1.943998098373413: 0%| | 3/1617 [00:213:04:47, 6.87s/it]该配置的关键参数均已核对 3B_lora_single_device.yaml 原文参数值说明model._component_torchtune.models.llama3_2.lora_llama3_2_3b带 LoRA 的 Llama-3.2-3B 构建器lora_attn_modules[q_proj, v_proj, output_proj]注意力中注入 LoRA 的线性层apply_lora_to_mlpTrueMLP 同样注入 LoRAlora_rank/lora_alpha64/128低秩维度与缩放系数惯例 alpha2×rankrank 越大精度与显存占用越高checkpointerFullModelHFCheckpointer从/tmp/Llama-3.2-3B-Instruct/读取两个model-0000X-of-00002.safetensors分片save_adapter_weights_onlyFalse同时保存合并后的全量权重关键供评测/生成/部署复用datasetalpaca_cleaned_datasetpacked: False微调数据batch_size/gradient_accumulation_steps4/8有效 batch size 4×8optimizerAdamWfusedlr: 3e-4weight_decay: 0.01配合 cosine warmup100 步调度dtype/devicebf16/cuda精度与设备enable_activation_checkpointingTrue激活重计算进一步降显存在batch_size4、dtypebfloat16下模型峰值显存约 16GB每个 epoch 总训练时长约 2–3 小时教程原文给出的参考值。训练完成后可用tree -a path/to/outputdir查看产出目录$ tree -a /tmp/torchtune/llama3_2_3B/lora_single_device /tmp/torchtune/llama3_2_3B/lora_single_device ├── epoch_0 │ ├── adapter_config.json │ ├── adapter_model.pt │ ├── adapter_model.safetensors │ ├── config.json │ ├── model-00001-of-00002.safetensors │ ├── model-00002-of-00002.safetensors │ ├── generation_config.json │ ├── LICENSE.txt │ ├── model.safetensors.index.json │ ├── original │ │ ├── orig_params.json │ │ ├── params.json │ │ └── tokenizer.model │ ├── original_repo_id.json │ ├── README.md │ ├── special_tokens_map.json │ ├── tokenizer_config.json │ ├── tokenizer.json │ └── USE_POLICY.md ├── epoch_1 │ ├── adapter_config.json │ ... ├── logs │ └── log_1734652101.txt └── recipe_state └── recipe_state.pt输出目录分三类recipe_state保存recipe_state.pt含恢复最近中间 epoch 所需的全部信息优化器状态、已完成 epoch 数等。checkpointer 的协议注释明确说明中间检查点在每个 epoch 结束时写出recipe_state.pt每 epoch 覆盖一次以避免目录膨胀见 torchtune/training/checkpointing/_checkpointer.pylogs训练全程的日志输出loss、显存、异常等epoch_{N}训练后的模型权重 模型元数据。做推理或上传模型中心时直接使用这个目录。各文件的作用教程原文逐项说明adapter_model.safetensors与adapter_model.ptLoRA adapter 权重.pt版本是重复保存的用于断点续训model-{}-of-{}.safetensors合并后的全量模型权重非 adapterLoRA 微调下仅当save_adapter_weights_onlyFalse时生成——即把基座模型与训练好的 adapter 合并方便推理adapter_config.jsonHugging Face PEFT 加载 adapter 时读取插入位置model.safetensors.index.jsonHugging Facefrom_pretrained()加载分片权重时读取的索引其余文件原本就在checkpoint_dir中训练时自动拷贝大于 100MiB 且以.safetensors/.pth/.pt/.bin结尾的文件会被跳过保持目录轻量——这一规则在源码中由SUFFIXES_TO_NOT_COPY与max_file_size_mb100控制见 torchtune/training/checkpointing/_utils.py。3.4 为什么产物能直接被 HF / vLLM 消费state-dict 不变性上述合并权重直接可用并非巧合。torchtune/training/checkpointing/_checkpointer.py 中_CheckpointerInterface的协议注释定义了 torchtune 的state-dict invariantstate-dict 不变性设计原则checkpointer 保证输出检查点与原始检查点具有相同格式——相同的 key、拆分到相同数量的文件中因此输出检查点可以直接配合原始元数据文件使用无需编写 torchtune 专用转换器即可在 gpt-fast、llama.cpp 等生态工具中加载。FullModelHFCheckpointer正是这一原则下面向 Hugging Face 格式*.safetensorsindex.json的实现这也是本教程评测、生成、部署环节全程直接指向epoch_0目录的底层原因。四、第三步评测微调后的模型4.1 用 EleutherAI Eval Harness 跑结构化评测torchtune 与 EleutherAI 的评测框架lm-evaluation-harness集成对应 recipe 为 recipes/eleuther_eval.py默认配置是 recipes/configs/eleuther_evaluation.yaml。先安装依赖pip install lm_eval0.4.5由于要修改的配置项较多先把配置拷到本地工作目录tune cp的实现见 torchtune/_cli/cp.py它同样遍历 recipe 注册表定位源文件复制时按文件类型自动补.py/.yaml后缀并支持--no-clobber与--make-parents$ tune cp eleuther_evaluation ./custom_eval_config.yaml Copied file to custom_eval_config.yaml将custom_eval_config.yaml改为指向微调产物注意使用合并后的权重而不是 LoRA adapter# TODO: update to your desired epoch output_dir: /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0 # Tokenizer tokenizer: _component_: torchtune.models.llama3.llama3_tokenizer path: ${output_dir}/original/tokenizer.model model: # Notice that we dont pass the lora model. We are using the merged weights, _component_: torchtune.models.llama3_2.llama3_2_3b checkpointer: _component_: torchtune.training.FullModelHFCheckpointer checkpoint_dir: ${output_dir} checkpoint_files: [ model-00001-of-00002.safetensors, model-00002-of-00002.safetensors, ] output_dir: ${output_dir} model_type: LLAMA3_2 ### OTHER PARAMETERS -- NOT RELATED TO THIS CHECKPOINT # Environment device: cuda dtype: bf16 seed: 1234 # It is not recommended to change this seed, b/c it matches EleutherAIs default seed # EleutherAI specific eval args tasks: [truthfulqa_mc2] limit: null max_seq_length: 4096 batch_size: 8 enable_kv_cache: True # Quantization specific args quantizer: null本教程选用 harness 中的truthfulqa_mc2任务它度量模型回答问题的真实性倾向统计一个问题 若干真/假回答设置下的 zero-shot 准确率。seed: 1234不建议改动因为它与 EleutherAI 的默认种子保持一致保证结果可对齐。然后运行$ tune run eleuther_eval --config ./custom_eval_config.yaml [evaluator.py:324] Running loglikelihood requests ...默认配置中model为llama2_7b、checkpoint_dir为/tmp/Llama-2-7b-hf迁移到其他模型时需要同时替换model、tokenizer和checkpointer三个组件且model_type要与FullModelHFCheckpointer支持的重命名逻辑匹配。4.2 生成一些输出看看结构化指标之外还要亲手验证模型在你关心的 prompt 上能否生成有意义的文本。这一步使用generatereciperecipes/generate.py与默认配置 recipes/configs/generation.yaml$ tune cp generation ./custom_generation_config.yaml Copied file to custom_generation_config.yaml $ mkdir /tmp/torchtune/llama3_2_3B/lora_single_device/out只需替换两个字段——output_dir与checkpoint_files所在的两处路径配置其余保持不变checkpoint_dir: /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0 output_dir: /tmp/torchtune/llama3_2_3B/lora_single_device/out # Tokenizer tokenizer: _component_: torchtune.models.llama3.llama3_tokenizer path: ${checkpoint_dir}/original/tokenizer.model prompt_template: null model: # Notice that we dont pass the lora model. We are using the merged weights, _component_: torchtune.models.llama3_2.llama3_2_3b checkpointer: _component_: torchtune.training.FullModelHFCheckpointer checkpoint_dir: ${checkpoint_dir} checkpoint_files: [ model-00001-of-00002.safetensors, model-00002-of-00002.safetensors, ] output_dir: ${output_dir} model_type: LLAMA3_2 ### OTHER PARAMETERS -- NOT RELATED TO THIS CHECKPOINT device: cuda dtype: bf16 seed: 1234 # Generation arguments; defaults taken from gpt-fast prompt: system: null user: Tell me a joke. max_new_tokens: 300 temperature: 0.6 # 0.8 and 0.6 are popular values to try top_k: 300 enable_kv_cache: True quantizer: null其中llama3_2_3b构建器定义在 torchtune/models/llama3_2/_model_builders.py28 层、24 头、8 个 KV 头、embed_dim3072、max_seq_len131072、默认绑定 word embeddings。生成参数沿用默认值top_k300、温度 0.6–0.8 区间这些参数控制采样概率的分布方式官方建议先用默认值观察模型表现再调参。运行时还可以用 CLI 覆盖机制直接改 prompt$ tune run generate --config ./custom_generation_config.yaml prompt.userTell me a joke. Tell me a joke. Heres a joke for you: What do you call a fake noodle? An impasta!4.3 引入量化torchao对推理吞吐有要求时torchtune 借助 torchao 完成训练后量化PTQ。安装 torchao 后核心调用只有两行# 也支持 int8_weight_only()、int8_dynamic_activation_int8_weight() 等更多技术 from torchao.quantization.quant_api import quantize_, int4_weight_only quantize_(model, int4_weight_only())量化后通常配合torch.compile获取加速。仓库中还提供了独立的quantizereciperecipes/quantize.py配置为 recipes/configs/quantization.yaml与qat_single_device、qat_distributed、qat_lora_finetune_distributed等量化感知训练 recipe可在注册表中查到各自支持的模型配置。教程同时指出对 Llama 系模型还可以直接用 torchao 自带的generate.py脚本在量化模型上跑生成与 torchao 官方的性能/精度参考结果做对照。五、在真实场景中使用模型模型达标后通常要做部署、发布或分享。得益于上面提到的 state-dict 不变性设计torchtune 产物就是标准 Hugging Face 格式无需任何转换。5.1 用法一Hugging Facefrom_pretrained() PEFT adapterCase 1基座模型 训练好的 adapter。从 Hub 加载基座模型再用PeftModel在其上挂载 adapter。PEFT 会自动在 adapter 目录下寻找adapter_model.safetensors权重和adapter_config.json插入位置from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer # TODO: update it to your chosen epoch trained_model_path /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0 # Define the model and adapter paths original_model_name meta-llama/Llama-3.2-1B-Instruct model AutoModelForCausalLM.from_pretrained(original_model_name) # huggingface will look for adapter_model.safetensors and adapter_config.json peft_model PeftModel.from_pretrained(model, trained_model_path) # Load the tokenizer tokenizer AutoTokenizer.from_pretrained(original_model_name) # Function to generate text def generate_text(model, tokenizer, prompt, max_length50): inputs tokenizer(prompt, return_tensorspt) outputs model.generate(**inputs, max_lengthmax_length) return tokenizer.decode(outputs[0], skip_special_tokensTrue) prompt tell me a joke: print(Base model output:, generate_text(peft_model, tokenizer, prompt))注意original_model_name必须与你实际微调的基座一致教程示例此处写作 1B实际按你的微调目标替换为对应仓库名。Case 2直接使用合并权重。此时 Hugging Face 读取model.safetensors.index.json确定应加载哪些分片文件代码更简洁from transformers import AutoModelForCausalLM, AutoTokenizer # TODO: update it to your chosen epoch trained_model_path /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0 model AutoModelForCausalLM.from_pretrained( pretrained_model_name_or_pathtrained_model_path, ) # Load the tokenizer tokenizer AutoTokenizer.from_pretrained(trained_model_path, safetensorsTrue) # Function to generate text def generate_text(model, tokenizer, prompt, max_length50): inputs tokenizer(prompt, return_tensorspt) outputs model.generate(**inputs, max_lengthmax_length) return tokenizer.decode(outputs[0], skip_special_tokensTrue) prompt Complete the sentence: Once upon a time... print(Base model output:, generate_text(model, tokenizer, prompt))5.2 用法二vLLM 推理服务vLLM 是面向 LLM 推理与服务的库提供高吞吐 serving、continuous batching、量化与投机解码等能力。它可以直接加载任意.safetensors文件由于我们已经把 adapter 权重合并进全量权重为避免 vLLM 被 adapter 文件干扰可先把 adapter 文件移走rm /tmp/torchtune/llama3_2_3B/lora_single_device/base_model/adapter_model.safetensors然后运行from vllm import LLM, SamplingParams def print_outputs(outputs): for output in outputs: prompt output.prompt generated_text output.outputs[0].text print(fPrompt: {prompt!r}, Generated text: {generated_text!r}) print(- * 80) # TODO: update it to your chosen epoch llm LLM( model/tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0, load_formatsafetensors, kv_cache_dtypeauto, ) sampling_params SamplingParams(max_tokens16, temperature0.5) conversation [ {role: system, content: You are a helpful assistant}, {role: user, content: Hello}, {role: assistant, content: Hello! How can I assist you today?}, { role: user, content: Write an essay about the importance of higher education., }, ] outputs llm.chat(conversation, sampling_paramssampling_params, use_tqdmFalse) print_outputs(outputs)5.3 用法三上传到 Hugging Face Hub分享模型最省事的途径是huggingface_hub的文件夹上传 APIepoch_0目录已经包含推理所需的全部文件import huggingface_hub api huggingface_hub.HfApi() # TODO: update it to your chosen epoch trained_model_path /tmp/torchtune/llama3_2_3B/lora_single_device/epoch_0 username huggingface_hub.whoami()[name] repo_name my-model-trained-with-torchtune # if the repo doesnt exist repo_id huggingface_hub.create_repo(repo_name).repo_id # if it already exists repo_id f{username}/{repo_name} api.upload_folder( folder_pathtrained_model_path, repo_idrepo_id, repo_typemodel, create_prFalse )如果偏好命令行也可以直接使用huggingface-cli upload。六、小结整条链路的命令速查把教程中的操作浓缩成一张速查表所有路径均以仓库根目录为准便于对照复现阶段命令 / 文件说明下载tune download meta-llama/Llama-3.2-3B-Instruct --ignore-patterns original/consolidated.00.pth实现见 torchtune/_cli/download.py找配置tune ls数据来源 torchtune/_recipe_registry.py微调tune run lora_finetune_single_device --config llama3_2/3B_lora_single_device配置 recipes/configs/llama3_2/3B_lora_single_device.yaml拷贝配置tune cp eleuther_evaluation ./custom_eval_config.yaml实现见 torchtune/_cli/cp.py评测tune run eleuther_eval --config ./custom_eval_config.yamlrecipe recipes/eleuther_eval.py生成tune run generate --config ./custom_generation_config.yaml prompt.userTell me a joke. recipe recipes/generate.py量化quantize_(model, int4_weight_only())torchaorecipe recipes/quantize.py产物epoch_0/合并权重 index.json adapter 元数据设计见 torchtune/training/checkpointing/_checkpointer.py掌握这条下载 → 微调 → 评测 → 生成 → 量化 → 部署的端到端流水线后你可以把其中任意一环替换为注册表中的其他 recipe分布式全量微调、DPO、知识蒸馏、QAT、GRPO 等而产物格式与生态接入方式保持不变——这正是 torchtune 以 recipe config state-dict 不变 checkpointer 为核心设计带来的收益。【免费下载链接】torchtunePyTorch native post-training library项目地址: https://gitcode.com/GitHub_Trending/to/torchtune创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考