LLMWare Prompt 类实战用 add_source_document 与 prompt_with_source 构建本地化发票 RAG 与文档摘要流水线【免费下载链接】llmwareUnified framework for building enterprise RAG pipelines with small, specialized models项目地址: https://gitcode.com/GitHub_Trending/ll/llmware本文基于 LLMWare 仓库文档 docs/examples/prompts.md 中“以示例讲解 Prompts”的主题展开完整讲解两个可本地运行的核心示例基于Prompt().prompt_with_source()的发票批处理 RAG 场景以及基于slim-summary-tool的超长文档摘要场景。读完本文你将掌握 llmware 中“文档解析 → 挂载为 Prompt 源材料 → 模板化推理 → 状态持久化与人工审核 CSV 导出”的完整调用链并能结合 llmware/prompts.py 的源码理解每一步背后的状态管理与模板拼装机制。Prompt 类llmware 推理过程的状态中枢在 llmware 中Prompt类是推理inference过程的统一入口。根据 llmware/prompts.py 的类定义Prompt负责推理的预处理、执行、后处理以及一系列相关推理的端到端状态管理。理解本文两个示例之前先把握三个核心状态属性见 llmware/prompts.py#L159-L198interaction_history当前 Prompt 会话的主“活跃”历史记录每次register调用都会向其中追加一条推理记录prompt、llm_response、evidence、human_feedback等状态变量均在其中完整清单见llm_state_varsdialog_tracker从interaction_history中提取的“user/bot”对话追踪列表source_materials一个“有状态”的源材料列表每个条目是一个包含batch_id、text、metadata、batch_stats、biblio等键的字典即后续prompt_with_source()所使用的“上下文”。此外Prompt.__init__在构造时会自动通过PromptState签发或加载prompt_id并在LLMWareConfig.get_llmware_path()下确保prompt_history目录存在见 llmware/prompts.py#L149-L214。默认推理参数为temperature0.3、llm_max_output_len200、prompt_wrapperhuman_bot并假设最小 2048 全上下文窗口50% 输入/50% 输出设定初始context_window_size1000——加载模型后该值会按模型实际max_input_len更新。Prompt还提供了多种“挂载源”的方法族本文示例用到的是add_source_document()其余还包括add_source_new_query()对 library 跑一次查询作为源、add_source_query_results()、add_source_library()、add_source_wikipedia()、add_source_yahoo_finance()、add_source_website()等见 llmware/prompts.py#L358-L486共同点是把结果统一交给Sources(self).package_source(...)打包进source_materials。示例一发票处理——解析 prompt_with_source 的端到端批处理原文档的第一个示例展示了一个可本地运行、不依赖数据库和向量嵌入的发票处理场景将解析parsing与prompt_with_sources结合对一批发票逐一提问并把完整输出保存为两种格式——(1).jsonl供上游应用/数据库集成(2) CSV 供人工在 Excel 中复核。首次运行时示例代码会从公开仓库拉取样例发票文档PDF/DOCX/PPTX/XLSX/CSV/TXT 均可替换为自己的文件设置run_on_cpuTrue即可在笔记本电脑上运行。完整示例代码如下继承自 docs/examples/prompts.md This example shows an end-to-end scenario for invoice processing that can be run locally and without a database. The example shows how to combine the use of parsing combined with prompts_with_sources to rapidly iterate through a batch of invoices and ask a set of questions, and then save the full output to both (1) .jsonl for integration into an upstream application/database and (2) to a CSV for human review in excel. note: the sample code pulls from a public repo to load the sample invoice documents the first time - please feel free to substitute with your own invoice documents (PDF/DOCX/PPTX/XLSX/CSV/TXT) if you prefer. this example does not require a database or embedding this example can be run locally on a laptop by setting run_on_cpuTrue if run_on_cpuFalse, then please see the example launch_llmware_inference_server.py to configure and set up a pop-up GPU inference server in just a few minutes import os import re from llmware.prompts import Prompt, HumanInTheLoop from llmware.configs import LLMWareConfig from llmware.setup import Setup from llmware.models import ModelCatalog def invoice_processing(run_on_cpuTrue): # Step 1 - Pull down the sample files from S3 through the .load_sample_files() command # --note: if you need to refresh the sample files, set over_writeTrue print(update: Downloading Sample Files) sample_files_path Setup().load_sample_files(over_writeFalse) invoices_path os.path.join(sample_files_path, Invoices) # Step 2 - simple sample query list - each question will be asked to each invoice query_list [What is the total amount of the invoice?, What is the invoice number?, What are the names of the two parties?] # Step 3 - Load Model if run_on_cpu: # load local bling model that can run on cpu/laptop # note: bling-1b-0.1 is the *fastest* *smallest*, but will make more errors than larger BLING models # model_name llmware/bling-1b-0.1 # try the new bling-phi-3 quantized with gguf - most accurate model_name bling-phi-3-gguf else: # use GPU-based inference server to process # *** see the launch_llmware_inference_server.py example script to setup *** server_uri_string http://11.123.456.789:8088 # insert your server_uri_string server_secret_key demo-test ModelCatalog().setup_custom_llmware_inference_server(server_uri_string, secret_keyserver_secret_key) model_name llmware-inference-server # attach inference server to prompt object prompter Prompt().load_model(model_name) # Step 4 - main loop thru folder of invoices for i, invoice in enumerate(os.listdir(invoices_path)): # just in case (legacy on mac os file system - not needed on linux or windows) if invoice ! .DS_Store: print(\nAnalyzing invoice: , str(i 1), invoice) for question in query_list: # Step 4A - parses the invoices in memory and attaches as a source to the Prompt source prompter.add_source_document(invoices_path,invoice) # Step 4B - executes the prompt on the LLM (with the loaded source) output prompter.prompt_with_source(question,prompt_namedefault_with_context) for i, response in enumerate(output): print(LLM Response - , question, - , re.sub([\n], , response[llm_response])) prompter.clear_source_materials() # Save jsonl report with full transaction history to /prompt_history folder print(\nupdate: prompt state saved at: , os.path.join(LLMWareConfig.get_prompt_path(),prompter.prompt_id)) prompter.save_state() # Generate CSV report for easy Human review in Excel csv_output HumanInTheLoop(prompter).export_current_interaction_to_csv() print(\nupdate: csv output for human review - , csv_output) return 0 if __name__ __main__: invoice_processing(run_on_cpuTrue)Step 1 源码印证Setup().load_sample_files()从哪里拉取文件load_sample_files()实现在 llmware/setup.py#L74-L103它先确保LLMWareConfig.get_llmware_path()工作区存在然后把样例文件固定下载到llmware_path/sample_files目录该路径不可配置。若目录已存在且over_writeFalse直接返回缓存路径否则会向公共 S3 桶桶名由配置项llmware_sample_files_bucket决定默认值为llmware-sample-docs见 llmware/configs.py#L93拉取 zip 包、解压并删除压缩包。Setup类文档字符串中列出的八个样例域包括AgreementsLarge约 80 份样例合同、Agreements约 15 份雇佣协议、UN-Resolutions-500500 份联合国决议、Invoices约 40 份发票样例、FinDocs约 15 份财务年报/10K、AWS-Transcribe、SmallLibrary约 10 份混合文档类型、Images约 3 张 OCR 图片。本示例用到的正是Invoices与后文的SmallLibrary、Agreements子目录。Step 3 源码印证CPU 本地模型与 GPU 推理服务器两种模式CPU 本地模式默认选用bling-phi-3-gguf。从模型注册表 llmware/model_configs.py#L603-L611 可以确认其实现细节model_family为GGUFGenerativeModelmodel_category为generative_localcontext_window为 4096prompt_wrapper为human_bottemperature为 0.0GGUF 文件bling-phi-3.gguf从 Hugging Face 仓库llmware/bling-phi-3-gguf拉取。基准分数字典llmware/model_configs.py#L4357-L4365记录其基座模型为microsoft/Phi-3-mini-4k-instruct、参数量 3.8B。备选注释中提到的llmware/bling-1b-0.1是最小最快的 BLING 模型适合对速度要求更高、可容忍更多错误的场景。GPU 推理服务器模式调用ModelCatalog().setup_custom_llmware_inference_server(server_uri_string, secret_key...)。从源码 llmware/models.py#L920-L932 看该方法本质是写入两个环境变量LLMWARE_GPT_URI服务地址与USER_MANAGED_LLMWARE_GPT_API_KEY密钥之后以model_name llmware-inference-server加载即可。文档同时提示GPU 服务器可通过配套示例launch_llmware_inference_server.py在几分钟内拉起一个“pop-up”推理服务。加载模型统一走Prompt.load_model()llmware/prompts.py#L216-L253非 Hugging Face 路径经ModelCatalog().load_model()加载Hugging Face 路径from_hfTrue则通过PyTorchLoader载入自定义生成式模型并按human_bot包装器适配。方法在末尾把context_window_size设为模型的max_input_len并把llm_max_output_len设为传入的max_output默认 200。Step 4 源码印证add_source_document与prompt_with_source的调用链add_source_document(input_fp, input_fn, queryNone)的实现llmware/prompts.py#L488-L509分三步Parser().parse_one(input_fp, input_fn)在内存中解析该文档任意受支持类型不写库若传入可选query用Utilities().fast_search_dicts()在内存中做过滤只保留匹配 query 的块去停用词Sources(self).package_source(output, aggregate_sourceTrue)将解析结果按上下文窗口聚合成一个或多个 batch追加到self.source_materials若无文本则记录 warning。prompt_with_source(prompt, prompt_nameNone, source_id_listNone, first_source_onlyTrue, max_outputNone, temperatureNone, verboseFalse)的完整签名与行为llmware/prompts.py#L563-L649值得注意若未挂载任何源材料会打 warning 并以空上下文执行可能得到意外结果first_source_onlyTrue默认只使用第一个源 batchfirst_source_onlyFalse时会对source_materials的每个 batch 迭代调用模型此时可用source_id_list如[0,1,5]指定参与推理的 batch 索引每次响应字典会自动并入该 batch 的evidence_metadata源元数据与biblio书目信息方便回答溯源prompt_name指定使用的预置模板。示例中使用的模板default_with_context在提示词目录 llmware/model_configs.py#L4238-L4244 中定义为{prompt_name: default_with_context, prompt_description: Default simple prompt when a question and context are passed., run_order: [blurb1, $context, blurb2, $query], blurb1: Please read the following text: , blurb2: Based on this text, please answer the question: , system_message: You are a helpful assistant who speaks with facts and no wasted words.}即最终 prompt 的拼装顺序为引导语 → 源文本$context→ “基于上文回答问题”引导 → 用户问题$query。同文件还注册了default_no_context、xsummary、not_found_classifier、top_level_select、yes_no、multiple_choice等大量可复用模板llmware/model_configs.py#L4150-L4260可替换prompt_name参数直接用于不同任务。主循环中每次提问后调用prompter.clear_source_materials()llmware/prompts.py#L287-L291把source_materials重置为空列表保证下一张发票使用全新源材料避免跨文档污染上下文。Step 5 源码印证save_state()与HumanInTheLoop的双通道输出JSONL 持久化Prompt.save_state()调用PromptState(self).save_state(self.prompt_id)llmware/prompts.py#L337-L342把完整推理事务历史写入LLMWareConfig.get_prompt_path()即prompt_history目录下以prompt_id命名的记录。示例中打印的正是该保存路径。CSV 人工审核HumanInTheLoop(prompter).export_current_interaction_to_csv()llmware/prompts.py#L1964-L1971内部调用PromptState(...).generate_interaction_report_current_state(...)把当前会话状态导出为 Excel 可打开的 CSV。按 llmware/prompts.py#L1933-L1947 中类文档自带的 doctest返回形如{report_name: interaction_report_....csv, report_fp: /home/user/llmware_data/prompt_history/interaction_report_....csv, results: 1}的字典。HumanInTheLoop还提供审核后回写能力add_or_update_human_rating(prompt_id, rating_dict)可更新human_rating、human_feedback、human_assessed_accuracy三个字段llmware/prompts.py#L1981-L2004update_llm_response_record()则在修改记录时把原值存入change_log列表以支持追溯llmware/prompts.py#L2006-L2036。示例二Document Summarizer——slim-summary-tool 的超长文档摘要原文档的第二个示例展示使用打包好的document_summarizerprompt底层模型slim-summary-tool对通常大于 LLM 上下文窗口的文档做摘要并演示如何用query与topic聚焦文档的特定片段。完整代码如下 This Example shows a packaged document_summarizer prompt using the slim-summary-tool. It shows a variety of techniques to summarize documents generally larger than a LLM context window, and how to assemble multiple source batches from the document, as well as using a query and topic to focus on specific segments of the document. import os from llmware.prompts import Prompt from llmware.setup import Setup def test_summarize_document(examplejd salinger): # pull a sample document (or substitute a file_path and file_name of your own) sample_files_path Setup().load_sample_files(over_writeFalse) topic None query None fp None fn None if example not in [jd salinger, employment terms, just the comp, un resolutions]: print (not found example) return [] if example jd salinger: fp os.path.join(sample_files_path, SmallLibrary) fn Jd-Salinger-Biography.docx topic jd salinger query None if example employment terms: fp os.path.join(sample_files_path, Agreements) fn Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf topic executive compensation terms query None if example just the comp: fp os.path.join(sample_files_path, Agreements) fn Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf topic executive compensation terms query base salary if example un resolutions: fp os.path.join(sample_files_path, SmallLibrary) fn N2126108.pdf # fn N2137825.pdf topic key points query None # optional parameters: query - will select among blocks with the query term # topic - will pass a topic/issue as the parameter to the model to focus the summary # max_batch_cap - caps the number of batches sent to the model # text_only - returns just the summary text aggregated kp Prompt().summarize_document_fc(fp, fn, topictopic, queryquery, text_onlyTrue, max_batch_cap15) print(f\nDocument summary completed - {len(kp)} Points) for i, points in enumerate(kp): print(i, points) return 0 if __name__ __main__: print(f\nExample: Summarize Documents\n) # 4 examples - [jd salinger, employment terms, just the comp, un resolutions] # -- jd salinger - summarizes key points about jd salinger from short biography document # -- employment terms - summarizes the executive compensation terms across 15 page document # -- just the comp - queries to find subset of document and then summarizes the key terms # -- un resolutions - summarizes the un resolutions document summary_direct test_summarize_document(exampleemployment terms)四个内置样例各演示一种用法jd salinger从短篇传记 docx 中提取人物要点employment terms对 15 页雇佣协议 PDF 按主题executive compensation terms做摘要just the comp先用querybase salary从文档块中筛选子集再聚焦摘要un resolutions则直接以 “key points” 为主题摘要联合国决议文档。summarize_document_fc的参数与内部实现方法签名为summarize_document_fc(self, fp, fn, topickey points, queryNone, text_onlyTrue, max_batch_cap15, summary_modelslim-summary-tool, real_time_updateTrue)llmware/prompts.py#L992-L1030四个常用可选参数含义如下参数默认值作用topickey points作为“指令文本”传给模型让摘要聚焦特定主题/议题queryNone非空时先在该文档的解析块中按 query 词筛选内存过滤去停用词仅保留匹配块作为源max_batch_cap15上限截断source_materials超过该值时只保留前 N 个 batch控制送入模型的批次数与成本text_onlyTrue文档注释中标记为可选返回形态注意源码实际返回值是去重后的要点列表listtext_only未改变该返回结构从源码结构看summarize_document_fc的执行流程为self.load_model(summary_model, temperature0.0, sampleFalse)以确定性采样加载slim-summary-tool并把llm_max_output_len固定为 150调用add_source_document(fp, fn)带或不带query过滤把文档切分并打包为多个 source batch——这正是“文档大于上下文窗口”时能工作的关键文档被切成多个 batch而非单次塞入模型对source_materials做max_batch_cap截断self.prompt_with_source(topic, first_source_onlyFalse, verboseTrue)让topic作为问题对每个 batch迭代推理汇总各 batch 返回的要点列表resp[llm_response]为列表做去重、去空串、并过滤以 “Not Found” 开头的条目最终返回key_points列表。slim-summary-tool的模型卡片定义在 llmware/model_configs.py#L1677-L1682GGUF 文件为slim-summarize.gguf来自 Hugging Face 仓库llmware/slim-summary-tool。llmware 还提供了面向 library 的姊妹方法summarize_document_from_library(library, doc_idNone, filenameNone, queryNone, text_onlyTrue, max_batch_cap10)llmware/prompts.py#L1032-L1075它通过Query(library)按doc_ID或file_source定位文档块可选带text_query_with_custom_filter过滤后再走摘要流程适合文档已入库的场景。运行前提、限制与进一步阅读运行环境两个示例均不依赖数据库或向量嵌入纯本地可跑CPU 笔记本直接设run_on_cpuTruebling-phi-3-gguf的 4096 上下文窗口与 GGUF 量化配置使其适合此类轻量批处理。首次运行load_sample_files()需网络访问公共 S3 桶可能耗时约一分钟源码日志原文如此说明样例文件持续更新需要最新版时设over_writeTrue。推理服务器模式示例代码中的server_uri_string与server_secret_key为占位值需替换为自己的服务地址与密钥对应环境变量LLMWARE_GPT_URI/USER_MANAGED_LLMWARE_GPT_API_KEY。状态与溯源prompt_with_source返回的每个响应字典自带evidence_metadata与bibliosave_state()落盘的 jsonl 保留了完整事务历史HumanInTheLoop的 CSV 则面向人工复核三者构成“机器输出 → 持久化 → 人工审核回写”的闭环。更多可运行代码仓库内 solutions/models/prompt_with_sources.py、solutions/models/document_summarizer.py 与 solutions/use_cases/invoice_processing.py 提供了本文两个示例对应的独立脚本版本可对照本文的源码分析直接运行API 层面的补充说明可参见 docs/components/prompt_with_sources.md。【免费下载链接】llmwareUnified framework for building enterprise RAG pipelines with small, specialized models项目地址: https://gitcode.com/GitHub_Trending/ll/llmware创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考