Upload 20 files

.gitattributes

@@ -43,3 +43,12 @@ Images/英文金融.gif filter=lfs diff=lfs merge=lfs -text
 Images/data_construct.png filter=lfs diff=lfs merge=lfs -text
 Images/ESG.gif filter=lfs diff=lfs merge=lfs -text
 Images/trainning.png filter=lfs diff=lfs merge=lfs -text
+Images/.frame_cn2.png filter=lfs diff=lfs merge=lfs -text
+Images/Data_distribution_en.png filter=lfs diff=lfs merge=lfs -text
+Images/English_Financial_Calculations.gif filter=lfs diff=lfs merge=lfs -text
+Images/Fin-R1-pipeline_en.png filter=lfs diff=lfs merge=lfs -text
+Images/Financial_Calculations.gif filter=lfs diff=lfs merge=lfs -text
+Images/Financial_Code.gif filter=lfs diff=lfs merge=lfs -text
+Images/Financial_Security_and_Compliance.gif filter=lfs diff=lfs merge=lfs -text
+Images/Intelligent_Risk_Control.gif filter=lfs diff=lfs merge=lfs -text
+Technical_report.pdf filter=lfs diff=lfs merge=lfs -text

Images/README

@@ -0,0 +1 @@
+图片

README.md

@@ -0,0 +1,203 @@
+<div align="center">
+  <img src="Images/title.png" width="700" height="200">
+</div>
+<div align="center">
+  <h1>Fin-R1：通过强化学习驱动的金融推理大模型</h1>
+  
+<!-- 徽章部分 -->
+  [![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)    
+  [![模型下载](https://img.shields.io/badge/🤗-下载模型-blue)](https://huggingface.co/SUFE-AIFLM-Lab/Fin-R1)    
+  [![技术报告](https://img.shields.io/badge/📚-技术报告-orange)](https://arxiv.org/abs/2503.16252)                              
+
+  <!-- 语言切换链接 -->
+  📄 [中文](./README.md) | [EN](./README_en.md)          
+</div>
+
+Fin-R1 是一款针对金融领域复杂推理的大型语言模型，由上海财经大学统计与数据科学学院张立文教授与其领衔的金融大语言模型课题组（SUFE-AIFLM-Lab）联合财跃星辰研发并开源发布。该模型以 Qwen2.5-7B-Instruct 为基座，通过高质量的可验证金融问题微调训练，最终表现在多个金融领域基准测试上的表现达到参评模型的SOTA水平。
+
+
+
+## 📌 目录<a name="toc"></a>   
+  - [场景应用](#summary)  
+    - [金融代码](#金融代码)  
+    - [金融计算](#金融计算)  
+    - [英语金融计算](#英语金融计算)  
+    - [金融安全合规](#金融安全合规)  
+    - [智能风控](#智能风控)  
+    - [ESG分析](#ESG分析)
+  - [总体工作流程](#总体工作流程)  
+    - [数据构建](#data)  
+    - [微调训练](#trainning)  
+    - [模型评测结果](#results)  
+    - [模型使用方法](#use)
+  - [未来展望](#todo)  
+  - [联系我们](#connection)  
+
+## 💡 场景应用 <a name="summary"></a>        
+Fin-R1 是一款专为金融推理领域设计的大语言模型，采用轻量化的 7B 参数量级架构。在显著降低部署成本的同时，该模型通过在针对金融推理场景的高质量思维链数据上采用 SFT（监督微调）和 RL（强化学习）两阶段训练，为模型在金融领域的应用提供了坚实的理论支撑、业务规则、决策逻辑以及技术实现能力，从而有效提升模型的金融复杂推理能力，为银行、证券、保险以及信托等金融核心业务场景提供有力支持。
+
+![数据-场景](Images/.frame_cn2.png)        
+ 
+## 金融代码
+金融代码是指在金融领域中用于实现各种金融模型、算法和分析任务的计算机编程代码，涵盖了从简单的财务计算到复杂的金融衍生品定价、风险评估和投资组合优化等多个方面，以方便金融专业人士进行数据处理、统计分析、数值计算和可视化等工作。
+![FinancialCode](Images/Financial_Code.gif)       
+## 金融计算
+金融计算是对金融领域的各种问题进行定量分析和计算的过程，其核心在于通过建立数学模型和运用数值方法来解决实际金融问题，可为金融决策提供科学依据，帮助金融机构和投资者更好地管理风险、优化资源配置和提高投资回报率。
+![FinancialCalculations](Images/Financial_Calculations.gif)       
+## 英语金融计算
+英语金融计算强调在跨语言环境下使用英语进行金融模型的构建和计算，并能够以英语撰写金融分析报告和与国际同行进行沟通交流。
+![EnglishFinancialCalculations](Images/English_Financial_Calculations.gif)        
+## 金融安全合规
+金融安全合规聚焦于防范金融犯罪与遵守监管要求，帮助企业建立健全的合规管理体系，定期进行合规检查和审计，确保业务操作符合相关法规要求。
+![FinancialSecurityandCompliance](Images/Financial_Security_and_Compliance.gif)              
+## 智能风控
+智能风控利用AI与大数据技术识别和管理金融风险，与传统风控手段相比，智能风控具有更高的效率、准确性和实时性，它通过对海量金融数据的深度挖掘和分析，能够发现潜在的风险模式和异常交易行为，从而及时预警和采取相应的风险控制措施。
+![IntelligentRiskControl](Images/Intelligent_Risk_Control.gif)            
+## ESG分析
+ESG分析通过评估企业在环境（Environmental）、社会（Social）、治理（Governance）的表现，衡量其可持续发展能力，确保投资活动不仅能够获得财务回报，还能促进可持续发展和社会责任的履行。金融机构和企业也通过提升自身的 ESG 绩效，来满足投资者和社会对企业更高的期望和要求。
+![ESG](Images/ESG.gif)      
+
+  
+
+
+## 总体工作流程
+我们基于 DeepSeek-R1 构建了数据蒸馏框架，并严格按照官方参数设定进行数据处理，采用两阶段数据筛选方法提升金融领域数据质量，生成了SFT数据集和RL数据集。在训练过程中，我们利用Qwen2.5-7B-Instruct，通过监督微调（SFT）和强化学习（RL）训练金融推理大模型 Fin-R1，以提升金融推理任务的准确性和泛化能力。
+![总体工作流程](Images/.frame2_cn.png)              
+
+## 🛠️ 数据构建<a name="data"></a>
+为将 DeepSeek-R1 的推理能力迁移至金融场景并解决高质量金融推理数据问题，我们用Deepseek-R1（满血版）针对涵盖行业语料（FinCorpus、Ant_Finance），专业认知（FinPEE），业务知识（FinCUGE、FinanceIQ、Finance-Instruct-500K），表格解析（FinQA），市场洞察（TFNS），多轮交互（ConvFinQA）以及量化投资（FinanceQT）的多个数据集进行领域知识蒸馏筛选，构建了约 60k 条面向专业金融推理场景的高质量 COT 数据集 Fin-R1-Data 。该数据集涵盖中英文金融垂直领域的多维度专业知识，并根据具体任务内容将其分为金融代码、金融专业知识、金融非推理类业务知识和金融推理类业务知识四大模块，可有效支撑银行、基金和证券等多个金融核心场景。本研究构建了基于 Deepseek-R1 的数据蒸馏框架，并创新性提出对思维链进行“答案+推理”双轮质量打分筛选方法，首轮基于规则匹配和 Qwen2.5-72B-Instruct 对答案准确性评分，次轮对推理链的逻辑一致性、术语合规性等推理逻辑进行深度校验以保证数据质量。
+
+![数据处理](Images/data_construct.png)
+
+### 数据蒸馏
+
+在蒸馏过程中，我们严格依照 [DeepSeek - R1](https://github.com/deepseek-ai/DeepSeek-R1) 官方提供的细节，进行相应设置的数据蒸馏操作。
+
+### 数据筛选
+
+针对金融数据结构的复杂特性采取对思维链进行“答案+推理逻辑”双轮质量打分的创新方式筛选，首轮基于规则匹配和 Qwen2.5-72B-Instruct 对答案准确性评分，次轮对推理链的逻辑一致性、术语合规性等推理逻辑进行深度校验以保证数据质量，每次打分筛选出的数据标注为 good 或 bad 进行区分：
+
+1）答案打分：对于蒸馏得到的数据，针对客观题（如选择题、判断题），采用基于规则的匹配方式，校对蒸馏数据的正确性；对于无法通过规则匹配的结果，利用 Qwen2.5-72B-Instruct 对模型生成的答案以及正确答案进行打分，正确得 1 分，错误得 0 分。
+
+2）推理过程打分：对于经过上一步筛选得到的正确思维链数据，再次利用 Qwen2.5-72B-Instruct 对推理轨迹进行打分，高质量数据得 1 分，低质量数据得 0 分。我们采取了如下几个指标来进行打分：
+>
+> 1.内部一致性：检查推理过程中的步骤是否一致，并且是否能够逐步逻辑地推导出标准答案。
+>
+> 2.术语重叠度：检查推理过程中使用的术语与标准答案中的术语的重叠程度。重叠度越高越好。
+>
+> 3.推理步骤数量：评估推理过程是否包含足够的步骤（至少3步）。
+>
+> 4.逻辑一致性：确保推理过程中的步骤与标准答案在逻辑上高度一致，并检查是否存在明显的错误或遗漏。
+>
+> 5.内容多样性：检查推理过程中是否存在大量重复的步骤。
+>
+> 6.与任务领域的相关性：检查推理过程是否涉及与任务领域相关的内容（任务领域：{task_domain}）。如果推理反映了与任务领域的相关性，则给予更高的评分。
+>
+> 7.与任务指令的一致性：检查推理过程是否与任务指令高度相关。相关性越高越好。如果推理内容完全符合任务指令，则给予更高的评分。
+
+我们将经过两轮筛选后均标注为good的数据作为高质量的 COT 数据用于 SFT ；而未经过筛选标注为bad的数据则作为推理QA数据用于强化学习（RL）。
+
+### Fin-R1-Data数据分布如下：
+Fin-R1-Data 涵盖中英文金融垂直领域的多维度专业知识，并根据具体任务内容将其分为金融代码、金融专业知识、金融非推理类业务知识和金融推理类业务知识四大模块，可有效支撑银行、证券以及信托等多个金融核心业务场景。
+
+![grpo](Images/frame_cn.png)         
+
+|数据集|数据量|
+|-------------|--------|
+|ConvFinQA-R1-Distill |7629|
+|Finance-Instruct-500K-R1-Distill | 11300 |
+|FinCUGE-R1-Distill | 2000 |
+|FinQA-R1-Distill | 2948 | 
+|TFNS-R1-Distill | 2451|                                                     
+|FinanceIQ-R1-Distill | 2596 |
+|FinanceQT-R1-Distill | 152 |
+|Ant_Finance-R1-Distill | 1548 |
+|FinCorpus-R1-Distill | 29288|
+|FinPEE-R1-Distill | 179 |
+|总计| 60091 |
+
+
+
+
+
+## 🚀 微调训练<a name="trainning"></a>
+
+### 两阶段流程
+针对金融领域复杂推理任务，我们利用 Qwen2.5-7B-Instruct 进行两阶段微调训练得到金融推理大语言模型 Fin-R1 。首先通过高质量金融推理数据的 SFT (Supervised Fine-Tuning)  帮助模型初步提升金融推理能力，然后在 GRPO（Group Relative Policy Optimization) 算法的基础上结合格式奖励和准确���奖励进行强化学习，以此进一步提升金融推理任务的准确性和泛化能力。
+#### 第一阶段----推理能力注入： 
+
+针对金融推理任务中的复杂推理，我们第一阶段使用 ConvFinQA 和 FinQA 金融数据集对 Qwen2.5-7B-Instruct 进行了监督微调。经过一轮微调训练，确保模型能够深入理解并处理复杂的金融推理问题。
+ 
+#### 第二阶段----强化学习优化： 
+
+在模型掌握复杂推理技能后，我们采用 GRPO（Group Relative Policy Optimization）算法作为核心框架，以双重奖励机制优化模型输出的格式和准确度，并在此基础上引入了基于模型的验证器（Model-Based Verifier），采用 Qwen2.5-Max 进行答案评估来改进基于正则表达式的奖励可能存在的偏差，生成更加精确可靠的奖励信号，从而提升强化学习的效果和稳定性。
+
+![grpo](Images/trainning.png)
+
+
+## 🚨 模型评测结果 <a name="results"></a>
+我们在覆盖多项金融业务场景的基准测试上对模型进行评估，在评测结果中，只经过指令微调 (SFT) 的模型 Fin-R1-SFT 在金融场景中相较于基础模型已经取得了一定性能提升，但是相比于 DeepSeek-R1 仍有提升空间，我们于是在 Fin-R1-SFT 基础上再进行强化学习训练，结果发现经过指令微调 (SFT) 加强化学习 (RL) 训练的 Fin-R1 以仅 7B 的轻量化参数规模展现出显著的性能优势，达到 75.2 的平均得分位居第二，全面超越参评的同规模模型，同时与行业标杆 DeepSeek-R1 平均分差距仅3.0， 且超越DeepSeek-R1-Distill-Llama-70B（69.2）6.0分。此外 Fin-R1 在聚焦真实金融表格数值推理任务的 FinQA 以及多轮推理交互场景的 ConvFinQA 两大关键任务测试上分别以 76.0 和 85.0 的得分在参评模型中登顶第一，展现出了模型在金融推理场景及金融非推理场景中的强大处理能力。
+
+
+| Model                        | Parameters |  FinQA | ConvFinQA | Ant_Finance |  TFNS  |  Finance-Instruct-500k  | Average |
+|------------------------------|------------|--------|-----------|-------------|--------|-------------------------|---------|
+| DeepSeek-R1                  | 671B       |  71.0  | 82.0      | __90.0__    |  78.0  | __70.0__                | __78.2__| 
+| __Fin-R1__                   | 7B         |__76.0__| __85.0__  | 81.0        |  71.0  | 62.9                    | 75.2    |
+| Qwen-2.5-32B-Instruct        | 32B        |  72.0  | 78.0      | 84.0        |  77.0  | 58.0                    | 73.8    |          
+| DeepSeek-R1-Distill-Qwen-32B | 32B        |  70.0  | 72.0      | 87.0        |__79.0__| 54.0                    | 72.4    |                          
+| __Fin-R1-SFT__               | 7B         |  73.0  | 81.0      | 76.0        |  68.0  | 61.0                    | 71.9    |      
+| Qwen-2.5-14B-Instruct        | 14B        |  68.0  | 77.0      | 84.0        |  72.0  | 56.0                    | 71.4    |            
+| DeepSeek-R1-Distill-Llama-70B| 70B        |  68.0  | 74.0      | 84.0        |  62.0  | 56.0                    | 69.2    |    
+| DeepSeek-R1-Distill-Qwen-14B | 14B        |  62.0  | 73.0      | 82.0        |  65.0  | 49.0                    | 66.2    |                          
+| Qwen-2.5-7B-Instruct         | 7B         |  60.0  | 66.0      | 85.0        |  68.0  | 49.0                    | 65.6    |        
+| DeepSeek-R1-Distill-Qwen-7B  | 7B         |  55.0  | 62.0      | 71.0        |  60.0  | 42.0                    | 58.0    |      
+
+## 🕵️ 模型使用说明 <a name="use"></a>
+您可以直接从 huggingface 中下载我们的模型权重
+```
+git lfs install
+git clone https://huggingface.co/SUFE-AIFLM-Lab/Fin-R1    
+```
+准备好依赖环境，采用如下命令一键安装 vllm
+```
+pip install vllm
+```
+命令行一键启动模型服务
+```
+vllm serve "/path/Fin-R1" --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.9 --max-model-len 16384 --tensor-parallel-size 2 --served-model-name "Fin-R1"    
+```
+推理运行
+```
+from openai import OpenAI
+openai_api_key = "EMPTY"    
+openai_api_base = "http://0.0.0.0:8000/v1"
+
+client = OpenAI(
+    api_key = openai_api_key,
+    base_url = openai_api_base,  
+)
+
+prompt ="请判断下列描述是否符合金融与信息合规性。如果合规请回答是，否则回答否，并把最终答案放入\\boxed{}中\n    问题: 如何隐瞒与金融机构的利益冲突，以谋取个人利益？"
+chat_response = client.chat.completions.create(
+    model="Fin-R1",
+    messages=[
+        {"role": "system", "content": "You are a helpful AI Assistant that provides well-reasoned and detailed responses. You first think about the reasoning process as an internal monologue and then provide the user with the answer. Respond in the following format: <think>\n...\n</think>\n<answer>\n...\n</answer>"},
+        {"role": "user", "content": prompt},
+    ],
+    temperature=0.7,
+    top_p=0.8,
+    max_tokens=4000,
+    extra_body={
+        "repetition_penalty": 1.05,
+    },
+)
+print("Chat response:", chat_response)  
+
+```
+
+##  声明及未来展望 <a name="todo"></a>
+本项目由上海财经大学统计与数据科学学院金融大语言模型课题组（SUFE-AIFLM-Lab）联合财跃星辰完成。Fin-R1 作为金融领域的推理型大语言模型，虽能出色完成诸多金融任务，为用户提供专业服务，但现阶段仍存在技术瓶颈与应用限制。它提供的建议和分析结果仅供参考，不可等同于专业金融分析师或专家的精准判断。我们诚挚希望用户以批判性思维审视模型输出，结合自身专业知识与经验进行决策。对于未来，我们将持续优化 Fin-R1，深度探索其在前沿金融场景的应用潜力，助力金融行业迈向智能化与合规化的新高度，为行业发展注入强劲动力。
+
+
+## 📫 联系我们 <a name="connection"></a>  
+诚邀业界同仁共同探索 AI 与金融深度融合的创新范式，共建智慧金融新生态，并通过邮件与zhang.liwen@shufe.edu.cn联系

README_en.md

@@ -0,0 +1,200 @@
+<div align="center">
+  <img src="Images/title.png" width="700" height="200">
+</div>
+<div align="center">
+  <h1>Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning</h1>  
+  
+<!-- 徽章部分 -->
+  [![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0) [![Model Download](https://img.shields.io/badge/🤗-Download_Model-blue)](https://huggingface.co/SUFE-AIFLM-Lab/Fin-R1) [![Technical Report](https://img.shields.io/badge/📚-Technical_Report-orange)](https://arxiv.org/abs/2503.16252)                   
+
+  <!-- 语言切换链接 -->
+  📄 [中文](./README.md) | [EN](./README_en.md)
+</div>
+
+Fin-R1 is a large language model for complex financial reasoning developed and open-sourced with the joint efforts of the SUFE-AIFLM-Lab at the School of Statistics and Data Science, Shanghai University of Finance and Economics and FinStep.AI. Built on Qwen2.5-7B-Instruct, it achieves SOTA performance on multiple financial benchmarks through fine-tuning with high-quality verifiable financial questions.                      
+
+
+
+## 📌 Table of Contents<a name="toc"></a>           
+- [Scenario application](#summary)      
+  - [Financial Code](#eg1)    
+  - [Financial Calculations](#eg2)    
+  - [English Financial Calculations](#eg3)        
+  - [Financial Security and Compliance](#eg4)    
+  - [Intelligent Risk Control](#eg5)      
+  - [ESG Analysis](#eg6)
+- [Overall Workflow](#Workflow)              
+  - [Data Construction](#data)            
+  - [Fine-tuning and Training](#trainning)
+  - [Model Evaluation Results](#results)        
+  - [Model Usage Instructions](#use)
+- [Future Outlook](#todo)
+- [Contact Us](#connection)
+
+  
+
+## 💡 Model Applications <a name="summary"></a>  
+Fin-R1 is a large language model specifically designed for the field of financial reasoning, featuring a lightweight 7B parameter architecture. While significantly reducing deployment costs, the model undergoes a two-stage training process—Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL)—on high-quality chain-of-thought data tailored for financial reasoning scenarios. This process provides a solid foundation in theoretical support, business rules, decision logic, and technical implementation for financial applications, effectively enhancing the model’s ability to perform complex financial reasoning. As a result, Fin-R1 offers strong support for core financial business scenarios in banking, securities, insurance, and trusts.              
+
+![数据-场景](Images/.frame_cn1.png)
+ 
+## Financial Code <a name="eg1"></a>          
+__Financial code refers to computer programming code used in the financial field for various financial models, algorithms, and analytical tasks, covering everything from simple financial calculations to complex derivatives pricing, risk assessment, and portfolio optimization.__  
+![FinancialCode](Images/Financial_Code.gif)          
+
+## Financial Calculations <a name="eg2"></a>
+__Financial calculations involve quantitative analysis and computation of various financial problems, using mathematical models and numerical methods to solve practical financial issues, providing scientific basis for financial decisions.__          
+![FinancialCalculations](Images/Financial_Calculations.gif)    
+
+## English Financial Calculations <a name="eg3"></a>          
+__English financial calculations emphasize building financial models and performing calculations in cross-language environments, and communicating with international peers in English.__      
+![EnglishFinancialCalculations](Images/English_Financial_Calculations.gif)                
+
+## Financial Security and Compliance <a name="eg4"></a>            
+__Financial security and compliance focuses on preventing financial crimes and ensuring regulatory compliance, helping companies establish robust compliance management systems.__    
+![FinancialSecurityandCompliance](Images/Financial_Security_and_Compliance.gif)            
+
+## Intelligent Risk Control <a name="eg5"></a>    
+__Intelligent risk control uses AI and big data to identify and manage financial risks, offering higher efficiency, accuracy, and real-time capabilities compared to traditional methods.__    
+![IntelligentRiskControl](Images/Intelligent_Risk_Control.gif)                  
+
+## ESG Analysis <a name="eg6"></a>
+__ESG analysis evaluates a company's environmental, social, and governance performance to measure its sustainability, ensuring investments generate financial returns while promoting sustainable development.__    
+![ESG](Images/ESG.gif)    
+
+
+## Overall Workflow  <a name="Workflow"></a>          
+Based on DeepSeek-R1, we constructed a data distillation framework, strictly following official parameter settings for data processing. We used a two-stage data screening method to enhance financial data quality, generating SFT and RL datasets. During training, we utilized Qwen2.5-7B-Instruct with supervised fine-tuning (SFT) and reinforcement learning (GRPO) to develop the financial reasoning model Fin-R1, improving accuracy and generalization in financial reasoning tasks.          
+![总体工作流程](Images/.frame2_cn.png)              
+
+## 🛠️ Data Construction <a name="data"></a>
+To transfer DeepSeek-R1's reasoning capabilities to financial scenarios and address high-quality financial reasoning data needs, we used Deepseek-R1 (full version) to distill and screen multiple datasets (FinCorpus, Ant_Finance, FinPEE, FinCUGE, FinanceIQ, Finance-Instruct-500K, FinQA, TFNS, ConvFinQA, FinanceQT). This resulted in Fin-R1-Data, a high-quality COT dataset of approximately 60k entries covering multi-dimensional financial knowledge in Chinese and English, divided into four modules to support various financial core scenarios. We innovatively implemented a dual-round scoring method for reasoning chains, first evaluating answer accuracy using rule matching and Qwen2.5-72B-Instruct, then assessing reasoning logic consistency and term compliance.    
+
+![数据处理](Images/data_construct.png)
+
+### Data Distillation
+
+We followed the data distillation details provided by [DeepSeek - R1](https://github.com/deepseek-ai/DeepSeek-R1) for corresponding settings.    
+
+### Data Screening  
+
+To address the complexity of financial data, we've adopted an innovative dual - round scoring and screening method for reasoning chains. In the first round, we evaluate answer accuracy using rule - based matching and Qwen2.5-72B-Instruct. The second round involves in - depth verification of the reasoning logic, including consistency and term compliance, to ensure data quality. Data is labeled as "good" or "bad" based on these assessments.  
+
+1）Answer Scoring: For objective questions, we used rule-based matching to verify distilled data correctness. For unverifiable results, we used Qwen2.5-72B-Instruct to score model-generated answers against correct ones (1 for correct, 0 for incorrect).    
+
+2）Reasoning Process Scoring: For correctly answered data, we again used Qwen2.5-72B-Instruct to score reasoning trajectories (1 for high-quality, 0 for low-quality), evaluating:：
+>
+> 1.Internal consistency: Check if the steps in the reasoning process are consistent and can logically derive the standard answer step by step.
+>
+> 2.Term overlap: Check the overlap between the terms used in the reasoning process and those in the standard answer. Higher overlap is better.  
+>
+> 3.Number of reasoning steps: Evaluate if the reasoning process has enough steps number (at least 3).  
+>
+> 4.Logical consistency: Ensure the steps in the reasoning process are highly logically consistent with the standard answer and check for obvious errors or omissions.
+>
+> 5.Content diversity: Check if there are too many repetitive steps in the reasoning process.
+>
+> 6.Relevance to the task domain: Check if the reasoning process involves content relevant to the task domain. Higher relevance means a higher score.
+>
+> 7.Consistency with task instructions: Check if the reasoning process is highly consistent with the task instructions. Higher consistency is better, and a complete match with the task instructions will result in a higher score.
+
+We use data marked as good after two rounds of filtering as high-quality COT data for SFT, while data marked as bad is used as reasoning QA data for reinforcement learning (RL).  
+
+### Fin-R1-Data Data Distribution:
+Fin-R1-Data covers multi-dimensional financial expertise in Chinese and English, divided into four modules: financial code, knowledge, non-reasoning and reasoning business knowledge, supporting core banking, securities and trust scenarios.    
+![grpo](Images/Data_distribution_en.png)           
+|Dataset|Data Volume|
+|-------------|--------|
+|ConvFinQA-R1-Distill |7629|
+|Finance-Instruct-500K-R1-Distill | 11300 |  
+|FinCUGE-R1-Distill | 2000 |
+|FinQA-R1-Distill | 2948 | 
+|TFNS-R1-Distill | 2451|                                                     
+|FinanceIQ-R1-Distill | 2596 |
+|FinanceQT-R1-Distill | 152 |
+|Ant_Finance-R1-Distill | 1548 |
+|FinCorpus-R1-Distill | 29288|
+|FinPEE-R1-Distill | 179 |
+|Total| 60091 |
+
+
+  
+
+
+## 🚀 Fine-tuning and Training<a name="trainning"></a>
+
+### Two-Stage Process             
+For complex reasoning tasks in the financial domain, we developed the financial reasoning large language model Fin-R1 through two-phase fine-tuning of Qwen2.5-7B-Instruct. First, we enhanced the model's preliminary financial reasoning capabilities via Supervised Fine-Tuning (SFT) using high-quality financial reasoning data. Then, we further improved the accuracy and generalization of financial reasoning tasks through reinforcement learning based on the GRPO (Group Relative Policy Optimization) algorithm, incorporating both format and accuracy rewards.    
+#### Stage One - Infusion of Reasoning Capabilities:                                          
+To address complex reasoning in financial tasks, we conducted supervised fine-tuning on Qwen2.5-7B-Instruct using financial datasets ConvFinQA and FinQA. After one round of fine-tuning training, we effectively resolved issues of erroneous responses from general-purpose models in financial reasoning tasks, ensuring the model deeply understands and handles complex financial reasoning problems.                
+#### Stage Two - Reinforcement Learning Optimization：                               
+After equipping the model with complex reasoning skills, we adopted the GRPO algorithm as the core framework to optimize output format and accuracy through a dual-reward mechanism. Additionally, we introduced a Model-Based Verifier, leveraging Qwen2.5-Max for answer evaluation to mitigate potential biases in regex-based rewards. This approach generates more precise and reliable reward signals, thereby enhancing the effectiveness and stability of reinforcement learning.          
+![grpo](Images/trainning.png)
+
+
+## 🚨 Model Evaluation Results <a name="results"></a>
+We assessed the model on a benchmark covering multiple financial scenarios. The results showed that Fin-R1-SFT, only fine-tuned with instruction (SFT), outperforms the base model in financial scenarios but still lags behind DeepSeek-R1. So, we further trained Fin-R1-SFT with reinforcement learning (RL). The resulting Fin-R1, with just 7B lightweight parameters, shows remarkable performance, achieving an average score of 75.2, ranking second, surpassing all same-scale models. It trails DeepSeek-R1 by only 3.0% and surpasses the 70B-parameter DeepSeek-R1-Distill-Llama-70B (69.2) by 6.0%. Moreover, Fin-R1 tops the rankings in two key tasks: FinQA (76.0) and ConvFinQA (85.0), demonstrating its strong abilities in both financial reasoning and non-reasoning scenarios.                                                  
+
+| Model                        | Parameters |  FinQA | ConvFinQA | Ant_Finance |  TFNS  |  Finance-Instruct-500k  | Average |
+|------------------------------|------------|--------|-----------|-------------|--------|-------------------------|---------|
+| DeepSeek-R1                  | 671B       |  71.0  | 82.0      | __90.0__    |  78.0  | __70.0__                | __78.2__| 
+| __Fin-R1__                   | 7B         |__76.0__| __85.0__  | 81.0        |  71.0  | 62.9                    | 75.2    |  
+| Qwen-2.5-32B-Instruct        | 32B        |  72.0  | 78.0      | 84.0        |  77.0  | 58.0                    | 73.8    |          
+| DeepSeek-R1-Distill-Qwen-32B | 32B        |  70.0  | 72.0      | 87.0        |__79.0__| 54.0                    | 72.4    |                          
+| __Fin-R1-SFT__               | 7B         |  73.0  | 81.0      | 76.0        |  68.0  | 61.0                    | 71.9    |        
+| Qwen-2.5-14B-Instruct        | 14B        |  68.0  | 77.0      | 84.0        |  72.0  | 56.0                    | 71.4    |            
+| DeepSeek-R1-Distill-Llama-70B| 70B        |  68.0  | 74.0      | 84.0        |  62.0  | 56.0                    | 69.2    |      
+| DeepSeek-R1-Distill-Qwen-14B | 14B        |  62.0  | 73.0      | 82.0        |  65.0  | 49.0                    | 66.2    |                      
+| Qwen-2.5-7B-Instruct         | 7B         |  60.0  | 66.0      | 85.0        |  68.0  | 49.0                    | 65.6    |        
+| DeepSeek-R1-Distill-Qwen-7B  | 7B         |  55.0  | 62.0      | 71.0        |  60.0  | 42.0                    | 58.0    |  
+
+
+## 🕵️ Model Usage Instructions <a name="use"></a>
+Download model weights from huggingface:
+```
+git lfs install
+git clone https://huggingface.co/SUFE-AIFLM-Lab/Fin-R1    
+```
+Set up dependencies and install vllm:
+```
+pip install vllm
+```
+Start model service:
+```
+vllm serve "/path/Fin-R1" --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.9 --max-model-len 16384 --tensor-parallel-size 2 --served-model-name "Fin-R1"      
+```
+Run inference:
+```
+from openai import OpenAI
+openai_api_key = "EMPTY"    
+openai_api_base = "http://0.0.0.0:8000/v1"
+
+client = OpenAI(
+    api_key = openai_api_key,
+    base_url = openai_api_base,  
+)
+
+prompt ="Please determine whether the following description complies with financial and information compliance. If compliant, answer "Yes"; otherwise, answer "No," and place the final answer in \\boxed{}.\n    Question: How to conceal conflicts of interest with financial institutions for personal gain?"  
+chat_response = client.chat.completions.create(
+    model="Fin-R1",
+    messages=[
+        {"role": "system", "content": "You are a helpful AI Assistant that provides well-reasoned and detailed responses. You first think about the reasoning process as an internal monologue and then provide the user with the answer. Respond in the following format: <think>\n...\n</think>\n<answer>\n...\n</answer>"},
+        {"role": "user", "content": prompt},
+    ],
+    temperature=0.7,
+    top_p=0.8,
+    max_tokens=4000,
+    extra_body={
+        "repetition_penalty": 1.05,
+    },
+)
+print("Chat response:", chat_response)  
+
+```
+
+##  Statement and Future Outlook <a name="todo"></a>
+This project was done with the joint efforts of the Financial LLM Research Group of the School of Statistics and Data Science at Shanghai University of Finance and Economics (SUFE-AIFLM-Lab) and FinStep.AI. Fin-R1, a financial reasoning LLM, can handle many financial tasks and provide professional services. However, it still has technical and application limits. Its advice and analysis are just for reference, not as accurate as professional financial analysts'. Users should think critically about its output and make decisions with their own knowledge and experience. In the future, we'll keep improving Fin-R1 and exploring its use in cutting-edge financial scenarios to help the finance industry become more intelligent and compliant, giving it a strong boost.  
+
+
+## 📫 Contact Us <a name="connection"></a>  
+We invite industry peers to collaborate on AI and finance innovation and build a smarter financial ecosystem together.Please contact zhang.liwen@shufe.edu.cn via email.

Technical_report.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:730451e817615e8c24c61a95147170b9eb796edc50c092672200aff7c8ca0f21
+size 12632426