ewhk9887 commited on
Commit
168a744
·
verified ·
1 Parent(s): d921872

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +82 -156
README.md CHANGED
@@ -4,199 +4,125 @@ tags:
4
  - unsloth
5
  - trl
6
  - sft
 
 
 
 
 
7
  ---
8
 
9
- # Model Card for Model ID
10
 
11
- <!-- Provide a quick summary of what the model is/does. -->
12
 
 
13
 
 
 
14
 
15
- ## Model Details
 
16
 
17
- ### Model Description
 
 
 
 
18
 
19
- <!-- Provide a longer summary of what this model is. -->
20
 
21
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
 
22
 
23
- - **Developed by:** [More Information Needed]
24
- - **Funded by [optional]:** [More Information Needed]
25
- - **Shared by [optional]:** [More Information Needed]
26
- - **Model type:** [More Information Needed]
27
- - **Language(s) (NLP):** [More Information Needed]
28
- - **License:** [More Information Needed]
29
- - **Finetuned from model [optional]:** [More Information Needed]
30
-
31
- ### Model Sources [optional]
32
-
33
- <!-- Provide the basic links for the model. -->
34
-
35
- - **Repository:** [More Information Needed]
36
- - **Paper [optional]:** [More Information Needed]
37
- - **Demo [optional]:** [More Information Needed]
38
-
39
- ## Uses
40
-
41
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
42
-
43
- ### Direct Use
44
-
45
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
46
-
47
- [More Information Needed]
48
-
49
- ### Downstream Use [optional]
50
-
51
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
52
-
53
- [More Information Needed]
54
-
55
- ### Out-of-Scope Use
56
-
57
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
58
-
59
- [More Information Needed]
60
-
61
- ## Bias, Risks, and Limitations
62
-
63
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
64
-
65
- [More Information Needed]
66
-
67
- ### Recommendations
68
-
69
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
70
-
71
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
72
-
73
- ## How to Get Started with the Model
74
-
75
- Use the code below to get started with the model.
76
-
77
- [More Information Needed]
78
-
79
- ## Training Details
80
-
81
- ### Training Data
82
-
83
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
84
-
85
- [More Information Needed]
86
-
87
- ### Training Procedure
88
-
89
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
90
-
91
- #### Preprocessing [optional]
92
-
93
- [More Information Needed]
94
-
95
-
96
- #### Training Hyperparameters
97
-
98
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
99
-
100
- #### Speeds, Sizes, Times [optional]
101
-
102
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
103
-
104
- [More Information Needed]
105
-
106
- ## Evaluation
107
-
108
- <!-- This section describes the evaluation protocols and provides the results. -->
109
-
110
- ### Testing Data, Factors & Metrics
111
-
112
- #### Testing Data
113
-
114
- <!-- This should link to a Dataset Card if possible. -->
115
-
116
- [More Information Needed]
117
-
118
- #### Factors
119
-
120
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
121
-
122
- [More Information Needed]
123
-
124
- #### Metrics
125
-
126
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
127
-
128
- [More Information Needed]
129
-
130
- ### Results
131
-
132
- [More Information Needed]
133
-
134
- #### Summary
135
-
136
-
137
-
138
- ## Model Examination [optional]
139
 
140
- <!-- Relevant interpretability work for the model goes here -->
141
 
142
- [More Information Needed]
143
 
144
- ## Environmental Impact
 
 
145
 
146
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
 
 
147
 
148
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
149
 
150
- - **Hardware Type:** [More Information Needed]
151
- - **Hours used:** [More Information Needed]
152
- - **Cloud Provider:** [More Information Needed]
153
- - **Compute Region:** [More Information Needed]
154
- - **Carbon Emitted:** [More Information Needed]
155
 
156
- ## Technical Specifications [optional]
 
157
 
158
- ### Model Architecture and Objective
159
 
160
- [More Information Needed]
 
161
 
162
- ### Compute Infrastructure
 
 
163
 
164
- [More Information Needed]
165
 
166
- #### Hardware
167
 
168
- [More Information Needed]
 
 
169
 
170
- #### Software
 
 
171
 
172
- [More Information Needed]
173
 
174
- ## Citation [optional]
 
175
 
176
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
 
177
 
178
- **BibTeX:**
 
179
 
180
- [More Information Needed]
 
 
 
 
 
 
181
 
182
- **APA:**
183
 
184
- [More Information Needed]
 
 
 
185
 
186
- ## Glossary [optional]
187
 
188
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
189
 
190
- [More Information Needed]
 
191
 
192
- ## More Information [optional]
 
 
193
 
194
- [More Information Needed]
195
 
196
- ## Model Card Authors [optional]
 
197
 
198
- [More Information Needed]
199
 
200
- ## Model Card Contact
 
201
 
202
- [More Information Needed]
 
4
  - unsloth
5
  - trl
6
  - sft
7
+ license: mit
8
+ datasets:
9
+ - iamtarun/python_code_instructions_18k_alpaca
10
+ base_model:
11
+ - deepseek-ai/DeepSeek-R1-Distill-Llama-8B
12
  ---
13
 
14
+ # Model Card for Finetuned DeepSeek-R1 Code Review Model
15
 
16
+ ## Model Details / 모델 세부 정보
17
 
18
+ ### Model Description / 모델 설명
19
 
20
+ **English:**
21
+ This model is a finetuned version of the DeepSeek-R1 Distill Llama model, adapted for performing code reviews in Korean. It has been fine-tuned using QLoRA and additional dataset transformations from the [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca) dataset, converting code generation prompts into code review prompts. The LoRA adapters have been merged into the base model to produce a self-contained model that can be deployed directly.
22
 
23
+ **한국어:**
24
+ 이 모델은 DeepSeek-R1 Distill Llama 모델을 기반으로, 한국어 코드 리뷰 작업에 맞게 파인튜닝된 모델입니다. [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca) 데이터셋의 코드 생성 프롬프트를 코드 리뷰 프롬프트로 변환하여 QLoRA 기법을 사용해 파인튜닝하였으며, LoRA 어댑터를 베이스 모델에 병합해 self-contained 형태로 제작되었습니다.
25
 
26
+ - **Developed by / 개발자:** [More Information Needed / 추가 정보 필요]
27
+ - **Model type / 모델 유형:** Causal Language Model with Finetuning for Code Review
28
+ - **Language(s) / 사용 언어:** Korean, English
29
+ - **License / 라이센스:** MIT
30
+ - **Base Model / 베이스 모델:** [deepseek-ai/DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)
31
 
32
+ ### Model Sources / 모델 소스
33
 
34
+ - **Repository / 리포지토리:** [More Information Needed / 추가 정보 필요]
35
+ - **Paper / 논문 (optional):** [More Information Needed / 추가 정보 필요]
36
+ - **Demo / 데모 (optional):** [More Information Needed / 추가 정보 필요]
37
 
38
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
40
+ ## Uses / 사용 용도
41
 
42
+ ### Direct Use / 직접 사용
43
 
44
+ **English:**
45
+ This model is intended for generating code reviews for Python code. It is designed to provide feedback on code quality, style, and possible improvements.
46
+ It is designed as prototype for programming education.
47
 
48
+ **한국어:**
49
+ 이 모델은 Python 코드를 대상으로 코드 리뷰(피드백, 스타일 개선 등)를 생성하기 위해 개발되었습니다.
50
+ 프로그래밍 교육을 위한 모델의 프로토타입으로 개발되었습니다.
51
 
52
+ ### Downstream Use / 다운스트림 사용 (optional)
53
 
54
+ **English:**
55
+ It can be integrated into developer tools, code analysis platforms, or educational environments to assist in code review tasks.
 
 
 
56
 
57
+ **한국어:**
58
+ 개발자 도구, 코드 분석 플랫폼 또는 교육 환경에 통합되어 코드 리뷰 작업을 보조하는 용도로 활용될 수 있습니다.
59
 
60
+ ### Out-of-Scope Use / 지원하지 않는 사용 예시
61
 
62
+ **English:**
63
+ This model is not optimized for generating full code, handling languages other than Python, or for use in critical production environments without human oversight.
64
 
65
+ **한국어:**
66
+ 이 모델은 코드 생성을 위한 모델이 아니며, 현재 데이터셋 상 Python 이외의 언어에 대해서는 최적화되어 있지 않습니다.
67
+ 이후 한국어 코드 리뷰 데이터셋과 Go와 Rust를 포함하는 모델은 추후에 업로드될 예정입니다.
68
 
69
+ ---
70
 
71
+ ## Bias, Risks, and Limitations / 편향, 위험 및 한계
72
 
73
+ **English:**
74
+ - The model has been trained on data that may have inherent biases, and its reviews are generated automatically.
75
+ - This model is not perfectly optimized for Korean language code review.
76
 
77
+ **한국어:**
78
+ - 모델이 생성한 리뷰에도 편향이 있을 수 있습니다.
79
+ - 한국어 코드 리뷰에 완벽하게 최적화되어있지 않을 수 있습니다.
80
 
81
+ ## How to get started with model / 모델 시작하기
82
 
83
+ ```python
84
+ from transformers import AutoTokenizer, AutoModelForCausalLM
85
 
86
+ model_name = "your_hf_username/merged-deepseek-r1-codereview"
87
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
88
+ model = AutoModelForCausalLM.from_pretrained(model_name)
89
 
90
+ prompt = """아래는 작성된 Python 코드입니다.
91
+ 코드의 장단점, 개선 사항, 코드 스타일 등에 대해 3~4줄 정도의 간결한 리뷰를 작성하세요.
92
 
93
+ # 코드:
94
+ ### Python 코드
95
+ def sum_sequence(sequence):
96
+ sum = 0
97
+ for num in sequence:
98
+ sum += num
99
+ return sum
100
 
101
+ ### 코드 리뷰:"""
102
 
103
+ inputs = tokenizer(prompt, return_tensors="pt")
104
+ outputs = model.generate(**inputs, max_length=300)
105
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
106
+ ```
107
 
108
+ ## Training Details / 학습 정보
109
 
110
+ ### Training Data / 학습 데이터
111
 
112
+ English:
113
+ The model was fine-tuned using the iamtarun/python_code_instructions_18k_alpaca dataset. The original code generation prompts were transformed into code review prompts to suit the task.
114
 
115
+ 한국어:
116
+ 이 모델은 iamtarun/python_code_instructions_18k_alpaca 데이터셋을 사용해 파인튜닝되었습니다. 기존의 코드 생성 프롬프트를 코드 리뷰 프롬프트로 변환하여 학습에 사용하였습니다.
117
+ Training Procedure / 학습 절차
118
 
119
+ English:
120
 
121
+ Preprocessing: The dataset was preprocessed to convert the code generation prompts into a standardized code review format.
122
+ Fine-tuning: The base model was fine-tuned using QLoRA with 4-bit quantization for efficiency. LoRA adapters were merged into the base model to produce a self-contained model.
123
 
124
+ 한국어:
125
 
126
+ 전처리: 코드 생성 프롬프트를 코드 리뷰 형식으로 변환하기 위해 데이터셋을 전처리하였습니다.
127
+ 파인튜닝: 효율성을 위해 4비트 양자화를 사용하여 QLoRA 기법으로 베이스 모델을 파인튜닝하였으며, LoRA 어댑터를 병합하여 self-contained 모델로 제작하였습니다.
128