Upload folder using huggingface_hub

README.md

@@ -1,108 +1,202 @@
 ---
-language: en
-license: llama3.1
-tags:
-- llama
-- transformer
-- 8b
-- 4bit
-- instruction-tuning
-- conversational
-- llama3
-- meta
-pipeline_tag: text-generation
-inference: true
-model_creator: 0xroyce
-model_type: LLaMA
-datasets:
-- 0xroyce/Plutus
-base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+base_model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
+library_name: peft
 ---
 
-# NOTE: MODEL IS BEING FINE-TUNED WITH DATEST LATEST
+# Model Card for Model ID
 
-# Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit
+<!-- Provide a quick summary of what the model is/does. -->
 
-Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit is a fine-tuned version of the LLaMA-3.1-8B model, specifically optimized for tasks related to finance, economics, trading, psychology, and social engineering. This model leverages the LLaMA architecture and employs 4-bit quantization to deliver high performance in resource-constrained environments while maintaining accuracy and relevance in natural language processing tasks.
 
-![Plutus Banner](https://iili.io/djQmWzu.webp)
 
 ## Model Details
 
-- **Model Type**: LLaMA
-- **Model Size**: 8 Billion Parameters
-- **Quantization**: 4-bit (bnb, bitsandbytes)
-- **Architecture**: Transformer-based
-- **Creator**: [0xroyce](https://huggingface.co/0xroyce)
+### Model Description
 
-## Training
+<!-- Provide a longer summary of what this model is. -->
 
-Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit was fine-tuned on the [**"Financial, Economic, and Psychological Analysis Texts"** dataset](https://huggingface.co/datasets/0xroyce/Plutus), which is a comprehensive collection of 85 influential books out of a planned 398. This dataset covers key areas such as:
 
-- **Finance and Investment**: Including stock market analysis, value investing, and exchange-traded funds (ETFs).
-- **Trading Strategies**: Focused on technical analysis, options trading, and algorithmic trading methods.
-- **Risk Management**: Featuring quantitative approaches to financial risk management and volatility analysis.
-- **Behavioral Finance and Psychology**: Exploring the psychological aspects of trading, persuasion, and psychological operations.
-- **Social Engineering and Security**: Highlighting manipulation techniques and cybersecurity threats.
 
-As the dataset contained only 21.36% of its planned content at the time of training, this version of the model is sometimes referred to as the '21% version.' This fine-tuning process enhances the model's ability to generate coherent and contextually relevant text in domains like financial analysis, economic theory, and trading strategies. The 4-bit quantization ensures that the model can be deployed in environments with limited computational resources without compromising performance.
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
 
-## Intended Use
+### Model Sources [optional]
 
-This model is well-suited for a variety of natural language processing tasks within the finance, economics, psychology, and cybersecurity domains, including but not limited to:
+<!-- Provide the basic links for the model. -->
 
-- **Financial Analysis**: Extracting insights and performing sentiment analysis on financial texts.
-- **Economic Modeling**: Generating contextually relevant economic theories and market predictions.
-- **Behavioral Finance Research**: Analyzing and generating text related to trading psychology and investor behavior.
-- **Cybersecurity and Social Engineering**: Studying manipulation techniques and generating security-related content.
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
 
-## Performance
+## Uses
 
-While specific benchmark scores for Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit are not provided, the model is designed to offer competitive performance within its parameter range, particularly for tasks involving financial, economic, and security-related data. The 4-bit quantization offers a balance between model size and computational efficiency, making it ideal for deployment in resource-limited settings.
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-## Limitations
+### Direct Use
 
-Despite its strengths, the Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit model has some limitations:
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-- **Domain-Specific Biases**: The model may generate biased content depending on the input, especially within specialized financial, psychological, or cybersecurity domains.
-- **Inference Speed**: Although optimized with 4-bit quantization, real-time application latency may still be an issue depending on the deployment environment.
-- **Context Length**: The model has a limited context window, which can affect its ability to process long-form documents or complex multi-turn conversations effectively.
+[More Information Needed]
 
-## How to Use
+### Downstream Use [optional]
 
-You can load and use the model with the following code:
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
+[More Information Needed]
 
-tokenizer = AutoTokenizer.from_pretrained("0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit")
-model = AutoModelForCausalLM.from_pretrained("0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit")
+### Out-of-Scope Use
 
-input_text = "Your text here"
-input_ids = tokenizer(input_text, return_tensors="pt").input_ids
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-output = model.generate(input_ids, max_length=50)
-print(tokenizer.decode(output[0], skip_special_tokens=True))
-```
+[More Information Needed]
 
-## Ethical Considerations
+## Bias, Risks, and Limitations
 
-The Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit model, like other large language models, can generate biased or potentially harmful content. Users are advised to implement content filtering and moderation when deploying this model in public-facing applications. Further fine-tuning is also encouraged to align the model with specific ethical guidelines or domain-specific requirements.
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-## Citation
+[More Information Needed]
 
-If you use this model in your research or applications, please cite it as follows:
+### Recommendations
 
-```bibtex
-@misc{0xroyce2024plutus,
-  author = {0xroyce},
-  title = {Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit},
-  year = {2024},
-  publisher = {Hugging Face},
-  howpublished = {\\url{https://huggingface.co/0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit}},
-}
-```
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-## Acknowledgements
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
-Special thanks to the open-source community and contributors who made this model possible.
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "gate_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebd4fb7173fc7f68194f0fbe4d425731b9981d6cd3649dfc9b268324f5d51883
+size 167832240