Upload LlamaForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "/root/alpaca/lora-llama3-70b-incontext-split1/42/checkpoint-165",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.2",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.46.2"
+}

model-00001-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e95905f6712a99b0d3adad80f75419ef707b44be05426fd38d6d6b16d3152cb
+size 4806672984

model-00002-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bec55df30b500cce8f80cd0afc213d10db87172b976945c5d9cf5cdc44d83afe
+size 4362142864

model-00003-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e84909f8673d1117ddf3d69c1a458821ba98a4f34e9200ce5fa82721c795a5e6
+size 4362142864

model-00004-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa24c5a4effc20659a37400cd4cf5174634df9bc8e3fad5eb2966e695f352f3b
+size 4966188864

model-00005-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ccff0cfa3b94fdc348fae95a4a9fa36f987df419ed7acc1208f73c18ea3f1b8
+size 4362142864

model-00006-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:871af7cbcce63eb67e80c434771179b4e71ee8d3e4dd40dd82353f7f4809dc63
+size 4362142864

model-00007-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2beaaca55c267bca3aaea245dcbb6786d4ba13f0dd4def900cd04cf696132570
+size 4966188864

model-00008-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0b3076b7c88d6ca88fe4d42f77b202219c1dfafe67bafa79d95a79dcfbda9f0
+size 4362142864

model-00009-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5caa4eb7692b4243ef4d95acdae4af3dae31ac512b20d9327f5fa517d39188d5
+size 4362142880

model-00010-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68bc6c1a34e4ddda4e61691255f12624fa34032545fa0e94cb2ac54333aa324b
+size 4966188880

model-00011-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71b2cec8534df3dbe47457fd036dc274c88210399cdcb84ab693cd1ccfd49501
+size 4362142872

model-00012-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f98e39737908ef53145ca46ff50f35bb51b05bb2032c73a8c512a87ff81fce32
+size 4362142872

model-00013-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7104c2f924fe59fed79d10a0f4fa6aad15f2a77d987ffb764d6e87f508391949
+size 4966188880

model-00014-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3722ccc1023581abd5994f59c1515c6ae04552a026f7ad7762774fa3d98157f4
+size 4362142872

model-00015-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83c9476c756d4dd7cb300e8fd768bcf88cfaf84ccff7d77da3cfbff95578879f
+size 4362142872

model-00016-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d1e98ea46de24f1e29703e6fbbd16d25a6985c9638a11c550da33bcf6ad0ef3c
+size 4966188880

model-00017-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a8bc24e9c3b5c1c7b00f56ab09819607479f6095194c013a0f7235c7c46612a
+size 4362142872

model-00018-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2558122b62aa5fe9097c98b39f622a16e948668e9d830b67581ad8ca5526938d
+size 4362142872

model-00019-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54fd77b551c2eadbc986014d86175dcdfbd53c9cc83c28b77460685e742e0e9c
+size 4966188880

model-00020-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff5f53b0024cae8006b87e00b46ab509ec56f66123711288c842f99f8cd4e051
+size 4362142872

model-00021-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b04563f9464336eb2822a404983e25c4d4639f71e9a9a707339407f92f4b551
+size 4362142872

model-00022-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd9bba26ddccb263655a39edf1940cd4edbc9cdb80b3634c78c6e3359789e532
+size 4966188880

model-00023-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:066b447337579ab99b151d32916e638fb15b6159b2e4978069f06bad8d93670a
+size 4362142872

model-00024-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b30e776f72bd290cc590ce8183e98ba4a7c81f68d54087134c4d2ddaa85744a0
+size 4362142872

model-00025-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e40da726b01578d69e2fae2200ed2c7e98b9c1b1f9860569039ccc2136f5519
+size 4966188880

model-00026-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f67af56ab6b7c0ce91a0f724df8a2534bd5a950c971541b39650ea864a15e772
+size 4362142872

model-00027-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5661c6f7ee240e0eb95e86e4e8ea8126fd73ceb349ec8ee4aa431d0a98f7b092
+size 4362142872

model-00028-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9f8a277b704ab1b3d3b06bf7f7e563508cad8adaacaee889752e0ff99ef932d
+size 4966188880

model-00029-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c58b445b6c7cd35df1945e4c85bab49dd62cf764ad4f53f3a5e3aade3095f58
+size 4362142872

model-00030-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87e799561c998adc2b879b9f8381096fb768f04ef230889fc20b129daeb5a767
+size 4362142872

model-00031-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:146c5bdd8c7ca1c03aa99513076856b40bbb5215d64c7b509bd2fb95ae5e667b
+size 4966188880

model-00032-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fbc4588ddb94d91dbccb1bc54c06b0e0f03a20b845a30a652965a26230b97c0
+size 4362142872

model-00033-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:324bf2775304bfb43fe70e30dadb79f6bd601240abc2a9092d90231e040da5ba
+size 4362142872

model-00034-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0e885373a3acb8700a1ddc5437808994e7a94b3fb2ed920cb5fb05899025c75
+size 4966188880

model-00035-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be1846bbbf634553a19a05b08ce8ee77933bc4898939158628ebd8d34d825d9f
+size 4362142872

model-00036-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8292fdd5a0474b75f4e5debe2831e5c05968784a9934c723a5dab441aad93c38
+size 4362142872

model-00037-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9263bcf17c00061b4c187a233b341b41d2c8b933768d5cdddbdf07983174a246
+size 4966188880

model-00038-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea0a7806767f04e3b5f9b6babced8ee9894490e3325a2a12ccea02b9a11e3f2a
+size 4362142872

model-00039-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5971d837bc5b3f4fd7127a3b719fdeb63bba3e49416267a1c1669f9d668916d1
+size 4362142872

model-00040-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:023d7cd10f27600e02538b68f63a50fc7f2100ca641df126349e7dac79d0b0be
+size 4966188880

model-00041-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e0c235592606f552ca71f8b05af3c87a3e4e6dac644ff9d741395aeee41eb17
+size 4362142872

model-00042-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b698536616550e991feb8195ab93172a9e9ed3c3a68831eaac7eca623359fb3d
+size 4362142872

model-00043-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:177cd52eaed575d44ba305ab6fddd7c9fc52843a7f9ea59d531c92c0862a4f09
+size 4966188880

model-00044-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6c496161552694e7dd6dd1f168ac694b01a77492d87c68639b0fcca6a34032e
+size 4362142872

model-00045-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b99cb116189aa8f558239051bc899090a03ecd060f5c008c1bbdfe19220c5107
+size 4362142872

model-00046-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:331906fcbb1b38aeb328c350025dd54aa86eea9c8007478f7b7fc5f0f1d93e40
+size 4966188880

model-00047-of-00062.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16ae0a0f70a529783736e6facd83ced0e4363698ad904bde14019681be93fc49
+size 4362142872