language: en
license: mit
tags:
- pytorch
- causal-lm
- language-model
- flash-attention
---
# PurelyUnfunctionalAI/GibberishGPT
This is a language model trained with Flash Attention. The model is based on a decoder-only transformer architecture.
## Model Details
- **Model Type:** Causal Language Model
- **Embedding Size:** 512
- **Hidden Layers:** 8
- **Attention Heads:** 8
- **Context Length:** 512
- **Flash Attention:** Enabled
## Usage
```python
import tiktoken
from transformers import AutoModelForCausalLM
# Load the tokenizer
tokenizer = tiktoken.get_encoding("gpt2")
# Load the model
model = AutoModelForCausalLM.from_pretrained("PurelyUnfunctionalAI/GibberishGPT")
# Encode input
input_text = "Your prompt here"
input_ids = tokenizer.encode(input_text)
input_tensor = torch.tensor([input_ids], dtype=torch.long)
# Generate
output = model.generate(input_tensor, max_length=100)
generated_text = tokenizer.decode(output[0].tolist())
print(generated_text)
```
## License
This model is available under the MIT License.
