Text2GPT 🤖

Text2GPT is a lightweight text generation model fine-tuned from DistilGPT2, with 81.9M parameters, designed for efficient and coherent text generation. It leverages the power of transformers and supports Safetensors for secure model loading. Ideal for creative writing, text completion, and more! 🚀

Features ✨

Generates human-like text with minimal input 📝
Supports Safetensors for safe and efficient loading 🔒
Fine-tuned for low-resource environments ⚡
Compatible with Hugging Face transformers and vLLM 🚀

Installation 🛠️

Install the required dependencies:

pip install transformers torch safetensors

Usage 🎮

Loading the Model with Transformers

Use the Hugging Face transformers library to load and generate text:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and tokenizer
model_name = "kulia-moon/Text2GPT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Generate text
input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using Pipeline for Simplicity

For quick text generation:

from transformers import pipeline

pipe = pipeline("text-generation", model="kulia-moon/Text2GPT")
print(pipe("My name is Julien and I like to", max_length=30, do_sample=True)[0]["generated_text"])

vLLM Deployment for Scalability

Deploy with vLLM for high-throughput inference:

docker run --runtime nvidia --gpus all -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model kulia-moon/Text2GPT

Then, serve the model:

docker exec -it my_vllm_container bash -c "vllm serve kulia-moon/Text2GPT"

Widget Examples 🖱️

Try these prompts on the model page:

"Once upon a time" ⏳
"My name is Julien and I like to" 😊
"Paris is an amazing place to visit," 🗼
"I like traveling by train because" 🚂

Example Output:

Input: "Once upon a time"
Output: "Once upon a time, a curious AI roamed the digital realm, crafting tales of wonder."

Model Details 📊

Architecture: DistilGPT2-based, 6 layers, 81.9M parameters
Base Model: distilbert/distilgpt2
Safetensors: Supported, 81,912,576 parameters (non-sharded, non-quantized)
Intended Use: Text generation, creative writing, dialogue completion
Limitations: May produce biased or repetitive outputs; not optimized for sensitive tasks

Evaluation Report 📈

Evaluation metrics are under development. Preliminary tests suggest performance comparable to DistilGPT2 (perplexity ~21.1 on WikiText-103). Contributions for detailed metrics are welcome via discussions! 🙌