README.md

---
license: apache-2.0
language:
- en
base_model:
- answerdotai/ModernBERT-base
pipeline_tag: text-classification
metrics:
- accuracy
---

# ModernBERT-FakeNewsClassifier

## Model Description

**ModernBERT-FakeNewsClassifier** is a fine-tuned version of [ModernBERT](https://huggingface.co/answerdotai/ModernBERT-base), optimized for the binary classification task of detecting fake news. This model processes news articles, including their titles, text content, subject, and publication date, to classify them as either **real (1)** or **fake (0)**. The model is fine-tuned on a dataset containing over 30,000 labeled examples, achieving high accuracy and robustness.

### Key Features:
- **Base Model**: ModernBERT, designed for long-context processing (up to 8,192 tokens).
- **Task**: Binary classification for fake news detection.
- **Architecture Highlights**:
  - Rotary Positional Embeddings (RoPE) for long-context support.
  - Local-global alternating attention for memory efficiency.
  - Flash Attention for optimized inference speed.

## Dataset

The dataset used for fine-tuning comprises over 30,000 examples, with the following features:
- **Title**: The headline of the news article.
- **Text**: The main body of the article.
- **Subject**: The category or topic of the article (e.g., Politics, Health).
- **Date**: The publication date of the article.
- **Label**: Binary labels indicating whether the article is fake (`0`) or real (`1`).

## Notebook: Training and Fine-Tuning
The repository includes the code.ipynb file, which provides:

- Step-by-step instructions for preprocessing the dataset.
- Fine-tuning the ModernBERT model for binary classification.
- Code for evaluating the model using metrics such as accuracy, F1-score, and AUC-ROC.
- You can directly open and run the notebook to replicate or customize the training process.


## Citation

If you use this model in your research or applications, please cite:

```
@misc{ModernBERT-FakeNewsClassifier,
  author = {Daksh Rathi},
  title = {ModernBERT-FakeNewsClassifier: A Transformer-Based Model for Fake News Detection},
  year = {2024},
  url = {https://huggingface.co/dakshrathi/ModernBERT-base-FakeNewsClassifier},
}