brjoey/CBSI-bert-large-uncased

CBSI-BERT Models

This model is trained on the replication data of Nițoi et al. (2023).
Check out their paper and website for more information.

The model is trained with the hyperparameters used by Nițoi et al. (2023).
In addition, different hyperparameters, seeds, and reinitialization of the first L layers were tested. The performance seems relatively stable across hyperparameter settings.

Alongside these models, FinBERT, different versions of RoBERTa, and EconBERT were tested. The performance of the BERT-based models reported here is significantly better. In addition, fine-tuned ModernBERT and CentralBank-BERT versions are also available.

Results

Model	F1 Score	Accuracy	Loss
CBSI-bert-base-uncased	0.88	0.88	0.49
CBSI-bert-large-uncased	0.92	0.92	0.45
CBSI-ModernBERT-base	0.93	0.93	0.40
CBSI-ModernBERT-large	0.91	0.91	0.53
CBSI-CentralBank-BERT	0.92	0.92	0.36

How to use

import pandas as pd
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

# Load model and tokenizer
model_name = "brjoey/CBSI-bert-large-uncased"
classifier = pipeline(
    "text-classification",
    model=model_name,
    tokenizer=model_name
)

# Define label mapping
cbsi_label_map = {
    0: "neutral",
    1: "dovish",
    2: "hawkish"
}

# Classify a column in a Pandas DataFrame - replace with your DataFrame
texts = [
    "The Governing Council decided to lower interest rates.",
    "The central bank will maintain its current policy stance."
]

df = pd.DataFrame({
    "text": texts
})

# Run classification 
predictions = classifier(
    df["text"].tolist()
)

# Store the results
df["label"], df["score"] = zip(*[
    (cbsi_label_map[int(pred["label"].split("_")[-1])], pred["score"])
    for pred in predictions
])

print("\n                              === Results ===\n")
print(df[["text", "label", "score"]])

---

Citation

If you use this model, please cite:
Data:
Nițoi Mihai; Pochea Maria-Miruna; Radu Ștefan-Constantin, 2023,
"Replication Data for: Unveiling the sentiment behind central bank narratives: A novel deep learning index",
https://doi.org/10.7910/DVN/40JFEK, Harvard Dataverse, V1

Paper:
Mihai Niţoi, Maria-Miruna Pochea, Ştefan-Constantin Radu,
Unveiling the sentiment behind central bank narratives: A novel deep learning index,
Journal of Behavioral and Experimental Finance, Volume 38, 2023, 100809, ISSN 2214-6350.
https://doi.org/10.1016/j.jbef.2023.100809

BERT:
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805. https://arxiv.org/abs/1810.04805