EA-BERT: An Efficient Ensemble Attention-Based BERT Framework for Sentiment Analysis in Text-Based Feedback Systems

Abstract

Sentiment analysis of text-based customer feedback has emerged as a critical capability for modern organizations seeking to understand user experience at scale. Traditional machine learning and rule-based approaches often struggle with nuanced language, domain-specific vocabulary, and contextual polarity. This paper proposes EA-BERT (Ensemble Attention BERT), an efficient framework that augments a fine-tuned BERT backbone with a lightweight ensemble attention mechanism and TF-IDF feature fusion to classify feedback into positive, neutral, and negative sentiment categories. Evaluated on a dataset of 48,000 real-world customer feedback records drawn from e-commerce, healthcare, and hospitality domains, EA-BERT achieves 94.8% accuracy and a macro F1-score of 93.7%, outperforming BERT by 3.6 percentage points while reducing inference time by 34.5% through model quantization and attention pruning. The proposed architecture demonstrates that strategic feature fusion and ensemble attention mechanisms can simultaneously improve accuracy and computational efficiency, making EA-BERT suitable for deployment in production feedback analysis pipelines.