Sentiment Analysis Using RoBERTa-Based Hybrid Model

Abstract: The sentiment analysis of movie reviews is a well-known issue within the Natural Language Processing (NLP) field and finds the primary applications in opinion mining and movie recommendation systems. Over the last few years, a number of studies have investigated the concept of hybrid deep learning structures that integrate transformer encoders with recurrent and structured prediction stack to enhance sentiment classification. The transformer encoder in the given study is substituted with a Robustly Optimized BERT pretraining approach (RoBERTa) and LSTM with a Bidirectional Long Short-Term Memory (BiLSTM) network. The suggested method combines RoBERTa-BiLSTM and a Conditional Random Field (CRF) layer to maintain consistency with base architecture. The suggested framework is tested on the IMDb movie review data set with a structured train validation test splits. The evaluation of performance is done by applying standard classification measures. The model based on RoBERTa achieves an accuracy of 91.01, a precision of 91.43, a recall of 90.50 and an F1-score of 90.97, and the results were higher than the results reported on the Transformer- LSTM-CRF. These results imply that improved contextual representations supplied by the contemporary pre-trained transformers have a positive effect on document-level sentiment classification.

Keywords: Sentiment Analysis, Deep Learning (DL), RoBERTa, BiLSTM, CRF, IMDb Dataset.