Utilizing RoBERTa and XLM-RoBERTa pre-trained model for structured sentiment analysis

Abstract

The surge in internet usage has amplified the trend of expressing sentiments across  various  platforms,  particularly  in  e-commerce.  Traditional  sentiment analysis  methods,  such  as aspect-based  sentiment  analysis (ABSA)  and targeted   sentiment   analysis,   fall   short   in   identifying   the   relationships between    opinion    tuples.    Moreover,    conventional    machine    learning approaches often yield inadequate results. To address these limitations, this study  introduces  anapproach  that  leverages  the  attention  values  of  pre-trained  RoBERTa  and  XLM-RoBERTa  models  for structured  sentiment analysis. This   method   aims   to   predict   all   opinion   tuples   and   their relationships   collectively,   providing   a   more   comprehensive   sentiment analysis.  The  proposed  model  demonstrates  significant  improvements  over existing  techniques,  with  the  XLM-RoBERTa  model  achieving  a  notable sentiment  graph F1(SF1)score  of  64.6%  on  the  OpeNERENdataset. Additionally,  the  RoBERTa  model  showed  satisfactory performance  on  the multi-perspective  question  answer (MPQA)and  DSUnisdatasets,  with  SF1 scores of 25.3% and 29.9%, respectively, surpassing baseline models. These results  underscore  the  potential  of this  proposedapproach  in  enhancing sentiment  analysis  across  diverse  datasets,  making  it  highly  applicable  for both academic research and practical applications in various industries.