This report conducts an in-depth sentiment analysis on Elden Ring user reviews, leveraging data extracted from the top 100 games on the Steam Spy API and detailed reviews from the SteamPowered API. The study employs a dual approach: manual sentiment analysis using a curated word list from Kaggle and automated analysis with the Bing and AFINN dictionaries from the Syuzhet package in R. These methods aim to dissect player satisfaction, highlighting elements in the game that generate positive or negative responses. The analysis extends to comparing the outcomes of these methodologies to understand their similarity and differences in sentiment interpretation. Additionally, an emotion analysis using the NRC function from the Syuzhet package is performed to delve deeper into the emotional variations within the reviews. The objective of this comprehensive analysis is to decode the emotional undercurrents of the reviews, thereby offering an insightful perspective on the gaming community's reception of Elden Ring and enriching our understanding of player feedback.
\section{Data Cleaning}
\section*{Data Cleaning}
The data cleaning process for the 'gamereviews' dataset was precisely executed to prepare the user reviews for detailed analysis. Initially, regular expressions were utilized to remove non-ASCII characters and URLs, standardizing the textual content and eliminating irrelevant external references. This step also included stretching out shortened words to their full form to keep the language consistent and reducing repeated letters or symbols in words. Further cleaning steps included the systematic removal of numerical digits, punctuation, and single-character responses, enhancing the data's relevance and readability.\\
Text normalization was a key aspect, with conversions to lowercase and the elimination of excess whitespace using regular expressions, thereby ensuring uniformity across the dataset. Stop words, barring the term "not," were strategically filtered out to focus on meaningful content, crucial for sentiment analysis. Additionally, the process involved spell checking against an English dictionary, reinforcing the textual accuracy and reliability. The cleaning process concludes with the generation of a word cloud, visually representing the most prevalent words in the refined reviews. This step provided a brief overview of dominant themes and insights, forming a foundation for subsequent in-depth textual analysis.
\section{Sentiment Analysis}
\section*{Sentiment Analysis}
The "rev.sentiment" dataset features sentiment scores from both manual analysis and automated methods using the Syuzhet package's Bing and AFINN dictionaries. A manual sentiment analysis of the dataset classified 4481 reviews as positive, 1905 as negative, and 3232 as neutral. The higher count of positive reviews suggests a generally favourable sentiment among users, while the presence of negative and neutral reviews indicates a diverse range of opinions. \\
@@ -116,11 +116,23 @@ The comparison between manual sentiment analysis using a Kaggle-sourced dictiona
The correlations between score.manual and both score.bing (approximately 0.184) and score.afinn (approximately 0.187) are relatively lower. This indicates that the manual sentiment scoring method does not align as closely with the automated methods. There is a moderate positive correlation between score.bing and score.afinn (correlation coefficient approximately 0.823). This suggests that these two scoring methods, though different, often agree on the sentiment of a review. The moderate to low correlations suggest that while there is some agreement between the different methods of sentiment analysis, each brings a unique perspective to the interpretation of the user reviews. This diversity in analysis methods can provide a more comprehensive understanding of user sentiment toward the game.\\
This table presents the correlation coefficients between the different sentiment score variables, with 1.00 indicating a perfect positive correlation and lower values indicating weaker correlations.\\
The table displays correlation coefficients between sentiment score variables, where 1.00 denotes perfect positive correlation and lower values signify weaker correlations.\\
\section{Emotion Analysis}
\section*{Emotion Analysis}
In the emotion analysis conducted using the NRC dictionary from the Syuzhet package, a bar graph was generated to visualise the frequency of various emotions extracted from user reviews. This graph provides a quantitative representation of the emotional landscape, highlighting the prevalence of certain feelings within the feedback.\\
@@ -131,20 +143,31 @@ In the emotion analysis conducted using the NRC dictionary from the Syuzhet pack
\label{fig:emotions_plot}
\end{figure}
The bar chart showcasing emotion scores from NRC sentiment analysis indicates a predominant positive sentiment within the text, as 'Positive' emotion scores highest. This points to a generally favourable tone in the reviews. Emotions of 'Anticipation' and 'Joy' are also prominent, suggesting a sense of eagerness and happiness among the responses. Conversely, the 'Negative' emotion is present but not as dominant as positive feelings, hinting at some elements of dissatisfaction. The spectrum of emotions captured includes 'Trust', 'Fear', 'Sadness', 'Anger', 'Surprise', and 'Disgust', with 'Trust' and 'Fear' being more significant than the others. Notably, 'Surprise' and 'Disgust' score the lowest, indicating that these feelings are less frequently expressed in the dataset.\\
The NRC sentiment analysis bar chart reveals a predominantly positive sentiment in user reviews, with high scores in 'Positive', followed by 'Anticipation' and 'Joy', suggesting a favorable and enthusiastic response. While 'Negative' emotions are present, they are less dominant than positive ones, indicating some dissatisfaction .The emotional spectrum in user reviews, as revealed by the NRC analysis, is diverse, with 'Trust' and 'Fear' being predominant, while 'Surprise' and 'Disgust' are less common. This pattern suggests an overall optimistic tone amidst a range of emotions, where negative feelings are less frequent, indicating nuanced emotional responses. It is important to note that the scores are not mutually exclusive, meaning that a single review can contain multiple emotions.\\
From this plot, one can infer that the overall emotional tone of the user reviews is optimistic, with a diverse yet lesser occurrence of negative emotions, reflecting a more nuanced emotional response to the subject matter. It is important to note that the scores are not mutually exclusive, meaning that a single review can contain multiple emotions.\\
The relevance of a manual sentiment score and the allocation of being positive or negative review consistency was conducted on a 'sent.compare' dataset, and the results are summarised in the confusion matrix shown in below mentioned table.
Furthermore, the relevance of a manual sentiment score and the allocation of being positive or negative review consistency was conducted on a 'sent.compare' dataset, and the results are summarised in the confusion matrix shown in below mentioned table.
\caption{Confusion Matrix for Sentiment Analysis}\end{table}
\begin{table}[ht]
\centering
\begin{tabular}{rrr}
\hline
& FALSE & TRUE \\
\hline
Negative & 275 & 1630 \\
Neutral & 170 & 3062 \\
Positive & 193 & 4288 \\
\hline
\end{tabular}
\caption{Confusion Matrix for Sentiment Analysis}
\end{table}
In analysing the presented confusion matrix detailing sentiment analysis results, the analysis demonstrates strong performance in identifying positive sentiments (True Positives: 4288). Challenges emerge in distinguishing negative and neutral sentiments, evident in higher counts of False Positives (1630) and False Negatives (170). The True Negative count (275) indicates accurate identification of negative sentiments. Additional metrics reveal 3337 reviews of correct agreement out of 9618 observations, resulting in an overall percentage agreement of approximately 34.70\%, offering valuable insights into the analysis's effectiveness in capturing sentiment patterns within the dataset.\\
The provided confusion matrix details the correlation between sentiment classifications (Negative, Neutral, Positive) and user votes (TRUE for upvoted, FALSE for not upvoted) in sentiment analysis. It shows that out of the Negative sentiment reviews, 275 were not upvoted, while 1630 were; for Neutral sentiment, 170 reviews weren't upvoted versus 3062 that were; and in Positive sentiment, 193 reviews didn't get upvotes, contrasted with a significant 4288 that did. This pattern suggests a higher likelihood of upvotes for reviews with Positive and Neutral sentiments compared to Negative ones, indicating user preferences and perceptions towards these sentiment categories.\\
\begin{minipage}{0.5\textwidth}
\centering
@@ -175,7 +198,7 @@ In analysing the presented confusion matrix detailing sentiment analysis results
\vspace{10mm}%
Moreover, the dataset exhibits a prevailing positivity, comprising 4,481 positive, 3,232 neutral, and 1,905 negative sentiments. An intriguing analysis of review helpfulness, as measured by 'votes\_up,' reveals compelling trends. On average, negative reviews amass 1.07 votes, neutral reviews garner 1.46 votes, and positive reviews accumulate 0.87 votes. Surprisingly, neutral reviews stand out as the most beneficial, implying that users highly appreciate well-balanced perspectives. Noteworthy is the revelation that the review with the highest helpfulness, bearing a sentiment label of neutral, secured the remarkable rank of 1412 among all reviews, positioning itself at index 1942. These findings provide concise yet insightful glimpses into sentiment prevalence and user engagement within the dataset. Furthermore, the average sentiment score was 0.2678, which is a positive value but doesn't indicate strong sentiment. In contrast, the overall review score for the game was 8, which is quite high on a typical 1-10 scale. This suggests that the average sentiment score does not strongly resemble the overall review score, potentially due to the limitations of sentiment analysis in capturing nuanced user opinions or the aggregation method of the overall score.\\
Moreover, the provided data showcases two aspects of user reviews based on sentiment. Table 3, "Average Helpful Votes by Sentiment," reveals that Negative, Neutral, and Positive sentiment reviews receive average helpful votes of 1.07, 1.46, and 0.87, respectively. This indicates that Neutral reviews are considered the most helpful, followed by Negative, with Positive reviews deemed least helpful on average. Table 4, "Distribution of Sentiments," shows the prevalence of each sentiment category, with 1905 Negative, 3232 Neutral, and 4481 Positive reviews. This demonstrates that Positive reviews are the most frequent, yet they don't align with the highest helpfulness, which is attributed to Neutral reviews. The data suggests a trend where users may find Neutral reviews more balanced and informative than the more extreme Positive or Negative ones..\\
\begin{figure}[ht]
\centering
@@ -184,13 +207,13 @@ Moreover, the dataset exhibits a prevailing positivity, comprising 4,481 positiv
\label{fig:sentiment_plot}
\end{figure}
The illustrated plot reflects variations in the total sentiment score over time, characterized by notable fluctuations and a significant upward spike towards the end of September. Such variations likely represent shifts in user opinions influenced by factors including game updates, demographic shifts in the player base, or external events that impacted public perception. The most pronounced peak in sentiment, with a score of 79, was recorded on November 22, 2023. This spike may correlate with a specific event that elicited a strong positive or negative reaction from the gaming community, underscoring the dynamic nature of user sentiment in response to ongoing developments within the game's ecosystem.
The plot shows significant fluctuations in the total sentiment score over time, peaking notably towards the end of September and reaching its highest point, a score of 79, on November 22, 2023. These variations suggest changes in user opinions, possibly due to game updates, demographic changes, or external influences. The significant peak in sentiment score in November suggests an event that markedly influenced the gaming community's opinion, reflecting the dynamic responsiveness of users to changes within the game's environment.
\section{Limitations}
\section*{Limitations}
Sentiment analysis using predefined word lists faces the challenge of context insensitivity, struggling to capture the varied connotations words can have in different situations. The static nature of these lists doesn't account for linguistic evolution, making it hard to detect sarcasm, colloquialisms, or the impact of modifiers like "very" or "not". Consequently, the sentiment expressed may be misinterpreted, leading to an inaccurate assessment of user opinions. The use of automated sentiment analysis with the Syuzhet package's dictionaries in R introduces variability due to differing methodologies and lexical sets. While some dictionaries provide a simple categorisation of sentiments, others assign a numerical value or a range of emotions, which might not reflect nuances specific to contexts like gaming. This variability necessitates a careful selection of analysis tools, understanding that they serve as an initial guide rather than a conclusive evaluation of sentiment.
\section{Conclusion}
\section*{Conclusion}
In summary, the sentiment analysis on the review's dataset has revealed the depth and complexity of user opinions. Alignment exists between manual and automated analysis methods, but each has inherent difficulties in accurately interpreting sentiment, as evidenced by the NRC emotion analysis which depicts a primarily positive sentiment combined with a spectrum of other emotions. Moreover, time-series analysis indicates that user sentiments are not stable, highlighting the dynamic nature of player feedback. However, this analysis also uncovers challenges such as context insensitivity and linguistic evolution, which can lead to misunderstandings. This underscores the need for sentiment analysis tools to evolve in complexity and adaptability to truly reflect the nuanced sentiment of user-generated content.
