Difference between revisions of "Vol-3194/paper20"

From BITPlan ceur-ws Wiki
Jump to navigation Jump to search
(edited by wikiedit)
 
(modified through wikirestore by wf)
Line 1: Line 1:
 
+
=Paper=
 
{{Paper
 
{{Paper
|wikidataid=Q117345145
+
|id=Vol-3194/paper20
 +
|storemode=property
 +
|title=A Survey of Sentimental Analysis Methods on COVID-19 Research
 +
|pdfUrl=https://ceur-ws.org/Vol-3194/paper20.pdf
 +
|volume=Vol-3194
 +
|authors=Areeba Umair,Elio Masciari
 +
|dblpUrl=https://dblp.org/rec/conf/sebd/UmairM22
 
}}
 
}}
 +
==A Survey of Sentimental Analysis Methods on COVID-19 Research==
 +
<pdf width="1500px">https://ceur-ws.org/Vol-3194/paper20.pdf</pdf>
 +
<pre>
 +
A Survey of Sentimental Analysis Methods on
 +
COVID-19 Research
 +
Areeba Umair1 , Elio Masciari1,2
 +
1
 +
    Department of Electrical Engineering and Information Technologies, University of Naples, Federico II
 +
2
 +
    Institute for High Performance Computing and Networking (ICAR), National Research Council, Naples, Italy
 +
 +
 +
                                        Abstract
 +
                                        In this era of social media, people share anything they feel or experience on social media in the form of
 +
                                        posts or comments. These posts, comments or reviews of the people can be analyzed using sentimental
 +
                                        analysis, which is emerging field in text mining. COVID-19 has people’s life all over the globe and thus
 +
                                        has declared as pandemic. Due to COVID, people are feeling panic, anxiety, rage, sorrow, misery, stress
 +
                                        and other issues. In this review, we have presented the sentimental analysis data sources, approaches,
 +
                                        scenarios, methods and tools by comparing thirty studies. The results illustrated that most researchers
 +
                                        have used SVM and Naive Bayes for sentimental analysis on COVID research. We also concluded that
 +
                                        most of the researchers work on the sentiments of students, reopening sentiments, vaccine sentiments
 +
                                        etc.
 +
 +
                                        Keywords
 +
                                        Social media Big Data, Sentiments related to COVID, Social Media Reviews, Data analytic, COVID-19
 +
 +
 +
 +
 +
1. Introduction
 +
Now-a-days, many people use social networks to express their opinion, thoughts or feedback
 +
about anything, [1]. In this era of technology, almost all the industries provide their customers
 +
with the ability to buy product online and also share their reviews or feedback on their website of
 +
social media pages [2]. This feedback can be positive or negative which can help other customer
 +
in making decision and help the industry to improve the product according to the customer
 +
need [3]. Such kind of review data on internet can be used in extraction of sentiments from the
 +
raw data that can be used for well-being of the society as well as business or organization [4],
 +
[5]. Sentimental analysis is the natural language processing tasks in which text is classified into
 +
positive, negative or neutral sentiments based on their meanings in the sentence. There are three
 +
types of sentimental analysis i.e. document level, sentence level and aspect level sentimental
 +
analysis. In order to gain the fine grain sentimental expression, aspect level sentimental analysis
 +
is used [1]. Let’s take an example to understand aspect level sentimental analysis. "The food
 +
is very tasty but its quality is low". In this example, "very tasty and "low" show two different
 +
sentiments i.e. positive and negative respectively. The traditional sentimental analysis methods
 +
have been eliminated due to advancement in artificial intelligence [3].
 +
 +
SEBD 2022: The 30th Italian Symposium on Advanced Database Systems, June 19-22, 2022, Tirrenia (PI), Italy
 +
$ areeba.umair@unina.it (A. Umair); elio.masciari@unina.it (E. Masciari)
 +
€ https://www.docenti.unina.it/elio.masciari (E. Masciari)
 +
                                      © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
 +
    CEUR
 +
    Workshop
 +
    Proceedings
 +
                  http://ceur-ws.org
 +
                  ISSN 1613-0073
 +
                                      CEUR Workshop Proceedings (CEUR-WS.org)
 +
�  The whole world is facing the biggest challenge in the form of COVID, which has destroyed
 +
the economy of many under-developed countries [6]. Corona-virus was discovered in Wuhan,
 +
China in the month of December 2019 and it has started spreading in the world and thus declared
 +
as pandemic. According to John Hopkins University, 435, 427, 191 people have been affected
 +
due to COVID, thus causing 5, 966, 417 number of deaths till 27 February 2022. People are
 +
facing different psychological problems due to COVID such as anger, depression, fear, and many
 +
others.
 +
  The traditional machine learning methods and deep learning methods are available to resolve
 +
the sentimental classification problems [7], [8]. The traditional ML (machine learning) classifiers
 +
for sentimental classification are Support Vector Machine (SVM) and Naive Bayes however,
 +
deep learning methods for sentimental classification are Recurrent Neural Network (RNN) and
 +
Convolutional Neural Network (CNN). These methods extract meaningful features automatically
 +
[1]. RNN has a recurrent nature due to which it suffers from gradient vanishing problem and
 +
CNN has short-comings for sequential dependencies. Thus, the literature shows the different
 +
issues and limitations in the exiting approaches such as low accuracy and performance and
 +
high complexity [3]. The inconsistent sentimental polarity in the sentence causes the word
 +
dependency to be weaken. In such scenario, attention mechanism can be fruitful for sentimental
 +
classification tasks.
 +
  In this research, we have collected thirty primary studies related to sentimental analysis with
 +
respect to COVID-19 and performed the survey. The purpose of the survey was to identify the
 +
main data sources which are providing COVID-19 related data and the widely used applications
 +
that have been applied on such data. This survey also identifies the applications or topics on
 +
which research is being processed with respect to COVID-19 sentimental analysis. At the end
 +
the future implications of COVID-research have also been presented in this survey.
 +
 +
 +
2. Methodology
 +
Thirty primary studies were selected for the comparison and their review was performed. Table
 +
1 has six columns, first columns shows the references while in second column, data sources used
 +
in COVID-19 research have been mentioned. The purpose of mentioning the data sources or data
 +
sets is to assist the new researchers in collecting the similar datasets for their research. We have
 +
also mentioned the volume of the dataset used in the primary studies in column 3. The methods
 +
and approaches frequently used for sentimental analysis of COVID-19 have been specified in
 +
column 4. The column 5 illustrated the application scenarios for COVID sentimental analysis
 +
research, it can give new directions for the future research. The future research directions have
 +
been shown in column 6.
 +
 +
2.1. COVID-19 Data-sets
 +
During the COVID pandemic, many people experience different mental issues which caused
 +
their emotions to change. The people used social media to express these emotions. Therefore,
 +
the social media provide huge amount of data to understand the peoples feelings and reactions
 +
to the situations they faced during pandemic. The data sources for COVID-19 research have
 +
been shown in Table 1. It illustrates that the main data source during COVID was twitter.
 +
�Twenty four out of thirty primary studies used twitter as a source in their research. However,
 +
the remaining data sources were WeChat account, Yelp, Reddit, and other Media forums.
 +
  Twitter: Twitter has been used worldwide for sharing the thoughts and opinions. It is most
 +
popular app having 81.47 million users [9]. People post their feelings in the form of "tweets".
 +
According to research in [10], almost 200 billion tweets are published in one year.
 +
 +
2.2. Sentiments classification methods
 +
With the increase of social media platforms and social media data, more powerful analytical tools
 +
should be developed. Different approaches were adopted for COVID-19 research to perform
 +
the sentimental analysis. They can be divided into three different types i.e. machine learning,
 +
lexicon based and hybrid.
 +
 +
2.2.1. ML and Deep Learning (DL) Methods:
 +
The ML methods which can be used for sentimental analysis are supervised learning approaches
 +
and unsupervised approaches.
 +
  The supervised learning works on the labelled data . Different researchers used different
 +
sentimental analysis methods on COVID data as seen in Table 1. In and [11], Naive Bayes
 +
algorithm was used as a supervised learning method for sentimental classification. Naive Bayes
 +
uses the Bayesian theorem given in equation 1.
 +
 +
                              𝑃 (𝐻|𝑋) = 𝑃 (𝑋|𝐻)𝑃 (𝐻)/𝑃 (𝑋)                                    (1)
 +
 +
Support vector machine works by finding the hyper-plane in the whole data by creating high
 +
dimensional feature from he feature space. [12], [13] and [14] used SVM in their research for
 +
sentimental analysis. Decision tree found different decision rules from the entire dataset and
 +
used them to train its model. Random forest also chooses random features and instance from
 +
the entire dataset. It has been used by [15], [13] and [14]. Many other researchers used other
 +
sentimental analysis techniques such as KNN Linear Regression [16], Logistic Regression [11],
 +
[14], LSTM [17], [13], RNN [18] and BERT model [10], [9] etc.
 +
  The unsupervised learning ML methods uses unlabeled data. There are different methods
 +
that have been applied on sentimental analysis during COVID. The researchers used K-means
 +
clustering in . However, many other studies such as [19], [20], [21], [14], [22], [23] have used
 +
Latent Dirichlet Allocation (LDA).
 +
 +
2.3. Application scenarios on COVID-19 data
 +
COVID has effected people’s life and thus they are facing different psychological issues. Many
 +
researchers pursued their research to analysis the people’s sentiments during COVID-19.
 +
 +
2.3.1. Mental health analysis of students during the lockdown
 +
To control the spread of COVID, social distancing was applied which reduced the human-to-
 +
human interactions. Many countries imposed lockdown, and closed the airspace, educational
 +
and other institutes. Due to lockdown, people specially students had to stay far away from their
 +
�homes, stuck in their hostels, and had to quit their educational activities, which causes anxiety
 +
and stress in students. Students express their sentiments using social platforms and researchers
 +
tried to explore their sentiments [20], [24] and students [25]. In [18], [26], [13], [11], [27], [28],
 +
[21], [22], [19], [29], [30], twitter data was used to understand the people’s sentiments during
 +
lockdown.
 +
 +
2.3.2. Reopening after COVID-19:
 +
Coronavirus has effected the lives of billion of peoples directly or indirectly. It has caused
 +
economical crisis all over the world which is a hurdle towards reopening [31]. The long-term
 +
closing of economy is a threat for any country to survive. Due to these reasons, people are
 +
forcing to reopen the businesses and going to normal life [32]. Hence, in [32] and [31], the
 +
researchers put their efforts on the discovering what are people thinking about re-opening after
 +
COVID-19.
 +
 +
2.3.3. Restaurant reviews
 +
In today’s digital era, the customers can share their opinion and feedback about quality of
 +
product or services they use from different organizations. These reviews help other customers
 +
to make decisions when they are about to use the service and product. The online reviews
 +
are associated with the star rating which effect the revenue of the restaurant. During COVID,
 +
special SOPs were announced for the restaurants and people were very concerned about the
 +
COVID-spread. Therefore, many restaurants got negative reviews for cold outdoor area and
 +
slow service. Researchers analyzed the people’ feedback about restaurants which helped the
 +
management of restaurants to maintain a quality food and ambience [15].
 +
 +
2.3.4. Vaccine sentiments and racial sentiments
 +
The development of COVID vaccine can be useful to control the spread of COVID. Therefore,
 +
it many industries put their efforts and develop different kind of vaccines. But, to control the
 +
COVID with vaccines, the acceptance and receiving of vaccines is the main requirement [33].
 +
If people are not willing to get themselves, it will be a clear hurdle in the control of COVID
 +
[34]. Researchers analyzed the public sentiments about vaccines in [9]. COVID also caused the
 +
feelings of discrimination across the boarders and therefore people became more racists [12].
 +
 +
 +
3. Comparison of Studies
 +
Table 1 shows the summary of the comparison of the primary studies used in this survey.
 +
 +
 +
 +
      Table 1: Comparison of methods and scenarios used for analyzing people’s sentiments related
 +
              to COVID-19
 +
�Ref  Data      Volume      of  Methods          Application                  Future Direction
 +
    Source/set Data
 +
[15] Yelp      112,412          GBDT,      RF,  Analyze        restaurant    Restaurant locations.
 +
                                  LSTM, SWEM        reviews
 +
[28] Tweets    20,325,929        CrystalFeel      Trends of fear, anger, sad-
 +
                                                                            Include other media
 +
                tweets                              ness, and joy            platforms.
 +
[35] Tweets    500,000          TextBlob          Finding tweets polarity  Explore other social
 +
                tweets                                                      media
 +
[9]  Tweets    57.5M            BERT            Vaccine sentiments        Real-time social media
 +
                                                                            monitoring
 +
[18] Tweets    N/A              NLP, RNN        Analyze sentiments        Visualization, cluster-
 +
                                                                            ing and classification
 +
[12] Tweets    3,377,295        SVM              Racial sentiment changes Temporal changes in
 +
                                                                            racial attitudes
 +
[36] Tweets    840,000          TextBlob, LDA, identification of Anxiety, Perception changes for
 +
                                                  stress, and trauma        different biographies
 +
[27] Tweets    57 454            NLP              Analyse the characteris- N/A
 +
                                                  tics of polish COVID-19
 +
[26] Tweets    370              WordCloud        Sentimental analysis      N/A
 +
[32] Tweets    293,597          Binary    logit Reopening sentiment      Socioeconomic and
 +
                                  model                                      household information
 +
[13] Tweets    7528              TextBlob, CNN- Perform sentiment analy- Use deep learning ap-
 +
                                  LSTM, RF, SVC, sis                        proaches
 +
                                  ETC, DT,
 +
[11] Tweets    900000            NB, LR,          Public sentiment associ- Include news articles
 +
                                                  ated with the progress of and personal commu-
 +
                                                  Coronavirus              nications data.
 +
[31] Tweets    293,597          N-gram, R pack- Reopen Sentiments          Replicate on other so-
 +
                                  ages Syuzhet                              cial media data
 +
                                  and sentimentr
 +
[29] Tweets    16 million        TClustVID        Investigate Topics and Explore other data
 +
                                                  Sentiment                repositories.
 +
[21] RateMDs    55,612 PORs      TF–IDF, LDA      Patients views            Trend in high death
 +
                                                                            and recovery rate
 +
[22] Tweets    4 million        LDA, NLP        COVID-19–related senti- Explore public trust
 +
                                                  ments
 +
[30] Tweets    13 million        Dynamic Topic Detecting Topic              More specific topics
 +
                                  Models
 +
[23] Qingbo    N/A              LDA              Emotional change          Precise location infor-
 +
                                                                            mation
 +
�4. Conclusion
 +
Twitter based sentimental classification is a new paradigm in the social media research. A review
 +
of almost thirty primary studies was performed in our research. The comparison of data sources
 +
used, volume of data used, approaches, and application scenarios with respect to COVID-19 was
 +
established. This survey presents its contribution in the field of sentimental analysis and open
 +
doors for the new researchers. This survey paper shows that twitter is the most popular data
 +
source for sentiments analysis and Naive Bayes and SVM are the algorithms which researchers
 +
used for sentimental analysis during COVID. During COVID-19, various researchers worked on
 +
the different dimensions such as mental health of students, reopening sentiments, restaurants
 +
reviews and vaccine sentiments. Thus, the use of advanced methods of machine learning and
 +
deep learning along with the social media data can explore more interesting topics in future.
 +
 +
 +
References
 +
[1] Z. Zhou, F. Liu, Q. Wang, R-Transformer network based on position and self-attention
 +
    mechanism for aspect-level sentiment classification, IEEE Access 7 (2019) 127754–127764.
 +
    doi:10.1109/ACCESS.2019.2938854.
 +
[2] M. Ceci, R. Corizzo, F. Fumarola, M. Ianni, D. Malerba, G. Maria, E. Masciari, M. Oliverio,
 +
    A. Rashkovska, Big data techniques for supporting accurate predictions of energy produc-
 +
    tion from renewable sources, volume 0, 2015, p. 62 – 71. doi:10.1145/2790755.2790762,
 +
    cited by: 15.
 +
[3] X. Wang, Y. Tong, Application of an emotional classification model in e-commerce
 +
    text based on an improved transformer model, PLoS One 16 (2021) 1–16. URL: http:
 +
    //dx.doi.org/10.1371/journal.pone.0247984. doi:10.1371/journal.pone.0247984.
 +
[4] A. Umair, E. Masciari, Sentimental and spatial analysis of covid-19 vaccines tweets, Journal
 +
    of Intelligent Information Systems (2022) 1–21.
 +
[5] B. Fazzinga, S. Flesca, F. Furfaro, E. Masciari, Rfid-data compression for supporting
 +
    aggregate queries, ACM Transactions on Database Systems 38 (2013) 1 – 45. doi:10.1145/
 +
    2487259.2487263, cited by: 11.
 +
[6] M. Ceci, R. Corizzo, F. Fumarola, M. Ianni, D. Malerba, G. Maria, E. Masciari, M. Oliverio,
 +
    A. Rashkovska, Big data techniques for supporting accurate predictions of energy pro-
 +
    duction from renewable sources, in: B. C. Desai, M. Toyama (Eds.), Proceedings of the
 +
    19th International Database Engineering & Applications Symposium, Yokohama, Japan,
 +
    July 13-15, 2015, ACM, 2015, pp. 62–71. URL: https://doi.org/10.1145/2790755.2790762.
 +
    doi:10.1145/2790755.2790762.
 +
[7] S. Flesca, E. Masciari, Efficient and effective web change detection, Data Knowl. Eng.
 +
    46 (2003) 203–224. URL: https://doi.org/10.1016/S0169-023X(02)00210-0. doi:10.1016/
 +
    S0169-023X(02)00210-0.
 +
[8] S. Flesca, F. Furfaro, E. Masciari, On the minimization of xpath queries, J. ACM 55 (2008) 2:1–
 +
    2:46. URL: https://doi.org/10.1145/1326554.1326556. doi:10.1145/1326554.1326556.
 +
[9] M. Müller, M. Salathé, Addressing machine learning concept drift reveals declining vaccine
 +
    sentiment during the COVID-19 pandemic, arXiv (2020) 1–12. arXiv:2012.02197.
 +
�[10] N. Chintalapudi, G. Battineni, F. Amenta, Sentimental Analysis of COVID-19 Tweets Using
 +
    Deep Learning Models, Infect. Dis. Rep. 13 (2021) 329–339. doi:10.3390/idr13020032.
 +
[11] J. Samuel, M. M. Rahman, G. G. N. Ali, Y. Samuel, A. Pelaez, P. H. J. Chong, M. Yakubov,
 +
    Feeling Positive about Reopening? New Normal Scenarios from COVID-19 US Reopen
 +
    Sentiment Analytics, IEEE Access 8 (2020) 142173–142190. doi:10.1109/ACCESS.2020.
 +
    3013933.
 +
[12] T. T. Nguyen, S. Criss, P. Dwivedi, D. Huang, J. Keralis, E. Hsu, L. Phan, L. H. Nguyen,
 +
    I. Yardi, M. M. Glymour, A. M. Allen, D. H. Chae, G. C. Gee, Q. C. Nguyen, Exploring
 +
    U.S. shifts in anti-Asian sentiment with the emergence of COVID-19, Int. J. Environ. Res.
 +
    Public Health 17 (2020) 1–13. doi:10.3390/ijerph17197032.
 +
[13] F. Rustam, M. Khalid, W. Aslam, V. Rupapara, A. Mehmood, G. S. Choi, A performance
 +
    comparison of supervised machine learning models for Covid-19 tweets sentiment analysis,
 +
    PLoS One 16 (2021) 1–23. URL: http://dx.doi.org/10.1371/journal.pone.0245909. doi:10.
 +
    1371/journal.pone.0245909.
 +
[14] X. Xiang, X. Lu, A. Halavanau, J. Xue, Y. Sun, P. H. L. Lai, Z. Wu, Modern Senicide in
 +
    the Face of a Pandemic: An Examination of Public Discourse and Sentiment About Older
 +
    Adults and COVID-19 Using Machine Learning, J. Gerontol. B. Psychol. Sci. Soc. Sci. 76
 +
    (2021) e190–e200. doi:10.1093/geronb/gbaa128.
 +
[15] Y. Luo, X. Xu, Comparative study of deep learning models for analyzing online restaurant
 +
    reviews in the era of the COVID-19 pandemic, Int. J. Hosp. Manag. 94 (2021) 102849. URL:
 +
    https://doi.org/10.1016/j.ijhm.2020.102849. doi:10.1016/j.ijhm.2020.102849.
 +
[16] H. Adamu, S. L. Lutfi, N. H. A. H. Malim, R. Hassan, A. Di Vaio, A. S. A. Mohamed, Framing
 +
    twitter public sentiment on Nigerian government COVID-19 palliatives distribution using
 +
    machine learning, Sustain. 13 (2021). doi:10.3390/su13063497.
 +
[17] H. Jelodar, Y. Wang, R. Orji, H. Huang, Deep sentiment classification and topic discovery
 +
    on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural
 +
    network approach, arXiv 24 (2020) 2733–2742.
 +
[18] L. Nemes, A. Kiss, Social media sentiment analysis based on COVID-19, J. Inf. Telecom-
 +
    mun. 5 (2021) 1–15. URL: https://doi.org/10.1080/24751839.2020.1790793. doi:10.1080/
 +
    24751839.2020.1790793.
 +
[19] M. Hung, E. Lauren, E. S. Hon, W. C. Birmingham, J. Xu, S. Su, S. D. Hon, J. Park, P. Dang,
 +
    M. S. Lipsky, Social network analysis of COVID-19 sentiments: Application of artificial
 +
    intelligence, J. Med. Internet Res. 22 (2020) 1–13. doi:10.2196/22590.
 +
[20] S. V. Praveen, R. Ittamalla, G. Deepak, Analyzing Indian general public’s perspective on
 +
    anxiety, stress and trauma during Covid-19 - A machine learning study of 840,000 tweets,
 +
    Diabetes Metab. Syndr. Clin. Res. Rev. 15 (2021) 667–671. URL: https://doi.org/10.1016/j.
 +
    dsx.2021.03.016. doi:10.1016/j.dsx.2021.03.016.
 +
[21] A. M. Shah, X. Yan, A. Qayyum, R. A. Naqvi, S. J. Shah, Mining topic and sentiment
 +
    dynamics in physician rating websites during the early wave of the COVID-19 pandemic:
 +
    Machine learning approach, Int. J. Med. Inform. 149 (2021). doi:10.1016/j.ijmedinf.
 +
    2021.104434.
 +
[22] J. Xue, J. Chen, C. Chen, C. Zheng, S. Li, T. Zhu, Public discourse and sentiment during the
 +
    COVID 19 pandemic: Using latent dirichlet allocation for topic modeling on twitter, PLoS
 +
    One 15 (2020) 1–12. URL: http://dx.doi.org/10.1371/journal.pone.0239441. doi:10.1371/
 +
�    journal.pone.0239441. arXiv:2005.08817.
 +
[23] B. Zhu, X. Zheng, H. Liu, J. Li, P. Wang, Analysis of spatiotemporal characteristics of
 +
    big data on social media sentiment with COVID-19 epidemic topics, Chaos, Solitons and
 +
    Fractals 140 (2020) 110123. URL: https://doi.org/10.1016/j.chaos.2020.110123. doi:10.1016/
 +
    j.chaos.2020.110123.
 +
[24] V. Ajantha Devi, A. Nayyar, Evaluation of Geotagging Twitter Data Using Sentiment
 +
    Analysis During COVID-19, volume 166, Springer Singapore, 2021. URL: http://dx.doi.org/
 +
    10.1007/978-981-15-9689-6{_}65. doi:10.1007/978-981-15-9689-6_65.
 +
[25] A. Agarwal, B. Agarwal, P. Harjule, A. Agarwal, Mental Health Analysis of Students in
 +
    Major Cities of India During COVID-19, Springer Singapore, 2021. URL: http://dx.doi.org/
 +
    10.1007/978-981-33-4236-1{_}4. doi:10.1007/978-981-33-4236-1_4.
 +
[26] S. Raheja, A. Asthana, Sentimental analysis of twitter comments on COVID-19, Proc.
 +
    Conflu. 2021 11th Int. Conf. Cloud Comput. Data Sci. Eng. (2021) 704–708. doi:10.1109/
 +
    Confluence51648.2021.9377048.
 +
[27] E. Probierz, A. Gałuszka, T. Dzida, Twitter text data from #Covid-19: Analysis of changes
 +
    in time using exploratory sentiment analysis, J. Phys. Conf. Ser. 1828 (2021). doi:10.1088/
 +
    1742-6596/1828/1/012138.
 +
[28] M. O. Lwin, J. Lu, A. Sheldenkar, P. J. Schulz, W. Shin, R. Gupta, Y. Yang, Global sentiments
 +
    surrounding the COVID-19 pandemic on Twitter: Analysis of Twitter trends, JMIR Public
 +
    Heal. Surveill. 6 (2020) 1–4. doi:10.2196/19447.
 +
[29] M. S. Satu, M. I. Khan, M. Mahmud, S. Uddin, M. A. Summers, J. M. Quinn, M. A. Moni,
 +
    TClustVID: A novel machine learning classification model to investigate topics and senti-
 +
    ment in COVID-19 tweets, medRxiv (2020). doi:10.1101/2020.08.04.20167973.
 +
[30] H. Yin, S. Yang, J. Li, Detecting Topic and Sentiment Dynamics Due to COVID-19 Pandemic
 +
    Using Social Media, volume 12447 LNAI, Springer International Publishing, 2020. doi:10.
 +
    1007/978-3-030-65390-3_46. arXiv:2007.02304.
 +
[31] J. Samuel, G. G. N. Ali, M. M. Rahman, E. Esawi, Y. Samuel, COVID-19 public sentiment
 +
    insights and machine learning for tweets classification, Inf. 11 (2020) 1–22. doi:10.3390/
 +
    info11060314. arXiv:2005.10898.
 +
[32] M. Mokhlesur Rahman, G. G. Nawaz Ali, X. J. Li, K. C. Paul, P. H. Chong, Twitter and
 +
    Census Data Analytics to Explore Socioeconomic Factors for Post-COVID-19 Reopening
 +
    Sentiment, arXiv (2020). doi:10.2139/ssrn.3639551.
 +
[33] H. Seale, A. E. Heywood, J. Leask, M. Sheel, D. N. Durrheim, K. Bolsewicz, R. Kaur, Exam-
 +
    ining Australian public perceptions and behaviors towards a future COVID-19 vaccine,
 +
    medRxiv (2020) 1–9. doi:10.1101/2020.09.29.20204396.
 +
[34] M. S. Green, R. Abdullah, S. Vered, D. Nitzan, A study of ethnic, gender and edu-
 +
    cational differences in attitudes toward COVID-19 vaccines in Israel – implications
 +
    for vaccination implementation policies, Isr. J. Health Policy Res. 10 (2021) 1–12.
 +
    doi:10.1186/s13584-021-00458-w.
 +
[35] K. H. Manguri, R. N. Ramadhan, P. R. Mohammed Amin, Twitter Sentiment Analysis on
 +
    Worldwide COVID-19 Outbreaks, Kurdistan J. Appl. Res. (2020) 54–65. doi:10.24017/
 +
    covid.8.
 +
[36] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Improving
 +
    Language Understanding by, OpenAI (2018) 1–10.
 +
 +
</pre>

Revision as of 18:01, 30 March 2023

Paper

Paper
edit
description  
id  Vol-3194/paper20
wikidataid  →Q117345145
title  A Survey of Sentimental Analysis Methods on COVID-19 Research
pdfUrl  https://ceur-ws.org/Vol-3194/paper20.pdf
dblpUrl  https://dblp.org/rec/conf/sebd/UmairM22
volume  Vol-3194→Vol-3194
session  →

A Survey of Sentimental Analysis Methods on COVID-19 Research

load PDF

A Survey of Sentimental Analysis Methods on
COVID-19 Research
Areeba Umair1 , Elio Masciari1,2
1
    Department of Electrical Engineering and Information Technologies, University of Naples, Federico II
2
    Institute for High Performance Computing and Networking (ICAR), National Research Council, Naples, Italy


                                         Abstract
                                         In this era of social media, people share anything they feel or experience on social media in the form of
                                         posts or comments. These posts, comments or reviews of the people can be analyzed using sentimental
                                         analysis, which is emerging field in text mining. COVID-19 has people’s life all over the globe and thus
                                         has declared as pandemic. Due to COVID, people are feeling panic, anxiety, rage, sorrow, misery, stress
                                         and other issues. In this review, we have presented the sentimental analysis data sources, approaches,
                                         scenarios, methods and tools by comparing thirty studies. The results illustrated that most researchers
                                         have used SVM and Naive Bayes for sentimental analysis on COVID research. We also concluded that
                                         most of the researchers work on the sentiments of students, reopening sentiments, vaccine sentiments
                                         etc.

                                         Keywords
                                         Social media Big Data, Sentiments related to COVID, Social Media Reviews, Data analytic, COVID-19




1. Introduction
Now-a-days, many people use social networks to express their opinion, thoughts or feedback
about anything, [1]. In this era of technology, almost all the industries provide their customers
with the ability to buy product online and also share their reviews or feedback on their website of
social media pages [2]. This feedback can be positive or negative which can help other customer
in making decision and help the industry to improve the product according to the customer
need [3]. Such kind of review data on internet can be used in extraction of sentiments from the
raw data that can be used for well-being of the society as well as business or organization [4],
[5]. Sentimental analysis is the natural language processing tasks in which text is classified into
positive, negative or neutral sentiments based on their meanings in the sentence. There are three
types of sentimental analysis i.e. document level, sentence level and aspect level sentimental
analysis. In order to gain the fine grain sentimental expression, aspect level sentimental analysis
is used [1]. Let’s take an example to understand aspect level sentimental analysis. "The food
is very tasty but its quality is low". In this example, "very tasty and "low" show two different
sentiments i.e. positive and negative respectively. The traditional sentimental analysis methods
have been eliminated due to advancement in artificial intelligence [3].

SEBD 2022: The 30th Italian Symposium on Advanced Database Systems, June 19-22, 2022, Tirrenia (PI), Italy
$ areeba.umair@unina.it (A. Umair); elio.masciari@unina.it (E. Masciari)
€ https://www.docenti.unina.it/elio.masciari (E. Masciari)
                                       © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
    CEUR
    Workshop
    Proceedings
                  http://ceur-ws.org
                  ISSN 1613-0073
                                       CEUR Workshop Proceedings (CEUR-WS.org)
�   The whole world is facing the biggest challenge in the form of COVID, which has destroyed
the economy of many under-developed countries [6]. Corona-virus was discovered in Wuhan,
China in the month of December 2019 and it has started spreading in the world and thus declared
as pandemic. According to John Hopkins University, 435, 427, 191 people have been affected
due to COVID, thus causing 5, 966, 417 number of deaths till 27 February 2022. People are
facing different psychological problems due to COVID such as anger, depression, fear, and many
others.
   The traditional machine learning methods and deep learning methods are available to resolve
the sentimental classification problems [7], [8]. The traditional ML (machine learning) classifiers
for sentimental classification are Support Vector Machine (SVM) and Naive Bayes however,
deep learning methods for sentimental classification are Recurrent Neural Network (RNN) and
Convolutional Neural Network (CNN). These methods extract meaningful features automatically
[1]. RNN has a recurrent nature due to which it suffers from gradient vanishing problem and
CNN has short-comings for sequential dependencies. Thus, the literature shows the different
issues and limitations in the exiting approaches such as low accuracy and performance and
high complexity [3]. The inconsistent sentimental polarity in the sentence causes the word
dependency to be weaken. In such scenario, attention mechanism can be fruitful for sentimental
classification tasks.
   In this research, we have collected thirty primary studies related to sentimental analysis with
respect to COVID-19 and performed the survey. The purpose of the survey was to identify the
main data sources which are providing COVID-19 related data and the widely used applications
that have been applied on such data. This survey also identifies the applications or topics on
which research is being processed with respect to COVID-19 sentimental analysis. At the end
the future implications of COVID-research have also been presented in this survey.


2. Methodology
Thirty primary studies were selected for the comparison and their review was performed. Table
1 has six columns, first columns shows the references while in second column, data sources used
in COVID-19 research have been mentioned. The purpose of mentioning the data sources or data
sets is to assist the new researchers in collecting the similar datasets for their research. We have
also mentioned the volume of the dataset used in the primary studies in column 3. The methods
and approaches frequently used for sentimental analysis of COVID-19 have been specified in
column 4. The column 5 illustrated the application scenarios for COVID sentimental analysis
research, it can give new directions for the future research. The future research directions have
been shown in column 6.

2.1. COVID-19 Data-sets
During the COVID pandemic, many people experience different mental issues which caused
their emotions to change. The people used social media to express these emotions. Therefore,
the social media provide huge amount of data to understand the peoples feelings and reactions
to the situations they faced during pandemic. The data sources for COVID-19 research have
been shown in Table 1. It illustrates that the main data source during COVID was twitter.
�Twenty four out of thirty primary studies used twitter as a source in their research. However,
the remaining data sources were WeChat account, Yelp, Reddit, and other Media forums.
  Twitter: Twitter has been used worldwide for sharing the thoughts and opinions. It is most
popular app having 81.47 million users [9]. People post their feelings in the form of "tweets".
According to research in [10], almost 200 billion tweets are published in one year.

2.2. Sentiments classification methods
With the increase of social media platforms and social media data, more powerful analytical tools
should be developed. Different approaches were adopted for COVID-19 research to perform
the sentimental analysis. They can be divided into three different types i.e. machine learning,
lexicon based and hybrid.

2.2.1. ML and Deep Learning (DL) Methods:
The ML methods which can be used for sentimental analysis are supervised learning approaches
and unsupervised approaches.
   The supervised learning works on the labelled data . Different researchers used different
sentimental analysis methods on COVID data as seen in Table 1. In and [11], Naive Bayes
algorithm was used as a supervised learning method for sentimental classification. Naive Bayes
uses the Bayesian theorem given in equation 1.

                              𝑃 (𝐻|𝑋) = 𝑃 (𝑋|𝐻)𝑃 (𝐻)/𝑃 (𝑋)                                    (1)

Support vector machine works by finding the hyper-plane in the whole data by creating high
dimensional feature from he feature space. [12], [13] and [14] used SVM in their research for
sentimental analysis. Decision tree found different decision rules from the entire dataset and
used them to train its model. Random forest also chooses random features and instance from
the entire dataset. It has been used by [15], [13] and [14]. Many other researchers used other
sentimental analysis techniques such as KNN Linear Regression [16], Logistic Regression [11],
[14], LSTM [17], [13], RNN [18] and BERT model [10], [9] etc.
   The unsupervised learning ML methods uses unlabeled data. There are different methods
that have been applied on sentimental analysis during COVID. The researchers used K-means
clustering in . However, many other studies such as [19], [20], [21], [14], [22], [23] have used
Latent Dirichlet Allocation (LDA).

2.3. Application scenarios on COVID-19 data
COVID has effected people’s life and thus they are facing different psychological issues. Many
researchers pursued their research to analysis the people’s sentiments during COVID-19.

2.3.1. Mental health analysis of students during the lockdown
To control the spread of COVID, social distancing was applied which reduced the human-to-
human interactions. Many countries imposed lockdown, and closed the airspace, educational
and other institutes. Due to lockdown, people specially students had to stay far away from their
�homes, stuck in their hostels, and had to quit their educational activities, which causes anxiety
and stress in students. Students express their sentiments using social platforms and researchers
tried to explore their sentiments [20], [24] and students [25]. In [18], [26], [13], [11], [27], [28],
[21], [22], [19], [29], [30], twitter data was used to understand the people’s sentiments during
lockdown.

2.3.2. Reopening after COVID-19:
Coronavirus has effected the lives of billion of peoples directly or indirectly. It has caused
economical crisis all over the world which is a hurdle towards reopening [31]. The long-term
closing of economy is a threat for any country to survive. Due to these reasons, people are
forcing to reopen the businesses and going to normal life [32]. Hence, in [32] and [31], the
researchers put their efforts on the discovering what are people thinking about re-opening after
COVID-19.

2.3.3. Restaurant reviews
In today’s digital era, the customers can share their opinion and feedback about quality of
product or services they use from different organizations. These reviews help other customers
to make decisions when they are about to use the service and product. The online reviews
are associated with the star rating which effect the revenue of the restaurant. During COVID,
special SOPs were announced for the restaurants and people were very concerned about the
COVID-spread. Therefore, many restaurants got negative reviews for cold outdoor area and
slow service. Researchers analyzed the people’ feedback about restaurants which helped the
management of restaurants to maintain a quality food and ambience [15].

2.3.4. Vaccine sentiments and racial sentiments
The development of COVID vaccine can be useful to control the spread of COVID. Therefore,
it many industries put their efforts and develop different kind of vaccines. But, to control the
COVID with vaccines, the acceptance and receiving of vaccines is the main requirement [33].
If people are not willing to get themselves, it will be a clear hurdle in the control of COVID
[34]. Researchers analyzed the public sentiments about vaccines in [9]. COVID also caused the
feelings of discrimination across the boarders and therefore people became more racists [12].


3. Comparison of Studies
Table 1 shows the summary of the comparison of the primary studies used in this survey.



      Table 1: Comparison of methods and scenarios used for analyzing people’s sentiments related
               to COVID-19
�Ref  Data       Volume       of   Methods           Application                   Future Direction
     Source/set Data
[15] Yelp       112,412           GBDT,       RF,   Analyze         restaurant    Restaurant locations.
                                  LSTM, SWEM        reviews
[28] Tweets     20,325,929        CrystalFeel       Trends of fear, anger, sad-
                                                                             Include other media
                tweets                              ness, and joy            platforms.
[35] Tweets     500,000           TextBlob          Finding tweets polarity  Explore other social
                tweets                                                       media
[9]   Tweets    57.5M             BERT             Vaccine sentiments        Real-time social media
                                                                             monitoring
[18] Tweets     N/A               NLP, RNN         Analyze sentiments        Visualization, cluster-
                                                                             ing and classification
[12] Tweets     3,377,295         SVM              Racial sentiment changes Temporal changes in
                                                                             racial attitudes
[36] Tweets     840,000           TextBlob, LDA, identification of Anxiety, Perception changes for
                                                   stress, and trauma        different biographies
[27] Tweets     57 454            NLP              Analyse the characteris- N/A
                                                   tics of polish COVID-19
[26] Tweets     370               WordCloud        Sentimental analysis      N/A
[32] Tweets     293,597           Binary     logit Reopening sentiment       Socioeconomic and
                                  model                                      household information
[13] Tweets     7528              TextBlob, CNN- Perform sentiment analy- Use deep learning ap-
                                  LSTM, RF, SVC, sis                         proaches
                                  ETC, DT,
[11] Tweets     900000            NB, LR,          Public sentiment associ- Include news articles
                                                   ated with the progress of and personal commu-
                                                   Coronavirus               nications data.
[31] Tweets     293,597           N-gram, R pack- Reopen Sentiments          Replicate on other so-
                                  ages Syuzhet                               cial media data
                                  and sentimentr
[29] Tweets     16 million        TClustVID        Investigate Topics and Explore other data
                                                   Sentiment                 repositories.
[21] RateMDs    55,612 PORs       TF–IDF, LDA      Patients views            Trend in high death
                                                                             and recovery rate
[22] Tweets     4 million         LDA, NLP         COVID-19–related senti- Explore public trust
                                                   ments
[30] Tweets     13 million        Dynamic Topic Detecting Topic              More specific topics
                                  Models
[23] Qingbo     N/A               LDA              Emotional change          Precise location infor-
                                                                             mation
�4. Conclusion
Twitter based sentimental classification is a new paradigm in the social media research. A review
of almost thirty primary studies was performed in our research. The comparison of data sources
used, volume of data used, approaches, and application scenarios with respect to COVID-19 was
established. This survey presents its contribution in the field of sentimental analysis and open
doors for the new researchers. This survey paper shows that twitter is the most popular data
source for sentiments analysis and Naive Bayes and SVM are the algorithms which researchers
used for sentimental analysis during COVID. During COVID-19, various researchers worked on
the different dimensions such as mental health of students, reopening sentiments, restaurants
reviews and vaccine sentiments. Thus, the use of advanced methods of machine learning and
deep learning along with the social media data can explore more interesting topics in future.


References
 [1] Z. Zhou, F. Liu, Q. Wang, R-Transformer network based on position and self-attention
     mechanism for aspect-level sentiment classification, IEEE Access 7 (2019) 127754–127764.
     doi:10.1109/ACCESS.2019.2938854.
 [2] M. Ceci, R. Corizzo, F. Fumarola, M. Ianni, D. Malerba, G. Maria, E. Masciari, M. Oliverio,
     A. Rashkovska, Big data techniques for supporting accurate predictions of energy produc-
     tion from renewable sources, volume 0, 2015, p. 62 – 71. doi:10.1145/2790755.2790762,
     cited by: 15.
 [3] X. Wang, Y. Tong, Application of an emotional classification model in e-commerce
     text based on an improved transformer model, PLoS One 16 (2021) 1–16. URL: http:
     //dx.doi.org/10.1371/journal.pone.0247984. doi:10.1371/journal.pone.0247984.
 [4] A. Umair, E. Masciari, Sentimental and spatial analysis of covid-19 vaccines tweets, Journal
     of Intelligent Information Systems (2022) 1–21.
 [5] B. Fazzinga, S. Flesca, F. Furfaro, E. Masciari, Rfid-data compression for supporting
     aggregate queries, ACM Transactions on Database Systems 38 (2013) 1 – 45. doi:10.1145/
     2487259.2487263, cited by: 11.
 [6] M. Ceci, R. Corizzo, F. Fumarola, M. Ianni, D. Malerba, G. Maria, E. Masciari, M. Oliverio,
     A. Rashkovska, Big data techniques for supporting accurate predictions of energy pro-
     duction from renewable sources, in: B. C. Desai, M. Toyama (Eds.), Proceedings of the
     19th International Database Engineering & Applications Symposium, Yokohama, Japan,
     July 13-15, 2015, ACM, 2015, pp. 62–71. URL: https://doi.org/10.1145/2790755.2790762.
     doi:10.1145/2790755.2790762.
 [7] S. Flesca, E. Masciari, Efficient and effective web change detection, Data Knowl. Eng.
     46 (2003) 203–224. URL: https://doi.org/10.1016/S0169-023X(02)00210-0. doi:10.1016/
     S0169-023X(02)00210-0.
 [8] S. Flesca, F. Furfaro, E. Masciari, On the minimization of xpath queries, J. ACM 55 (2008) 2:1–
     2:46. URL: https://doi.org/10.1145/1326554.1326556. doi:10.1145/1326554.1326556.
 [9] M. Müller, M. Salathé, Addressing machine learning concept drift reveals declining vaccine
     sentiment during the COVID-19 pandemic, arXiv (2020) 1–12. arXiv:2012.02197.
�[10] N. Chintalapudi, G. Battineni, F. Amenta, Sentimental Analysis of COVID-19 Tweets Using
     Deep Learning Models, Infect. Dis. Rep. 13 (2021) 329–339. doi:10.3390/idr13020032.
[11] J. Samuel, M. M. Rahman, G. G. N. Ali, Y. Samuel, A. Pelaez, P. H. J. Chong, M. Yakubov,
     Feeling Positive about Reopening? New Normal Scenarios from COVID-19 US Reopen
     Sentiment Analytics, IEEE Access 8 (2020) 142173–142190. doi:10.1109/ACCESS.2020.
     3013933.
[12] T. T. Nguyen, S. Criss, P. Dwivedi, D. Huang, J. Keralis, E. Hsu, L. Phan, L. H. Nguyen,
     I. Yardi, M. M. Glymour, A. M. Allen, D. H. Chae, G. C. Gee, Q. C. Nguyen, Exploring
     U.S. shifts in anti-Asian sentiment with the emergence of COVID-19, Int. J. Environ. Res.
     Public Health 17 (2020) 1–13. doi:10.3390/ijerph17197032.
[13] F. Rustam, M. Khalid, W. Aslam, V. Rupapara, A. Mehmood, G. S. Choi, A performance
     comparison of supervised machine learning models for Covid-19 tweets sentiment analysis,
     PLoS One 16 (2021) 1–23. URL: http://dx.doi.org/10.1371/journal.pone.0245909. doi:10.
     1371/journal.pone.0245909.
[14] X. Xiang, X. Lu, A. Halavanau, J. Xue, Y. Sun, P. H. L. Lai, Z. Wu, Modern Senicide in
     the Face of a Pandemic: An Examination of Public Discourse and Sentiment About Older
     Adults and COVID-19 Using Machine Learning, J. Gerontol. B. Psychol. Sci. Soc. Sci. 76
     (2021) e190–e200. doi:10.1093/geronb/gbaa128.
[15] Y. Luo, X. Xu, Comparative study of deep learning models for analyzing online restaurant
     reviews in the era of the COVID-19 pandemic, Int. J. Hosp. Manag. 94 (2021) 102849. URL:
     https://doi.org/10.1016/j.ijhm.2020.102849. doi:10.1016/j.ijhm.2020.102849.
[16] H. Adamu, S. L. Lutfi, N. H. A. H. Malim, R. Hassan, A. Di Vaio, A. S. A. Mohamed, Framing
     twitter public sentiment on Nigerian government COVID-19 palliatives distribution using
     machine learning, Sustain. 13 (2021). doi:10.3390/su13063497.
[17] H. Jelodar, Y. Wang, R. Orji, H. Huang, Deep sentiment classification and topic discovery
     on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural
     network approach, arXiv 24 (2020) 2733–2742.
[18] L. Nemes, A. Kiss, Social media sentiment analysis based on COVID-19, J. Inf. Telecom-
     mun. 5 (2021) 1–15. URL: https://doi.org/10.1080/24751839.2020.1790793. doi:10.1080/
     24751839.2020.1790793.
[19] M. Hung, E. Lauren, E. S. Hon, W. C. Birmingham, J. Xu, S. Su, S. D. Hon, J. Park, P. Dang,
     M. S. Lipsky, Social network analysis of COVID-19 sentiments: Application of artificial
     intelligence, J. Med. Internet Res. 22 (2020) 1–13. doi:10.2196/22590.
[20] S. V. Praveen, R. Ittamalla, G. Deepak, Analyzing Indian general public’s perspective on
     anxiety, stress and trauma during Covid-19 - A machine learning study of 840,000 tweets,
     Diabetes Metab. Syndr. Clin. Res. Rev. 15 (2021) 667–671. URL: https://doi.org/10.1016/j.
     dsx.2021.03.016. doi:10.1016/j.dsx.2021.03.016.
[21] A. M. Shah, X. Yan, A. Qayyum, R. A. Naqvi, S. J. Shah, Mining topic and sentiment
     dynamics in physician rating websites during the early wave of the COVID-19 pandemic:
     Machine learning approach, Int. J. Med. Inform. 149 (2021). doi:10.1016/j.ijmedinf.
     2021.104434.
[22] J. Xue, J. Chen, C. Chen, C. Zheng, S. Li, T. Zhu, Public discourse and sentiment during the
     COVID 19 pandemic: Using latent dirichlet allocation for topic modeling on twitter, PLoS
     One 15 (2020) 1–12. URL: http://dx.doi.org/10.1371/journal.pone.0239441. doi:10.1371/
�     journal.pone.0239441. arXiv:2005.08817.
[23] B. Zhu, X. Zheng, H. Liu, J. Li, P. Wang, Analysis of spatiotemporal characteristics of
     big data on social media sentiment with COVID-19 epidemic topics, Chaos, Solitons and
     Fractals 140 (2020) 110123. URL: https://doi.org/10.1016/j.chaos.2020.110123. doi:10.1016/
     j.chaos.2020.110123.
[24] V. Ajantha Devi, A. Nayyar, Evaluation of Geotagging Twitter Data Using Sentiment
     Analysis During COVID-19, volume 166, Springer Singapore, 2021. URL: http://dx.doi.org/
     10.1007/978-981-15-9689-6{_}65. doi:10.1007/978-981-15-9689-6_65.
[25] A. Agarwal, B. Agarwal, P. Harjule, A. Agarwal, Mental Health Analysis of Students in
     Major Cities of India During COVID-19, Springer Singapore, 2021. URL: http://dx.doi.org/
     10.1007/978-981-33-4236-1{_}4. doi:10.1007/978-981-33-4236-1_4.
[26] S. Raheja, A. Asthana, Sentimental analysis of twitter comments on COVID-19, Proc.
     Conflu. 2021 11th Int. Conf. Cloud Comput. Data Sci. Eng. (2021) 704–708. doi:10.1109/
     Confluence51648.2021.9377048.
[27] E. Probierz, A. Gałuszka, T. Dzida, Twitter text data from #Covid-19: Analysis of changes
     in time using exploratory sentiment analysis, J. Phys. Conf. Ser. 1828 (2021). doi:10.1088/
     1742-6596/1828/1/012138.
[28] M. O. Lwin, J. Lu, A. Sheldenkar, P. J. Schulz, W. Shin, R. Gupta, Y. Yang, Global sentiments
     surrounding the COVID-19 pandemic on Twitter: Analysis of Twitter trends, JMIR Public
     Heal. Surveill. 6 (2020) 1–4. doi:10.2196/19447.
[29] M. S. Satu, M. I. Khan, M. Mahmud, S. Uddin, M. A. Summers, J. M. Quinn, M. A. Moni,
     TClustVID: A novel machine learning classification model to investigate topics and senti-
     ment in COVID-19 tweets, medRxiv (2020). doi:10.1101/2020.08.04.20167973.
[30] H. Yin, S. Yang, J. Li, Detecting Topic and Sentiment Dynamics Due to COVID-19 Pandemic
     Using Social Media, volume 12447 LNAI, Springer International Publishing, 2020. doi:10.
     1007/978-3-030-65390-3_46. arXiv:2007.02304.
[31] J. Samuel, G. G. N. Ali, M. M. Rahman, E. Esawi, Y. Samuel, COVID-19 public sentiment
     insights and machine learning for tweets classification, Inf. 11 (2020) 1–22. doi:10.3390/
     info11060314. arXiv:2005.10898.
[32] M. Mokhlesur Rahman, G. G. Nawaz Ali, X. J. Li, K. C. Paul, P. H. Chong, Twitter and
     Census Data Analytics to Explore Socioeconomic Factors for Post-COVID-19 Reopening
     Sentiment, arXiv (2020). doi:10.2139/ssrn.3639551.
[33] H. Seale, A. E. Heywood, J. Leask, M. Sheel, D. N. Durrheim, K. Bolsewicz, R. Kaur, Exam-
     ining Australian public perceptions and behaviors towards a future COVID-19 vaccine,
     medRxiv (2020) 1–9. doi:10.1101/2020.09.29.20204396.
[34] M. S. Green, R. Abdullah, S. Vered, D. Nitzan, A study of ethnic, gender and edu-
     cational differences in attitudes toward COVID-19 vaccines in Israel – implications
     for vaccination implementation policies, Isr. J. Health Policy Res. 10 (2021) 1–12.
     doi:10.1186/s13584-021-00458-w.
[35] K. H. Manguri, R. N. Ramadhan, P. R. Mohammed Amin, Twitter Sentiment Analysis on
     Worldwide COVID-19 Outbreaks, Kurdistan J. Appl. Res. (2020) 54–65. doi:10.24017/
     covid.8.
[36] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Improving
     Language Understanding by, OpenAI (2018) 1–10.
�