SENTIMENT ANALYSIS ON MYPERTAMINA APPLICATION REVIEWS USING NBC AND SVM WITH NEGATION HANDLING

Amir Hamzah; Renna Yanwastika Ariyana

doi:10.56327/ijiscs.v8i2.1655

SENTIMENT ANALYSIS ON MYPERTAMINA APPLICATION REVIEWS USING NBC AND SVM WITH NEGATION HANDLING

Amir Hamzah⁽¹⁾, Renna Yanwastika Ariyana⁽²⁾,

(1) Departement of Informatics, Institute of Science & Technology AKPRIND, Yogyakarta
(2) Departement of Informatics, Institute of Science & Technology AKPRIND, Yogyakarta

Corresponding Author

Abstract

Pertamina has issued a cashless application for fuel purchases since July 2019, named as MyPertamina. The application aims to make it easier for customers to make payments in transactions at fuel stations. MyPertamina application can currently be downloaded on Google Playstore. Since its release until now, MyPertamina has been downloaded as many as 10 million with a rating of 3.1 and 339 thousand reviews. Unfortunately the low rating and user reviews dominated by negative comments show that the app's performance is still not satisfactory. The reviews data can be converted into valuable information by using entiment analysis. Many researchers have applied sentiment analysis to MyPertamina user comment data. However, there have been no studies that apply the handling of negation in MyPertamina reviews, even though negative comments are very often found the word negation, i.e ‘tidak’,’jangan’,’belum’ and ‘bukan’ that will change the sentiment of next sentiment word. Untreated negation words will lead to errors in classification which in turn will decrease accuracy. This study applies the handling of negation words using First Sentiment Word (FSW) and Fixed Window Length (FWL) methods. The classification algorithms used are Naïve Bayes Classifier (NBC) and Support Vector Machine (SVM). In this work, we analized 1000 comments consisting of 390 positive comments and 610 negative comments. The results showed that the best performance of negation handling is FSW. This method has improved accuracy by 2.5% and F1 by 1.5% using NBC algorithm and has improved accuracy by 2.9% nad F1 by 3.4% using SVM algorithm.

Keywords

negation handling, MyPertamina, NBC, SVM

References

U. Khaidarni, A. H. Fauzi, H. Nisa, R. G. Gumelar, and A. Muldi, “Analisis Difusi Inovasi Terhadap Aplikasi Mypertamina,” Jurnal Ilmiah Wahana Pendidikan, 2023.

F. Setya Ananto and F. N. Hasan, “Implementasi Algoritma Naïve Bayes Terhadap Analisis Sentimen Ulasan Aplikasi MyPertamina pada Google Play Store,” Jurnal ICT : Information Communication & Technology, vol. 23, no. 1, 2023.

A. M. Ginting, “KEBIJAKAN PEMBATASAN KONSUMSI BBM BERSUBSIDI MELALUI APLIKASI MYPERTAMINA,” KAJIAN SINGKAT TERHADAP ISU AKTUAL DAN STRATEGIS, vol. 14, no. 13, 2022.

R. A. Softina, F. M. Amin, and N. Wahyudi, “Analisis Faktor yang Mempengaruhi Innovation Resistance dan Intention to Use Terhadap Penerapan Pembayaran Non Tunai,” JURNAL SISTEM INFORMASI BISNIS, vol. 12, no. 1, 2022, doi: 10.21456/vol12iss1pp26-35.

M. A. Abdurrazzaq, “Analisis Ulasan Aplikasi MyPertamina Menggunakan Topic Modeling dengan Latent Dirichlet Allocation,” KALBISCIENTIA Jurnal Sains dan Teknologi, vol. 10, no. 1, 2023, doi: 10.53008/kalbiscientia.v10i1.694.

A. A. Sinurat, C. Hendriyani, and F. Damayanti, “MyPertamina Application To Increase Consumer Engagement,” The International Journal of Business Review (The Jobs Review), vol. 5, no. 1, 2022, doi: 10.17509/tjr.v5i1.48470.

A. U. Hasanah, B. Waspodo, and E. Rahajeng, “Analysis of MyPertamina Application User Satisfaction Using End User Computing Satisfaction Method,” Journal of Software Engineering Ampera, vol. 4, no. 1, 2023, doi: 10.51519/journalsea.v4i1.375.

C. G. Indrayanto, D. E. Ratnawati, and B. Rahayudi, “Analisis Sentimen Data Ulasan Pengguna Aplikasi MyPertamina di Indonesia pada Google Play Store menggunakan Metode Random Forest,” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 7, no. 3, 2023.

Nabilla Saumi Putri, “Analisis sentimen review aplikasi mypertamina pada twitter menggunakan metode naïve bayes classifier,” Jurnal CoSciTech (Computer Science and Information Technology), vol. 4, no. 1, 2023, doi: 10.37859/coscitech.v4i1.4789.

R. Maulana, A. Voutama, and T. Ridwan, “Analisis Sentimen Ulasan Aplikasi MyPertamina pada Google Play Store menggunakan Algoritma NBC,” Jurnal Teknologi Terpadu, vol. 9, no. 1, 2023, doi: 10.54914/jtt.v9i1.609.

Gilbert, Syariful Alam, and M. Imam Sulistyo, “ANALISIS SENTIMEN BERDASARKAN ULASAN PENGGUNA APLIKASI MYPERTAMINA PADA GOOGLE PLAYSTORE MENGGUNAKAN METODE NAÏVE BAYES,” STORAGE: Jurnal Ilmiah Teknik dan Ilmu Komputer, vol. 2, no. 3, 2023, doi: 10.55123/storage.v2i3.2333.

N. Yusliani, M. Diana MARIESKA, E. Lestari, M. Ridho Putra SULFA, and W. Arimurti, “The Development of Indonesian Sentiment Analysis with Negation Handling,” 2020.

M. Mustasaruddin, E. Budianita, M. Fikry, and F. Yanto, “Klasifikasi Sentiment Review Aplikasi MyPertamina Menggunakan Word Embedding FastText dan SVM (Support Vector Machine),” Jurnal Sistem Komputer dan Informatika (JSON), vol. 4, no. 3, 2023, doi: 10.30865/json.v4i3.5695.

S. L. Ting, W. H. Ip, and A. H. C. Tsang, “Is Naïve Bayes a Good Classifier for Document Classification?,” 2011. [Online]. Available: https://www.researchgate.net/publication/266463703

B. Heerschop, P. Van Iterson, A. Hogenboom, F. Frasincar, and U. Kaymak, “Analyzing sentiment in a large set of Web data while accounting for negation,” in Advances in Intelligent and Soft Computing, 2011, pp. 195–205. doi: 10.1007/978-3-642-18029-3_20.

A. Hogenboom, P. Van Iterson, B. Heerschop, F. Frasincar, and U. Kaymak, “Determining negation scope and strength in sentiment analysis,” in Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, 2011, pp. 2589–2594. doi: 10.1109/ICSMC.2011.6084066.

V. P. Ramadhan, P. Purwanto, and F. Alzami, “Sentiment Analysis of Community Response Indonesia Against Covid-19 on Twitter Based on Negation Handling,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, vol. 7, no. 2, pp. 161–168, Jun. 2022, doi: 10.22219/kinetik.v7i2.1429.

R. Amalia, M. A. Bijaksana, and D. Darmantoro, “Negation handling in sentiment classification using rule-based adapted from Indonesian language syntactic for Indonesian text in Twitter,” in Journal of Physics: Conference Series, 2018. doi: 10.1088/1742-6596/971/1/012039.

T. G. Prahasiwi and R. Kusumaningrum, “Implementation of negation handling techniques using modified syntactic rule in Indonesian sentiment analysis,” in Journal of Physics: Conference Series, Institute of Physics Publishing, Jun. 2019. doi: 10.1088/1742-6596/1217/1/012115.

A. R. Hamzah, “Sentiment Analysis untuk Opini Akademik Menggunakan Naive Bayes Classifier dan Information Gain,” Seminar Nasional Call For Paper Fakultas Teknik Universitas Batik Surakrta, pp. 1–22, 2021.

D. I. Sumantiawan, J. E. Suseno, and W. A. Syafei, “Sentiment Analysis of Customer Reviews Using Support Vector Machine and Smote-Tomek Links For Identify Customer Satisfaction,” J. Sistem Info. Bisnis, vol. 13, no. 1, pp. 1–9, Jun. 2023, doi: 10.21456/vol13iss1pp1-9.

M. Dadvar, C. Hauff, and F. De Jong, “Scope of Negation Detection in Sentiment Analysis.” [Online]. Available: http://wwwhome.ewi.utwente.nl/~dadvarm/dir2011/negation.txt

Sudaryono, Negasi dalam bahasa Indonesia : Suatu Tinjauan Sintaksis Semantik, Jakarta: Depdiknas. Jakarta: Depdiknas , 1992.

A. M. Ningtyas and G. B. Herwanto, “The Influence of Negation Handling on Sentiment Analysis in Bahasa Indonesia,” in Proceedings of 2018 5th International Conference on Data and Software Engineering, ICoDSE 2018, Institute of Electrical and Electronics Engineers Inc., Jul. 2018. doi: 10.1109/ICODSE.2018.8705802.

F. Koto and G. Y. Rahmaningtyas, “Inset lexicon: Evaluation of a word list for Indonesian sentiment analysis in microblogs,” in Proceedings of the 2017 International Conference on Asian Language Processing, IALP 2017, Institute of Electrical and Electronics Engineers Inc., Jul. 2017, pp. 391–394. doi: 10.1109/IALP.2017.8300625.

W. B. Trihanto, R. Arifudin, and A. Muslim, “Information Retrieval System for Determining The Title of Journal Trends in Indonesian Language Using TF-IDF and Naїve Bayes Classifier,” Scientific Journal of Informatics, vol. 4, no. 2, pp. 2407–7658, 2017, [Online]. Available: http://journal.unnes.ac.id/nju/index.php/sji

Full Text: PDF