Generator Kisah Daerah Berbasis Bahasa Jawa dengan Finetuned GPT-2
Kata Kunci:
Text Generator Berbahasa Jawa, GPT-2, Large Language Model, Low-Resource Language ModelAbstrak
Pengembangan text generator pada Bahasa Jawa bertujuan menciptakan generator kisah daerah yang memanfaatkan dua skema dataset, yaitu data hasil translasi dan data teks/dokumen digital berbahasa Jawa dengan variasi dua model pre-trained GPT-2. Pendekatan ini dimaksudkan untuk mengatasi masalah kualitas dan kehilangan unsur karakter kultural serta linguistik dalam proses pengumpulan dataset. Dengan demikian, pengembangan ini diharapkan dapat meningkatkan ekonomi dan pelestarian budaya Indonesia melalui kreativitas storytelling berbahasa jawa. Metode yang digunakan ialah fine-tuning GPT-2 pre-trained. Dari pengembangan metode tersebut, didapatkan skema model finetuned GPT-2 Medium Berbahasa Indonesia dengan skema dataset II serta scheduler inverse square root dengan perplexity 65,70.
Referensi
이제용, J., & Yoo, K. W. (2018). The structural relationship between selection attributes of traditional culture festival storytelling, brand equity, brand attitude, and brand loyalty. The Journal of the Korea Contents Association, 18, 647-659.
Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi, "Badan Bahasa petakan 652 bahasa daerah di Indonesia," Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi, [Online]. Available: https://www.kemdikbud.go.id/main/blog/2018/07/badan-bahasa-petakan-652-bahasa-daerah-di-indonesia. [Accessed: Jul. 1, 2023].
D. M. Eberhard, G. F. Simons, and C. D. Fennig (eds.), "Ethnologue: Languages of the World," 26th ed. Dallas, Texas: SIL International, 2023. [Online]. Available: http://www.ethnologue.com.
R. Wijaya and B. Pudjoatmodjo, "An overview and implementation of extraction-transformation-loading (ETL) process in data warehouse (Case study: Department of agriculture)," 2015 3rd International Conference on Information and Communication Technology (ICoICT), Nusa Dua, Bali, Indonesia, 2015, pp. 70-74, doi: 10.1109/ICoICT.2015.7231399.
Radford, A., Narasimhan, K., Salimans, T., dan Sutskever, I. 2018. Improving Language Understanding by Generative Pre-Training. OpenAI.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł,. dan Polosukhin, I. 2017. Attention is all you need. In Advances in neural information processing systems. pp.5998-6008.
Z. Chi, H. Huang, L. Liu, Y. Bai, X. Gao and X. -L. Mao, "Can Pretrained English Language Models Benefit Non-English NLP Systems in Low-Resource Scenarios?," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, doi: 10.1109/TASLP.2023.3267618.
W. Wongso, D. S. Setiawan and D. Suhartono, "Causal and Masked Language Modeling of Javanese Language using Transformer-based Architectures," 2021 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Depok, Indonesia, 2021, pp. 1-7, doi: 10.1109/ICACSIS53237.2021.9631331.
Akmal et al., “Indonesian-NLP/GPT2-medium-Indonesian · hugging face,” indonesian-nlp/gpt2-medium-indonesian · Hugging Face, https://huggingface.co/indonesian-nlp/gpt2-medium-indonesian (accessed Jul. 1, 2023).
I. Zafar and G. Tzanidou, Hands-on Convolutional Neural Networks with Tensorflow: Solve Computer Vision Problems with Modeling in Tensorflow and Python. Birmingham ; Mumbai: Packt, 2018.
Perplexity of fixed-length models, https://huggingface.co/docs/transformers/perplexity (accessed Jun. 30, 2023).
Unduhan
Diterbitkan
Terbitan
Bagian
Lisensi
Hak Cipta (c) 2024 Kayla Queenazima Santoso, Karunia Perjuangan Mustadl’afin, Lutfi Andriyanto, Igi Ardiyanto
Artikel ini berlisensi Creative Commons Attribution-NonCommercial 4.0 International License.