WEKO3
アイテム
Towards Human-Level Evaluation: Assessing the Potential of GPT-4 in Automated Evaluation and Feedback Generation on Japanese Essays
http://hdl.handle.net/10228/0002001157
http://hdl.handle.net/10228/000200115787184095-719c-450a-b1fa-f0b70da56f96
| 名前 / ファイル | ライセンス | アクション |
|---|---|---|
10443063.pdf (177 KB)
|
|
| Item type | 共通アイテムタイプ(1) | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 公開日 | 2025-01-24 | |||||||||||||||
| タイトル | ||||||||||||||||
| タイトル | Towards Human-Level Evaluation: Assessing the Potential of GPT-4 in Automated Evaluation and Feedback Generation on Japanese Essays | |||||||||||||||
| 言語 | en | |||||||||||||||
| 著者 |
Nakamoto, Sayaka
× Nakamoto, Sayaka
× Okamoto, Yoshiaki
× Nakakouchi, Takashi
× 嶋田, 和孝
WEKO
13734
|
|||||||||||||||
| 著作権関連情報 | ||||||||||||||||
| 権利情報 | Copyright (c) 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |||||||||||||||
| 抄録 | ||||||||||||||||
| 内容記述タイプ | Abstract | |||||||||||||||
| 内容記述 | In recent years, Automated Writing Evaluation (AWE) has been extensively researched within the field of AI in education. This paper explores generative AI, such as GPT-4, which has garnered significant attention for its ability to score essays and provide feedback to students. We designed prompts for GPT-4 to assign scores and rationales based on a given rubric and to generate feedback beneficial for students' development. We compared the evaluations produced by GPT-4 with those made by human evaluators. The results demonstrate GPT-4's potential to assist in generating evaluations at a human level. In addition, we analysed the consistency of the scoring and the quality of the rationales and feedback generated by GPT-4. In this paper, we will share our analysis and also describe the points that need to be improved for implementation in practice. | |||||||||||||||
| 言語 | en | |||||||||||||||
| 備考 | ||||||||||||||||
| 内容記述タイプ | Other | |||||||||||||||
| 内容記述 | 16th International Conference on Learning Technologies and Learning Environments (LTLE 2024), July 6-12, 2024, Takamatsu, Japan | |||||||||||||||
| 言語 | en | |||||||||||||||
| 書誌情報 |
en : 2024 16th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) 発行日 2024-07-06 |
|||||||||||||||
| 出版社 | ||||||||||||||||
| 出版者 | IEEE | |||||||||||||||
| キーワード | ||||||||||||||||
| 主題Scheme | Other | |||||||||||||||
| 主題 | Automated Writing Evaluation | |||||||||||||||
| キーワード | ||||||||||||||||
| 主題Scheme | Other | |||||||||||||||
| 主題 | Automated Essay Scoring | |||||||||||||||
| キーワード | ||||||||||||||||
| 主題Scheme | Other | |||||||||||||||
| 主題 | Japanese Short Essay | |||||||||||||||
| キーワード | ||||||||||||||||
| 主題Scheme | Other | |||||||||||||||
| 主題 | GPT-4 | |||||||||||||||
| キーワード | ||||||||||||||||
| 主題Scheme | Other | |||||||||||||||
| 主題 | LLM | |||||||||||||||
| 言語 | ||||||||||||||||
| 言語 | eng | |||||||||||||||
| 資源タイプ | ||||||||||||||||
| 資源タイプ識別子 | http://purl.org/coar/resource_type/c_6501 | |||||||||||||||
| 資源タイプ | journal article | |||||||||||||||
| 出版タイプ | ||||||||||||||||
| 出版タイプ | AM | |||||||||||||||
| 出版タイプResource | http://purl.org/coar/version/c_ab4af688f83e57aa | |||||||||||||||
| DOI | ||||||||||||||||
| 識別子タイプ | DOI | |||||||||||||||
| 関連識別子 | https://doi.org/10.1109/IIAI-AAI63651.2024.00039 | |||||||||||||||
| ISBN | ||||||||||||||||
| 識別子タイプ | ISBN | |||||||||||||||
| 関連識別子 | 979-8-3503-7790-3 | |||||||||||||||
| 会議記述 | ||||||||||||||||
| 会議名 | International Conference on Learning Technologies and Learning Environments | |||||||||||||||
| 回次 | 16 | |||||||||||||||
| 開始年 | 2024 | |||||||||||||||
| 開始月 | 07 | |||||||||||||||
| 開始日 | 06 | |||||||||||||||
| 終了年 | 2024 | |||||||||||||||
| 終了月 | 07 | |||||||||||||||
| 終了日 | 12 | |||||||||||||||
| 開催国 | JPN | |||||||||||||||
| 査読の有無 | ||||||||||||||||
| 値 | yes | |||||||||||||||
| 研究者情報 | ||||||||||||||||
| URL | https://hyokadb02.jimu.kyutech.ac.jp/html/196_ja.html | |||||||||||||||
| 論文ID（連携） | ||||||||||||||||
| 値 | 10443063 | |||||||||||||||
| 連携ID | ||||||||||||||||
| 値 | 12443 | |||||||||||||||
