A Multi-Perspective Evaluation of Assistance Methods for Specifying Requirements for Reinforcement Learning
Volume 11, Issue 6, Page No 18–27, 2026
Adv. Sci. Technol. Eng. Syst. J. 11(6), 18–27 (2026);
DOI: 10.25046/aj110602
Keywords: Reinforcement Learning, State Transition Diagram, Requirement
This study proposes a method to assist in the creation of requirement documents for high-quality state transition diagrams, with the aim of simplifying the creation of reinforcement learning environments. It provides templates and precautions to assist inexperienced engineers in refining their requirements. These requirements can be semi-automatically converted into state transition diagrams. The effectiveness of various assistive elements is evaluated by comparing edit distances pre and post-modification. The method also applies the same assistance to LLMs to analyze differences from human performance. Experiments on specifying the card game "Old Maid" confirm the effectiveness of model requirements and precautions. The results reveal the effectiveness of model documents and precautions, but no significant effect with the precaution position. The results showed that model documents and writing precautions reduced textual edit distances, while the position of precautions had no statistically significant effect. LLM-generated requirements also showed relatively low edit distances, although their quality requires further evaluation.
- N. Kalra, S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transportation Research Part A, 94, 182–193, 2016, doi: 10.1016/j.tra.2016.09.010.
- A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, V. Koltun, “CARLA: An open urban driving simulator”, CoRL, 2017, doi: 10.48550/arXiv.1711.03938.
- H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, O. Beijbom, “nuScenes: A multimodal dataset for autonomous driving”, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11621–11631, 2020, doi: 10.1109/CVPR42600.2020.01164.
- F. Codevilla, E. Santana, A. Lopez, A. Gaidon, “Exploring the limitations of behavior cloning for autonomous driving”, in International Conference on Computer Vision (ICCV), 9328–9337, 2019, doi: 10.1109/ICCV.2019.00942.
- Y. Zhang, A. Carballo, H. Yang, K. Takeda, “Autonomous driving in adverse weather conditions: A survey”, arXiv preprint arXiv:2112.08936, 2021, doi: 10.1016/j.isprsjprs.2022.12.021.
- S. Zang, M. Ding, D. Smith, P. Tyler, T. Rakotoarivelo, M. A. Kaafar, “The impact of adverse weather conditions on autonomous vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car”, IEEE Vehicular Technology Magazine, 14, 103–111, 2019, doi: 10.1109/MVT.2019.2892497.
- D. Neumeister, D. Pape, “Automated vehicles and adverse weather: Final report”, U.S. Department of Transportation, Federal Highway Administration, June 2019. Available: www.its.dot.gov/index.htm
- R. Li, T. Qin, C. Widdershoven, “ISS-Scenario: Scenario-based testing in CARLA”, in Theoretical Aspects of Software Engineering (TASE), 279–286, 2024, doi: 10.1007/978-3-031-64626-3_16.
- M. Čávojský, E. Šlapak, M. Dopiriak, G. Bugar, J. Gazda, “3CSim: CARLA corner case simulation for control assessment in autonomous driving”, arXiv preprint arXiv:2409.10524, 2024, doi: 10.48550/arXiv.2409.10524.
- B. Osiński, P. Milos, A. Jakubowski, P. Zięcina, M. Martyniak, C. Galias, A. Breuer, S. Homoceanu, H. Michalewski, “CARLA real traffic scenarios – novel training ground and benchmark for autonomous driving”, arXiv preprint arXiv:2012.11329, 2020, doi: 10.48550/arXiv.2012.11329.
- D. J. Fremont, E. Kim, Y. V. Pant, S. A. Seshia, A. Acharya, X. Bruso, P. Wells, S. Lemke, Q. Lu, S. Mehta, “Formal scenario-based testing of autonomous vehicles: From simulation to the real world”, in International Conference on Intelligent Transportation (ITSC), 1–8, 2020, doi: 10.1109/ITSC45102.2020.9294368.
- N. Hanselmann, K. Renz, K. Chitta, A. Bhattacharyya, A. Geiger, “KING: Generating safety-critical driving scenarios for robust imitation via kinematics gradients”, in Proceedings of the European Conference on Computer Vision, 333–350, 2022, doi: 10.1007/978-3-031-19839-7_20.
- H.-S. Cho, Y.-J. Park, M. Park, J. Son, “Study on designing scenarios to evaluate adverse condition positioning for highly reliable autonomous driving”, The Transactions of the Korean Society of Automotive Engineers, 31, 1021–1037, 2023, doi: 10.7467/KSAE.2023.31.12.1021.
- Noureddine El Abid Amrani, Ezzrhari Fatima Ezzahra, Mohamed Youssfi, Sidi Mohamed Snineh, Omar Bouattane, "A New Technique to Accelerate the Learning Process in Agents based on Reinforcement Learning", Advances in Science, Technology and Engineering Systems Journal, vol. 7, no. 3, pp. 62–69, 2022. doi: 10.25046/aj070307
- Achmad Ichwan Yasir, Gede Putra Kusuma, "Deep Deterministic Policy Gradients for Optimizing Simulated PoA Blockchain Networks Based on Healthcare Data Characteristics", Advances in Science, Technology and Engineering Systems Journal, vol. 6, no. 1, pp. 757–764, 2021. doi: 10.25046/aj060183
- Sho Hirose, Daisuke Kitakoshi, Akihiro Yamashita, Kentarou Suzuki, Masato Suzuki, "A Study on Intelligent Dialogue Agent for Older Adults’ Preventive Care – Towards Development of a Comprehensive Preventive Care System", Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 6, pp. 09–21, 2020. doi: 10.25046/aj050602
- Wiem Zemzem, Ines Hosni, "A New Distributed Reinforcement Learning Approach for Multiagent Cooperation Using Team-mate Modeling and Joint Action Generalization", Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 2, pp. 01–12, 2020. doi: 10.25046/aj050201
- Hiroyuki Yamamoto, Tomohiro Hayashida, Ichiro Nishizaki, Shinya Sekizaki, "Hypervolume-Based Multi-Objective Reinforcement Learning: Interactive Approach", Advances in Science, Technology and Engineering Systems Journal, vol. 4, no. 1, pp. 93–100, 2019. doi: 10.25046/aj040110
- Khaled Slhoub, Marco Carvalho, "Towards Process Standardization for Requirements Analysis of Agent-Based Systems", Advances in Science, Technology and Engineering Systems Journal, vol. 3, no. 3, pp. 80–91, 2018. doi: 10.25046/aj030311
- Indranil Nath, "A Derived Metrics as a Measurement to Support Efficient Requirements Analysis and Release Management", Advances in Science, Technology and Engineering Systems Journal, vol. 2, no. 3, pp. 36–40, 2017. doi: 10.25046/aj020306