Estimation of Probabilistic and Time Characteristics of Human-Machine Dialogue in Natural Language
DOI:
https://doi.org/10.52575/2687-0932-2023-50-1-162-172Keywords:
natural language processing, dialogue system, question-answer system, estimation of probabilistic-temporal characteristics, human-machine dialogueAbstract
The results of a study aimed at improving the process of evaluating the characteristics of a human-machine dialogue carried out in natural language are presented. The relevance of the development of tools for estimating the probabilistic-temporal characteristics of a question-answer system from the point of view of achieving the goal of a human-machine dialogue, which is to satisfy a specific information need of the user, is substantiated. A model of the human-machine dialogue process based on the use of the mathematical apparatus of probabilistic-time graphs is presented. The developed model is focused on calculating the probability of achieving the goal of the dialogue and its average duration, taking into account the characteristics of the question-answer system and the values of indicators characterizing the user's characteristics. It is shown that the application of the model makes it possible to justify the choice of dialogue systems with specific characteristics and recommend them to certain user groups to meet information needs.
Downloads
References
Агузумцян Р.В., Великанова А.С., Польщиков К.А., Игитян Е.В., Лихошерстов Р.В. 2021. О применении интеллектуальных технологий обработки естественного языка и средств виртуальной реальности для поддержки принятия решений при подборе исполнителей проектов. Экономика. Информатика, 48(2): 392–404. DOI 10.52575/2687-0932-2021-48-2-392-404.
Махди Т.Н., Игитян Е.В., Польщиков К.А., Корсунов Н.И. 2022. Оценивание эффективности функционирования диалоговой системы на основе применения нечеткого вывода с нейросетевой настройкой. Экономика. Информатика, 49(2): 356–374. DOI 10.52575/2687-0932-2022-49-2-356-374.
Польщиков К.А., Лазарев С.А., Константинов И.С., Польщикова О.Н., Свойкина Л.Ф., Игитян Е.В., Балакшин М.С. 2020. Модель для оценки эффективности выполнения робототехнической системой коммуникативных функций. СТИН, 6: 4–7.
Польщиков К.А., Польщикова О.Н., Игитян Е.В., Балакшин М.С. 2019. Алгоритм поддержки принятия решений по выбору средств обработки больших массивов естественно-языковых данных. Научные ведомости Белгородского государственного университета. Серия: Экономика. Информатика. 46(3): 553–562. DOI 10.18413/2411-3808-2019-46-3-553-562.
Ai H., Litman D.J. 2008. Assessing Dialog System User Simulation Evaluation Measures Using Human Judges. Proceedings of ACL-08: 622–629.
Deriu J., Rodrigo A., Otegi A. 2021. Survey on evaluation methods for dialogue systems. Artificial Intelligence Review, 54: 755–810.
Dodge J., Gane A., Zhang X., Bordes A., Chopra S., Miller A., Szlam A., Weston J. 2016. Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems. arXiv. URL: https://doi.org/10.48550/arXiv.1511.06931 (accessed: 15.02.2023).
Dziri N., Rashkin H., Linzen T., Reitter D. 2022. Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark. Transactions of the Association for Computational Linguistics, 10: 1066–1083.
Ji T., Graham Y., Jones G.J.F., Lyu C., Liu Q. 2022. Achieving Reliable Human Assessment of Open-Domain Dialogue Systems. arXiv. URL: https://doi.org/10.48550/arXiv.2203.05899 (accessed: 15.02.2023).
Lоpez-Cоzar R., De la Torre A., Segura J.C., Rubio A.J. 2003. Assessment of dialogue systems by means of a new simulation technique. Speech Communication, 40(3): 387–407.
Polshchykov K., Lazarev S., Polshchykova O., Igityan E. 2019. The Algorithm for Decision-Making Supporting on the Selection of Processing Means for Big Arrays of Natural Language Data. Lobachevskii Journal of Mathematics, 40(11): 1831–1836.
Polshchykov K.A., Lazarev S.A., Konstantinov I.S., Polshchykova O.N., Svoikina L.F., Igityan E.V., Balakshin M.S. 2020. Assessing the Efficiency of Robot Communication. Russian Engineering Research, 40: 936–938.
Polshchykov K.A., Velikanova A.S., Igityan E.V. 2022. Neural network natural language processing tools for identifying personal priorities in the project performers selection in the field of smart agriculture. IOP Conference Series: Earth and Environmental Science, 1069: 012012.
Shah H., Warwick K., Vallverdú J., Wu D. 2016. Can machines talk? Comparison of Eliza with modern dialogue systems. Computers in Human Behavior, 58: 278–295.
Tseng B.-H., Dai Y., Kreyssig F., Byrne B. 2021. Transferable Dialogue Systems and User Simulators. arXiv. URL: https://doi.org/10.48550/arXiv.2107.11904 (accessed: 15.02.2023).
Velikanova A.S., Polshchykov K.A., Likhosherstov R.V., Polshchykova A.K. 2021. The use of virtual reality and fuzzy neural network tools to identify the focus on achieving project results. Journal of Physics: Conference Series. 2nd International Scientific Conference on Artificial Intelligence and Digital Technologies in Technical Systems 2021, Volgograd, 2060: 173707.
Wen T.-H., Gasic M., Mrksic N., Rojas-Barahona L.M., Su P.-H., Vandyke D., Young S. 2016. Multi-domain Neural Network Language Generation for Spoken Dialogue Systems. arXiv. URL: https://doi.org/10.48550/arXiv.1603.01232 (accessed: 15.02.2023).
Wen T.-H., Gasic M., Mrksic N., Su P.-H., Vandyke D., Young S. 2015. Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems. arXiv. URL: https://doi.org/10.48550/arXiv.1508.01745 (accessed: 15.02.2023).
Wen T.-H., Vandyke D., Mrksic N., Gasic M., Rojas-Barahona L.M., Su P.-H., Ultes S., Young S. 2017. A Network-based End-to-End Trainable Task-oriented Dialogue System. arXiv. URL: https://doi.org/10.48550/arXiv.1604.04562 (accessed: 15.02.2023).
Xiang J., Liu Y., Cai D., Li H., Lian D., Liu L. 2021. Assessing Dialogue Systems with Distribution Distances. arXiv. URL: https://doi.org/10.48550/arXiv.2105.02573 (accessed: 15.02.2023).
Abstract views: 47
Share
Published
How to Cite
Issue
Section
Copyright (c) 2023 Economics. Information Technologies
This work is licensed under a Creative Commons Attribution 4.0 International License.
