Desenvolvimento de Artefatos para Apoiar o Design e a Avaliação de Chatbots focando em Usabilidade e User Experience

Abstract

Chatbots are software that communicate and interact in a human-like way, through natural language, with human users. These tools are useful for a variety of purposes ranging from customer service, sales, education and learning, healthcare, to entertainment. In recent years, chatbots have become popular and there has been significant growth in the emergence of these tools on the market, especially textual chatbots. This rise has encouraged developers to create their own tools, in addition to attracting the efforts of researchers to this area. Despite the emphasis, there is a need to propose technologies capable of supporting developers and development teams in the process of designing and evaluating quality chatbots, since although these tools are on the rise, technologies to guarantee their quality and user satisfaction are not has kept pace. Given this, this master’s research proposes to develop artifacts applicable to the design and evaluation process of chatbots, based on quality attributes related to Usability and User Experience (UX), identified in literature reviews. The focus on Usability and UX was due to the impact that these aspects have on user satisfaction and the perceived quality of the system. The f irst artifact is the inspection checklist U2Chatbot, developed with the aim of assisting development teams in the process of identifying defects in text-based chatbots, improving the quality of these tools before their launch on the market. Despite the creation of checklist, it was observed that some developers have a visual need to understand important aspects that chatbots should display and that for this audience, only the U2Chatbot check items would not be enough. Thus, in a complementary way, the DP-U2Chatbot artifact was developed, which is a set of interface design recommendations, containing 21 important instructions to support developers in the process of building chatbots. After developing the proposed technologies, they were subjected to the necessary evaluations. To evaluate checklist U2Chatbot, an experimental study was conducted and the results indicated that participants found the technology useful for discovering defects in chatbots, however, ease of use could improve. Furthermore, it was found that the participants’ experience did not significantly influence the efficacy results and efficiency of the technique. For the DP-U2Chatbot design patterns, a simple assessment was carried out regarding the understandability and usefulness of the technology. The results indicated that in general the technology is easy to understand and useful in supporting the design of chatbots, however, the visual examples of some patterns could be improved to make them clearer to developers.