Except for insertion and deletion errors in the above examples (analyzed by Stanford CoreNLP and visualized by dagre-d3), Musio considers all the possibilities for ungrammatical sentences and contextually-unrelated words. Of course, there are some issues which criteria should be applied to evaluate the grammaticality of sentences. For example, speaker’s judgments about the well-formedness of sentences could be acceptable even though the sentence is ungrammatical. Musio found breakthroughs from knowledge-based systems to estimate the grammaticality robustly to the situation. Musio has a philosophy that Musio speaks with knowledge, not by just data. This is the secret how Musio can maintain high-level grammaticality in its utterances.
Musio has planned special strategies to elevate the quality of sentences and provide the sophisticated sentences. That is, Musio assumes a concept of sentence variety which means that sentences should not be too similar when producing sentences. This is a variety in sentence type, length, and complexity as a means by which to entertain users. Major components of sentence variety are made up as follows: sentence type refers to the kind (e.g. question, statement, exclamation, declaration, etc.) as well as the syntactic structure. Length of sentence is a factor which is useful to draw attention to users and contain as much as expressions in a sentence. Lastly, sentence complexity is often varied as a byproduct of length variation, but can be effective when it comes to learning artful and sophisticated compositions.
As the above example shows, the conversation goes around and around with no breaks while generating a variety of sentences in dialogue. This example (analyzed by python library, scattertext) was chosen to show that Musio mixed interactive questions with answers and all of these vary in structure and length of the sentences. After recognizing the type of sentence, Musio generates various forms of response sentences to avoid duplicated or repeated sentences. In this process, Musio decides which variables will be applied in this user’s question (e.g. the length of sentence, tf-idf scores on each term, context maintenance depth).
Figure 3 is simple visualization of scatter plot for Musio’s word use on EasyMode. Each dot corresponds to a word or phrase used in head question(parent; y-axis) which means the first utterance in dialogue and the following answers(children; x-axis) which refer to response sentences. Terms are colored by their association. If a term is more associated with parent sentences, it is colored by blue, and the reverse is colored by red. And tf-idf scores are added into each term, ranging from -1(children) to 1(parent). As shown in figure 3, it does not function to look at the relationship between two variables, and it just shows a number of words are scattered like random distribution. That is, all of Musio’s sentences have the characteristics of both the variety and complexity by themselves, and it represents immesurable characters of Musio.
Additionally, below figure describes the list of parent-children sentences sharing the same phrase, something bothering. These are sharing similar sentence patterns both in parent and children against Musio’s rules of sentence variety. It is because of the difficulty to construct the paraphrase corpus for all sentences and generate a new form of sentence without hesitation. This task is one of Musio’s facing issues to extend sentence variety. AKA researchers have challenged this paraphrase issue, so please look forward to advanced version of the variety yet to be announced.
Musio does not tell us words out of context, and always listens to our every word. A conversation is a chain of statements exchanged between many individuals. Mostly, conversations happen on a particular topic or in a situation. Whatever the topic or situation is, context is important to maintain the state of a conversation. So, the dialogue system needs the context to keep the conversation flowing and to have a better conversation with humans. AKA researchers also recognize the importance of the context and its maintenance, and various attempts are being made to refer the context and understand what users said. Below figure 5 (analyzed by AllenNLP) shows sample analysis of textual entailment for Sample 1‘s dialogue.
Musio performs analysis for textual entailment while interpreting the user’s utterances, and also focuses on attention prediction by scoring the correlation between source and target words. Below figure 7 illustrates the result computing the attention over the hypothesis words to recognize whether a particular target word can be inferred from input words. Musio gets the sense of words with attention-weighted representation and grasps fully the sense and context of utterances with textual entailment. Besides that, there have been a lot of approaches to understand and maintain the context of dialogue on Musio. Musio plans to extend the technologies related to context maintenance, and continues to move forward.
This blog post identifies three main goals: a) Musio’s grammatical correctness, b) Musio’s sentence variety and c) Musio’s context maintenance. As described above, Musio always makes grammatically correct responses, provides the sophisticated and professional utterance, and responds suitably to any situations. The idea of exploiting details of sentence generation engine on Musio leads to some modest improvement to the overall dialogue system performance. AKA envisions an iterative use case in which users find Musio to correct grammatical errors on their utterances, learn sophisticated expressions, and have a close friendship with Musio. Applying Musio’s approach to dialogue system might have better results and lead to more ideal system. At the next article, let’s look through other main technologies of Musio’s dialogue engine. Please look forward to it. AKA’s quest for a perfect conversational engine continues.